1 |
\chapter{Slab Allocator} |
\chapter{Slab Allocator} |
2 |
|
|
3 |
The majority of memory allocation requests in the kernel are for small, frequently used data structures. For this purpose the slab allocator is perfect. |
The majority of memory allocation requests in the kernel are for small, |
4 |
The basic idea behind a slab allocator is to have lists of commonly used objects available packed into pages. This avoids the overhead of allocating and destroying commonly used types of objects such as inode\_caches, dentry\_caches or vm\_area\_structs while using memory more efficiently. The slab allocator used by linux is the same as the one outlined in Bonwick's~\cite{slab} paper. Some terminology: |
frequently used data structures. For this purpose the slab allocator |
5 |
|
is perfect. The basic idea behind a slab allocator is to have lists |
6 |
|
of commonly used objects available packed into pages. This avoids the |
7 |
|
overhead of allocating and destroying commonly used types of objects such |
8 |
|
as inode\_caches, dentry\_caches or vm\_area\_structs while using memory |
9 |
|
more efficiently. The slab allocator used by linux is the same as the one |
10 |
|
outlined in Bonwick's~\cite{slab} paper. Some terminology: |
11 |
|
|
12 |
\begin{description} |
\begin{description} |
13 |
\item[cache] It is a store of recently used objects of the same type. In the slab allocator, it is the highest logical unit of storage. It has a human parseable name like dentry\_cache etc. |
\item[cache] It is a store of recently used objects of the same type. In the slab allocator, it is the highest logical unit of storage. It has a human parseable name like dentry\_cache etc. |
15 |
\item[object] This is the smallest unit. It resides on the slab and would be something like a single dentry. |
\item[object] This is the smallest unit. It resides on the slab and would be something like a single dentry. |
16 |
\end{description} |
\end{description} |
17 |
|
|
18 |
The objective is that a single page can now be used to contain a number |
The objective is that a single page can now be used to contain a number of |
19 |
of objects thus saving memory and avoiding internal fragmentation. |
objects thus saving memory and avoiding internal fragmentation. The slabs |
20 |
The slabs are organised into three types, |
are organised into three types, full slabs, partial slabs and empty ones. |
21 |
full slabs, partial slabs and empty ones. Partial slabs are used if |
Partial slabs are used if available to avoid fragmentation. To see all |
22 |
available to avoid fragmentation. To see all information on caches and |
information on caches and slabs available in a system, type {\bf cat |
23 |
slabs available in a system, type {\bf cat /proc/slabinfo} to see a list. |
/proc/slabinfo} to see a list. The fields correspond to: |
|
The fields correspond to: |
|
24 |
|
|
25 |
\vspace{15pt} |
\vspace{15pt} |
26 |
\noindent \begin{tabular}{ll} |
\noindent \begin{tabular}{ll} |
47 |
Further statistics appear if CONFIG\_DEBUG\_SLAB is set during make config |
Further statistics appear if CONFIG\_DEBUG\_SLAB is set during make config |
48 |
but these are essentially bean counters and not particularly interesting. |
but these are essentially bean counters and not particularly interesting. |
49 |
|
|
50 |
When SMP is enabled, each cache allocates a small array of objects for each |
When SMP is enabled, each cache allocates a small array of objects for each CPU |
51 |
CPU available. The reasoning behind having per-CPU slab-caches is that accessing data |
available. The reasoning behind having per-CPU slab-caches is that accessing |
52 |
global to all CPUs requires a number of spinlocks to be held (so as to avoid |
data global to all CPUs requires a number of spinlocks to be held (so as to |
53 |
race-conditions) which is expensive. Also, having per-CPU data like this |
avoid race-conditions) which is expensive. Also, having per-CPU data like |
54 |
helps in bringing down the number of |
this helps in bringing down the number of hardware cache-coherency issues: |
55 |
hardware cache-coherency issues: if more than one CPU references some |
if more than one CPU references some particular piece of data (and therefore |
56 |
particular piece of data (and therefore has it in its on-chip CPU cache) the |
has it in its on-chip CPU cache) the SMP hardware has to worry about keeping |
57 |
SMP hardware has to worry about keeping those CPU caches synchronised between |
those CPU caches synchronised between them. |
58 |
them. |
|
59 |
Hence, each cache has a short per-cpu array called {\bf cpudata} of type |
Hence, each cache has a short per-cpu array called {\bf cpudata} of type |
60 |
{\bf cpucache\_t}. This is a very simple struct with only two members: |
{\bf cpucache\_t}. This is a very simple struct with only two members: |
61 |
|
|
62 |
\vspace{15pt} |
\vspace{15pt} |
71 |
overflows. Once it overflows, half of the entries are placed in a global |
overflows. Once it overflows, half of the entries are placed in a global |
72 |
cache minimising the amount of spinlock operations required. |
cache minimising the amount of spinlock operations required. |
73 |
|
|
|
|
|
|
|
|
74 |
\newpage |
\newpage |
75 |
\section{Cache Structure} |
\section{Cache Structure} |
76 |
|
|
117 |
|
|
118 |
\vspace{10pt} |
\vspace{10pt} |
119 |
|
|
120 |
To ensure that callers of \textit{kmem\_cache\_create} don't use the wrong |
To ensure that callers of \textit{kmem\_cache\_create} don't use the |
121 |
flags, the bitmask is compared against a CREATE\_MASK defined in \textit{slab.c}. |
wrong flags, the bitmask is compared against a CREATE\_MASK defined in |
122 |
CREATE\_MASK consists of all the legal flags that can be used |
\textit{slab.c}. CREATE\_MASK consists of all the legal flags that can be |
123 |
when creating a cache. If an illegal flag is used, BUG() is invoked. |
used when creating a cache. If an illegal flag is used, BUG() is invoked. |
124 |
|
|
125 |
\subsection{Slab structure} |
\subsection{Slab structure} |
126 |
|
|
127 |
As mentioned, a slab consists of one or more pages assigned to contain objects. |
As mentioned, a slab consists of one or more pages assigned to contain objects. |
128 |
The job of this struct is to manage the objects in the slab. The |
The job of this struct is to manage the objects in the slab. The struct to |
129 |
struct to describe a slab is simple: |
describe a slab is simple: |
130 |
|
|
131 |
\subsubsection{struct slab\_s}\index{struct slab\_s} |
\subsubsection{struct slab\_s}\index{struct slab\_s} |
132 |
\begin{verbatim} |
\begin{verbatim} |
147 |
\item[free] Used for linking free objects together. |
\item[free] Used for linking free objects together. |
148 |
\end{description} |
\end{description} |
149 |
|
|
150 |
The slab\_t struct has to be stored somewhere. It can be either |
The slab\_t struct has to be stored somewhere. It can be either stored |
151 |
stored off slab in which case the memory will be allocated from one of the |
off slab in which case the memory will be allocated from one of the sizes |
152 |
sizes caches. Else it will be stored within the slab itself. |
caches. Else it will be stored within the slab itself. The sizes caches are |
153 |
|
described in a later section dealing with kmalloc. They are caches which |
154 |
|
store blocks of memory of sizes that are powers of two. |
155 |
|
|
156 |
\subsection{Overall Structure} |
\subsection{Overall Structure} |
157 |
|
|
|
|
|
158 |
\begin{figure} |
\begin{figure} |
159 |
\img{slab.png}{slab} |
\img{slab.png}{slab} |
160 |
\caption{Relationship between cache and slab descriptors} |
\caption{Relationship between cache and slab descriptors} |
161 |
\label{fig:slab1} |
\label{fig:slab1} |
162 |
\end{figure} |
\end{figure} |
163 |
|
|
164 |
Caches are linked together with the \textit{next} field. Each cache |
Caches are linked together with the \textit{next} field. Each cache consists |
165 |
consists of one or more slabs which are blocks of memory of |
of one or more slabs which are blocks of memory of one or more pages. Each |
166 |
one or more pages. Each slab contains multiple numbers of |
slab contains multiple numbers of objects, possibly with gaps between |
167 |
objects, possibly with gaps between them so that they hit different cache |
them so that they hit different cache lines. If, during cache creation, |
168 |
lines. The slab\_t or slab management structure may be kept on |
the flag SLAB\_HWCACHE\_ALIGN is specified, the objsize is adjusted up to |
169 |
the slab or off it. If on the slab, it is at the beginning. If off-cache, |
L1\_CACHE\_BYTES so that the objects will be cache aligned. This will create |
170 |
|
the gaps between objects. The slab\_t or slab management structure may be kept |
171 |
|
on the slab or off it. If on the slab, it is at the beginning. If off-cache, |
172 |
it is stored in an appropriately sized memory cache. |
it is stored in an appropriately sized memory cache. |
173 |
|
|
174 |
\begin{verbatim} |
\begin{verbatim} |
212 |
|
|
213 |
\end{verbatim} |
\end{verbatim} |
214 |
|
|
215 |
\sloppypar The \texttt{struct page}`s \textit{list} element is used to track where cache\_t and slab\_t are stored (see kmem\_cache\_grow). The list-$>$next pointer points to kmem\_cache\_t (the cache it belongs to) and list-$>$prev points to slab\_t (the slab it is part of). So given an object, we can easily find the associated cache and slab through these pointers. |
\sloppypar The \texttt{struct page}`s \textit{list} element is used to |
216 |
|
track where cache\_t and slab\_t are stored (see kmem\_cache\_grow). The |
217 |
|
list-$>$next pointer points to kmem\_cache\_t (the cache it belongs to) and |
218 |
|
list-$>$prev points to slab\_t (the slab it is part of). So given an object, |
219 |
|
we can easily find the associated cache and slab through these pointers. |
220 |
|
|
221 |
\section{Initialisation} |
\section{Initialisation} |
222 |
|
|
230 |
\item Sets the cache\_cache colour |
\item Sets the cache\_cache colour |
231 |
\end{itemize} |
\end{itemize} |
232 |
|
|
233 |
The term \emph{cache chain} is simply a fancy name for a circular linked |
The term \emph{cache chain} is simply a fancy name for a circular linked list |
234 |
list of caches the slab allocator knows about. It then goes on to |
of caches the slab allocator knows about. It then goes on to initialise |
235 |
initialise a cache of caches called {\bf kmem\_cache}. This is a cache |
a cache of caches called {\bf kmem\_cache}. This is a cache of objects of |
236 |
of objects of type {\bf kmem\_cache\_t} which describes information about |
type {\bf kmem\_cache\_t} which describes information about the cache itself. |
|
the cache itself. |
|
237 |
|
|
238 |
\subsection{Initialising cache\_cache} |
\subsection{Initialising cache\_cache} |
239 |
|
|
265 |
|
|
266 |
\subsection{Initialising cache\_sizes} |
\subsection{Initialising cache\_sizes} |
267 |
|
|
268 |
\emph{kmem\_cache\_sizes\_init()} is called to create a set of caches |
\emph{kmem\_cache\_sizes\_init()} is called to create a set of caches of |
269 |
of different sizes. On a system with a page size of 4096, the smallest |
different sizes. On a system with a page size of 4096, the smallest chunk |
270 |
chunk is 32 bytes, otherwise it is 64 bytes. |
is 32 bytes, otherwise it is 64 bytes. Two caches will be created for every |
271 |
Two caches will be created for every size, both of them cacheline-aligned, |
size, both of them cacheline-aligned, and one suitable for ISA DMA. So the |
272 |
and one suitable for ISA DMA. |
smallest caches of memory are called {\emph size-32} and {\emph size-32(DMA)}. |
273 |
So the smallest caches of |
Caches for each subsequent power of two will be created until two caches of |
274 |
memory are called {\emph size-32} and {\emph size-32(DMA)}. Caches for |
size of 131072 bytes are created. These will be used by \emph{kmalloc} later. |
|
each subsequent power of two will be created until two caches of size of |
|
|
131072 bytes are created. These will be used by \emph{kmalloc} later. |
|
275 |
Refer to section~\ref{fun:kcsi} for the implementation details. |
Refer to section~\ref{fun:kcsi} for the implementation details. |
276 |
|
|
|
|
|
|
|
|
|
|
|
277 |
\section{Initialising Objects} |
\section{Initialising Objects} |
278 |
\subsection{Function kmem\_cache\_init\_objs()} |
\subsection{Function kmem\_cache\_init\_objs()} |
279 |
\textit{File: }\url{mm/slab.c} |
\textit{File: }\url{mm/slab.c} |
293 |
\end{verbatim} |
\end{verbatim} |
294 |
|
|
295 |
This steps through the number of objects that can be contained onslab. |
This steps through the number of objects that can be contained onslab. |
296 |
(cachep-$>$objsize * i) will give an offset from s\_mem where \textit{i}th object is. [note: s\_mem is used to point to the first object]. |
(cachep-$>$objsize * i) will give an offset from s\_mem where \textit{i}th |
297 |
|
object is. [note: s\_mem is used to point to the first object]. |
298 |
|
|
299 |
\begin{verbatim} |
\begin{verbatim} |
300 |
#if DEBUG |
#if DEBUG |
307 |
#endif |
#endif |
308 |
\end{verbatim} |
\end{verbatim} |
309 |
|
|
310 |
If debugging is enabled, RED\_MAGIC1 will be written at the beginning and end of |
If debugging is enabled, RED\_MAGIC1 will be written at the beginning and |
311 |
the object. Later when the object is used, this will be checked again. If the |
end of the object. Later when the object is used, this will be checked |
312 |
values are not still RED\_MAGIC1, it's known that the object was activated twice |
again. If the values are not still RED\_MAGIC1, it's known that the object |
313 |
or else was overrun. |
was activated twice or else was overrun. |
314 |
|
|
315 |
\begin{verbatim} |
\begin{verbatim} |
316 |
if (cachep->ctor) |
if (cachep->ctor) |
317 |
cachep->ctor(objp, cachep, ctor_flags); |
cachep->ctor(objp, cachep, ctor_flags); |
318 |
\end{verbatim} |
\end{verbatim} |
319 |
|
|
320 |
A constructor is called for the object if available. Users are warned that a |
A constructor is called for the object if available. Users are warned that |
321 |
cache with a constructor can not allocate memory from itself because it would |
a cache with a constructor can not allocate memory from itself because it |
322 |
end up recursively calling this. |
would end up recursively calling this. |
323 |
|
|
324 |
\begin{verbatim} |
\begin{verbatim} |
325 |
#if DEBUG |
#if DEBUG |
327 |
objp -= BYTES_PER_WORD; |
objp -= BYTES_PER_WORD; |
328 |
\end{verbatim} |
\end{verbatim} |
329 |
|
|
330 |
Next block of debugging code will adjust the address of objp to take into |
This block of debugging code will adjust the address of objp to take into |
331 |
account the size of RED\_MAGIC1 |
account the size of RED\_MAGIC1 that was added before calling the constructor. |
332 |
|
The constructor receives a pointer to the actual data block and not the |
333 |
|
debugging marker. |
334 |
|
|
335 |
\begin{verbatim} |
\begin{verbatim} |
336 |
if (cachep->flags & SLAB_POISON) |
if (cachep->flags & SLAB_POISON) |
370 |
Mark the end of the slab with BUFCTL\_END. free is set to 0 so that the first |
Mark the end of the slab with BUFCTL\_END. free is set to 0 so that the first |
371 |
object allocated will be the first object on the slab. |
object allocated will be the first object on the slab. |
372 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
373 |
\section{Allocating Objects} |
\section{Allocating Objects} |
374 |
|
|
375 |
\emph{kmem\_cache\_alloc()} is badly named as it doesn't allocate a new |
\emph{kmem\_cache\_alloc()} is badly named as it doesn't allocate a new cache, |
376 |
cache, it allocates a new object. Creating a new cache will be dealt |
it allocates a new object. Creating a new cache will be dealt with later |
377 |
with later as creating of a cache depends on being able to allocate a |
as creating of a cache depends on being able to allocate a kmem\_cache first. |
378 |
kmem\_cache first. |
|
379 |
\subsection{Function \_\_kmem\_cache\_alloc()} |
\subsection{Function \_\_kmem\_cache\_alloc()} |
380 |
\textit{File: }\url{mm/slab.c} |
\textit{File: }\url{mm/slab.c} |
381 |
\textit{Prototype: } |
\textit{Prototype: } |
382 |
\begin{verbatim} |
\begin{verbatim} |
383 |
void * __kmem_cache_alloc (kmem_cache_t *cachep, |
void * __kmem_cache_alloc (kmem_cache_t *cachep, |
384 |
int flags) |
int flags) |
385 |
\end{verbatim} |
\end{verbatim} |
386 |
|
|
387 |
\noindent The function takes two parameters: |
\noindent The function takes two parameters: |
388 |
|
|
393 |
\end{tabularx} |
\end{tabularx} |
394 |
|
|
395 |
\vspace{10pt} |
\vspace{10pt} |
396 |
\noindent The flags can be one of: |
|
397 |
|
\noindent The flags are defined in \emph{include/linux/slab.h} and correspond |
398 |
|
to GFP page flag options, mainly of important to the allocator. Callers |
399 |
|
sometimes call with either SLAB\_ or GFP\_ flags. This section will only |
400 |
|
deal with the SLAB\_ flags and what they mean. They can be one of: |
401 |
|
|
402 |
\vspace{10pt} |
\vspace{10pt} |
403 |
\begin{tabularx}{15cm}{lX} |
\begin{tabularx}{15cm}{lX} |
404 |
GFP\_USER & Allocate memory on behalf of user. May sleep \\ |
|
405 |
GFP\_KERNEL & Allocate normal kernel ram. May sleep \\ |
SLAB\_NOFS & This flag tells the page free logic to not make any |
406 |
GFP\_ATOMIC & Allocation will not sleep \\ |
calls to the filesystem layer. This is important for the |
407 |
|
allocation of buffer heads for instance where it is important |
408 |
|
the filesystem does not end up recursively calling itself \\ |
409 |
|
|
410 |
|
SLAB\_NOIO & Do not start any IO. For example, in |
411 |
|
\texttt{try\_to\_free\_buffers}, no attempt to write out |
412 |
|
busy buffer pages will be made if this slab flag is used \\ |
413 |
|
|
414 |
|
SLAB\_NOHIGHIO & Treated the same as SLAB\_NOIO according to buffer.c \\ |
415 |
|
|
416 |
|
SLAB\_ATOMIC & Allocations made with this flag may take whatever measures |
417 |
|
necessary to get a page without sleeping. This is used |
418 |
|
for the buffer head emergency pool for instance. The page |
419 |
|
allocator will not sleep when this flag is set \\ |
420 |
|
|
421 |
|
SLAB\_USER & This translates to say that the allocator may sleep, make FS |
422 |
|
calls and engage in IO. In reality, the flag does not appear |
423 |
|
to be used anywhere in the code and is probably included to |
424 |
|
have a nice one to one mapping to the GFP\_ flags\\ |
425 |
|
|
426 |
|
SLAB\_KERNEL & Used when the caller just wants the object to be allocated |
427 |
|
and are not particular about what needs to be done to get |
428 |
|
it. The caller will perform IO, sleep and can make calls to |
429 |
|
teh filesystem \\ |
430 |
|
|
431 |
|
SLAB\_NFS & Supplied to provide a mapping to GFP\_NFS. In reality, it is |
432 |
|
never used. The only caller that needs it uses GFP\_NFS |
433 |
|
directly \\ |
434 |
|
|
435 |
|
SLAB\_DMA & Used to flag a cache that is the should allocate memory |
436 |
|
suitable for use with DMA. This will make the allocation from |
437 |
|
the sizes cache dealing with DMA and if the page allocator |
438 |
|
is used, it'll only allocate from ZONE\_DMA \\ |
439 |
|
\end{tabularx} |
440 |
|
|
441 |
|
For completness, there is two other SLAB flags which exist. they are |
442 |
|
|
443 |
|
\begin{tabularx}{15cm}{lX} |
444 |
|
|
445 |
|
SLAB\_LEVEL\_MASK & This rarely used mask removes any bits from the flags |
446 |
|
which the slab allocator is not aware of \\ |
447 |
|
|
448 |
|
SLAB\_NO\_GROW & This flags a cache that the number of slabs within it should |
449 |
|
not grow. It only appears to be used by |
450 |
|
\texttt{kmem\_cache\_grow} but does not appear to be set |
451 |
|
anywhere in the code \\ |
452 |
\end{tabularx} |
\end{tabularx} |
453 |
|
|
454 |
\vspace{10pt} |
\vspace{10pt} |
455 |
|
|
456 |
They largely affect how the buddy allocator will behave later. |
They largely affect how the buddy allocator will behave later. |
457 |
\emph{kmem\_cache\_alloc} calls \emph{\_\_kmem\_cache\_alloc} directly. |
\emph{kmem\_cache\_alloc} calls \emph{\_\_kmem\_cache\_alloc} directly. |
458 |
It comes in two flavors, UP and SMP. |
It comes in two flavors, UP and SMP. |
468 |
kmem_cache_alloc_head(cachep, flags); |
kmem_cache_alloc_head(cachep, flags); |
469 |
|
|
470 |
\end{verbatim} |
\end{verbatim} |
471 |
\emph{kmem\_cache\_alloc\_head()} is named strangely. It asserts that the wrong |
|
472 |
combination of SLAB\_DMA and GFP\_DMA are not used with the flags. |
\emph{kmem\_cache\_alloc\_head()} is a simple sanity check. It asserts that |
473 |
|
the wrong combination of SLAB\_DMA and GFP\_DMA are not used with the flags. |
474 |
|
|
475 |
\begin{verbatim} |
\begin{verbatim} |
476 |
try_again: |
try_again: |
477 |
local_irq_save(save_flags); |
local_irq_save(save_flags); |
481 |
return objp; |
return objp; |
482 |
|
|
483 |
\end{verbatim} |
\end{verbatim} |
484 |
The macro \texttt{kmem\_cache\_alloc\_one} which has been describer in section~\ref{mac:kcao} allocates an object if there is a partially allocated or completely free slab available. |
|
485 |
|
The macro \texttt{kmem\_cache\_alloc\_one} which will be described in |
486 |
|
section~\ref{mac:kcao} allocates an object if there is a partially allocated |
487 |
|
or completely free slab available. \texttt{local\_irq\_save} will disables |
488 |
|
interrupts and saves the flags. This will guarantee synchronisation which |
489 |
|
is needed for \texttt{kmem\_cache\_alloc\_one}. A spinlock can not be used |
490 |
|
because an interrupt handler can not take out a spinlock and an interrupt |
491 |
|
handler can call this function. |
492 |
|
|
493 |
\begin{verbatim} |
\begin{verbatim} |
494 |
alloc_new_slab: |
alloc_new_slab: |
495 |
local_irq_restore(save_flags); |
local_irq_restore(save_flags); |
502 |
return NULL; |
return NULL; |
503 |
} |
} |
504 |
\end{verbatim} |
\end{verbatim} |
|
Note the label alloc\_new\_slab which has no goto apparently, is used in \texttt{kmem\_cache\_alloc\_one}. We come here if there are no free or partially free slabs available. So we grow the cache by one more slab and try again. |
|
505 |
|
|
506 |
|
Note the label alloc\_new\_slab which has no goto apparently, is used in |
507 |
|
\texttt{kmem\_cache\_alloc\_one}. We come here if there are no free or |
508 |
|
partially free slabs available. So we grow the cache by one more slab and |
509 |
|
try again. |
510 |
|
|
511 |
\subsubsection{Allocation on SMP} |
\subsubsection{Allocation on SMP} |
512 |
|
|
513 |
There are two principle differences between allocations on a UP and on SMP. The |
There are two principle differences between allocations on UP and on SMP. The |
514 |
first one is the use of spinlocks, they become necessary for SMP. The second |
first one is the use of spinlocks, they become necessary for SMP. The |
515 |
is that slabs and objects are bound to processors for better use of cache. |
second is that slabs and objects are bound to processors for better use |
516 |
We'll see how this is achieved. First, this is what \_\_kmem\_cache\_alloc |
of hardware cache. We'll see how this is achieved. First, this is what |
517 |
looks like for the SMP case. |
\_\_kmem\_cache\_alloc looks like for the SMP case. |
518 |
|
|
519 |
Most of this is the same as for the UP case so we'll only deal with the SMP related code. |
Most of this is the same as for the UP case so we'll only deal with the SMP |
520 |
|
related code. |
521 |
|
|
522 |
\begin{verbatim} |
\begin{verbatim} |
523 |
void * __kmem_cache_alloc (kmem_cache_t *cachep, int flags) |
void * __kmem_cache_alloc (kmem_cache_t *cachep, int flags) |
535 |
\end{verbatim} |
\end{verbatim} |
536 |
|
|
537 |
cc\_data is a macro which returns the cpucache\_s struct for this CPU. The |
cc\_data is a macro which returns the cpucache\_s struct for this CPU. The |
538 |
struct has two members \texttt{avail} and \texttt{limit}. \texttt{avail} |
struct has two members \texttt{avail} and \texttt{limit}. \texttt{avail} |
539 |
is how many objects are available and \texttt{limit} is the maximum |
is how many objects are available and \texttt{limit} is the maximum number |
540 |
number that this processor may have. |
that this processor may have. |
541 |
|
|
542 |
\begin{verbatim} |
\begin{verbatim} |
543 |
|
|
548 |
\end{verbatim} |
\end{verbatim} |
549 |
|
|
550 |
If the cpucache\_t data is available, check to see if there is an object |
If the cpucache\_t data is available, check to see if there is an object |
551 |
available. If there is, allocate it. From the cc\_entry macro, it would appear |
available. If there is, allocate it. From the cc\_entry macro, it would |
552 |
that the objects are stored in memory after the cpucache\_t . |
appear that the objects are stored in memory after the cpucache\_t . |
553 |
|
|
554 |
\begin{verbatim} |
\begin{verbatim} |
555 |
|
|
561 |
} |
} |
562 |
\end{verbatim} |
\end{verbatim} |
563 |
|
|
564 |
Else, there isn't an object available from the cache so more have to be |
Else, there isn't an object available from the cache so more have to |
565 |
allocated. The function \texttt{kmem\_cache\_alloc\_batch()} will be discussed in detail in section~\ref{fun:kcab}. |
be allocated. The function \texttt{kmem\_cache\_alloc\_batch()} will be |
566 |
|
discussed in detail in section~\ref{fun:kcab}. |
567 |
|
|
568 |
\begin{verbatim} |
\begin{verbatim} |
569 |
|
|
575 |
} |
} |
576 |
\end{verbatim} |
\end{verbatim} |
577 |
|
|
578 |
If a cpucache is not availble, just allocate one object in the same way a UP |
If a cpucache is not availiable, just allocate one object in the same way |
579 |
does it. |
a UP does it except that a spinlock is held. |
580 |
|
|
581 |
\begin{verbatim} |
\begin{verbatim} |
582 |
|
|
583 |
local_irq_restore(save_flags); |
local_irq_restore(save_flags); |
646 |
void * kmem_cache_alloc_one_tail (kmem_cache_t *cachep, |
void * kmem_cache_alloc_one_tail (kmem_cache_t *cachep, |
647 |
slab_t *slabp) |
slab_t *slabp) |
648 |
\end{verbatim} |
\end{verbatim} |
649 |
|
|
650 |
Once a slab is found that can be used, \emph{kmem\_cache\_alloc\_one\_tail()} |
Once a slab is found that can be used, \emph{kmem\_cache\_alloc\_one\_tail()} |
651 |
is called. The main complexity in this funcion is in the debugging so lets |
is called. The main complexity in this funcion is in the debugging so lets |
652 |
examine it in pieces: |
examine it in pieces: |
668 |
slabp->free=slab_bufctl(slabp)[slabp->free]; |
slabp->free=slab_bufctl(slabp)[slabp->free]; |
669 |
\end{verbatim} |
\end{verbatim} |
670 |
|
|
671 |
s\_mem is the pointer to the beginning of the objects within the slab |
s\_mem is the pointer to the beginning of the objects within the slab and free |
672 |
and free is the index of the first object on the slab's |
is the index of the first object on the slab's freelist. Multiplying it by the |
673 |
freelist. Multiplying it by the size of each object will |
size of each object will make objp the address of a free object. slab\_bufctl |
674 |
make objp the address of a free object. slab\_bufctl is a macro which casts |
is a macro which casts kmem\_bufctl\_t to slab\_t and adds 1 to it effectively |
675 |
kmem\_bufctl\_t to slab\_t and adds 1 to it effectively giving the address of |
giving the address of the next free object. |
|
the next free object. |
|
676 |
|
|
677 |
Without debugging, the objp would be returned as is, but with debugging |
Without debugging, the objp would be returned as is, but with debugging |
678 |
enabled more work is done. |
enabled more work is done. |
714 |
|
|
715 |
Return the object which has been allocated. |
Return the object which has been allocated. |
716 |
|
|
|
|
|
|
|
|
|
|
|
717 |
\subsection{Function kmem\_cache\_alloc\_batch()}\label{fun:kcab} |
\subsection{Function kmem\_cache\_alloc\_batch()}\label{fun:kcab} |
718 |
\textit{File: }\url{mm/slab.c} |
\textit{File: }\url{mm/slab.c} |
719 |
\textit{Prototype: } |
\textit{Prototype: } |
722 |
cpucache_t* cc, |
cpucache_t* cc, |
723 |
int flags) |
int flags) |
724 |
\end{verbatim} |
\end{verbatim} |
725 |
kmem\_cache\_alloc\_batch() is very simple. It allocates batchcount number of |
|
726 |
new objects and places each of them on the cpucache to be used for later |
kmem\_cache\_alloc\_batch() is very simple. It allocates batchcount number |
727 |
|
of new objects and places each of them on the cpucache to be used for later |
728 |
allocations. This leads to better cache utilization. |
allocations. This leads to better cache utilization. |
729 |
|
|
730 |
\begin{verbatim} |
\begin{verbatim} |
760 |
break; |
break; |
761 |
\end{verbatim} |
\end{verbatim} |
762 |
|
|
763 |
If there isn't a free one, break which will either return an object that has |
If there isn't a free one, break which will either return an object that |
764 |
been allocated or else return NULL which will grow the cache. |
has been allocated or else return NULL which will grow the cache. |
765 |
|
|
766 |
\begin{verbatim} |
\begin{verbatim} |
767 |
list_del(entry); |
list_del(entry); |
790 |
return NULL; |
return NULL; |
791 |
\end{verbatim} |
\end{verbatim} |
792 |
|
|
793 |
Free the spinlock and return an object if possible. Otherwise return NULL to |
Free the spinlock and return an object if possible. Otherwise return NULL |
794 |
the cache can be grown. |
to the cache can be grown. |
|
|
|
|
|
|
|
|
|
795 |
|
|
796 |
\section{Creating a Cache} |
\section{Creating a Cache} |
797 |
\subsection{Function kmem\_cache\_create()}\index{kmem\_cache\_create()} |
\subsection{Function kmem\_cache\_create()}\index{kmem\_cache\_create()} |
807 |
void (*dtor)(void*, kmem_cache_t *, unsigned long)) |
void (*dtor)(void*, kmem_cache_t *, unsigned long)) |
808 |
\end{verbatim} |
\end{verbatim} |
809 |
|
|
810 |
This function is responsible for creating new caches and adding |
This function is responsible for creating new caches and adding them to |
811 |
them to the cache chain. For clarity, |
the cache chain. For clarity, debugging information and sanity checks will |
812 |
debugging information and sanity checks will be ignored as they are only |
be ignored as they are only important during development and secondary to |
813 |
important during development and secondary to the slab allocator itself. |
the slab allocator itself. The only check that is important is the check |
814 |
The only check that is important is the check of flags against the |
of flags against the CREATE\_MASK as the caller may request flags that are |
815 |
CREATE\_MASK as the caller may request flags that are simply not available. |
simply not available. |
816 |
|
|
817 |
The arguements to kmem\_cache\_create are as follows |
The arguements to kmem\_cache\_create are as follows |
818 |
|
|
828 |
\vspace{10pt} |
\vspace{10pt} |
829 |
|
|
830 |
The whole beginning of the function is all debugging checks similar to what |
The whole beginning of the function is all debugging checks similar to what |
831 |
has been dealt with to date, so we'll start with the last sanity check that is |
has been dealt with to date, so we'll start with the last sanity check that |
832 |
made so that you can see where we are starting from |
is made so that you can see where we are starting from |
833 |
|
|
834 |
\begin{verbatim} |
\begin{verbatim} |
835 |
/* |
/* |
850 |
memset(cachep, 0, sizeof(kmem_cache_t)); |
memset(cachep, 0, sizeof(kmem_cache_t)); |
851 |
\end{verbatim} |
\end{verbatim} |
852 |
|
|
853 |
Request a kmem\_cache\_t from the cache\_cache. Remember this is a cache of |
Request a kmem\_cache\_t from the cache\_cache. Remember this is a cache |
854 |
cache descriptors. It's not a catch 22 problem as the cache\_cache is |
of cache descriptors. It's not a catch 22 problem as the cache\_cache is |
855 |
statically initialised. |
statically initialised. |
856 |
|
|
857 |
\begin{verbatim} |
\begin{verbatim} |
903 |
|
|
904 |
If the cache is SLAB\_HWCACHE\_ALIGN, it's aligning on the size of |
If the cache is SLAB\_HWCACHE\_ALIGN, it's aligning on the size of |
905 |
L1\_CACHE\_BYES which is quiet large, 32 bytes on an intel. So, align is |
L1\_CACHE\_BYES which is quiet large, 32 bytes on an intel. So, align is |
906 |
adjusted to that two objects could fit in a cache line. If 2 would fit, then |
adjusted to that two objects could fit in a cache line. If 2 would fit, |
907 |
try 4, until as many objects are packed in. Then size is adjusted to the new |
then try 4, until as many objects are packed in. Then size is adjusted to |
908 |
alignment |
the new alignment |
909 |
|
|
910 |
\begin{verbatim} |
\begin{verbatim} |
911 |
/* Cal size (in pages) of slabs, and the num |
/* Cal size (in pages) of slabs, and the num |
940 |
} |
} |
941 |
\end{verbatim} |
\end{verbatim} |
942 |
|
|
943 |
The break\_flag is set so that the gfporder is reduced only once when |
The break\_flag is set so that the gfporder is reduced only once when off-slab |
944 |
off-slab slab\_t's are in use. The second check is so the order doesn't |
slab\_t's are in use. The second check is so the order doesn't get higher |
945 |
get higher than whats possible. If num is zero, it means the gfporder is |
than whats possible. If num is zero, it means the gfporder is too low and |
946 |
too low and needs to be increased. The last check is if the slab\_t is |
needs to be increased. The last check is if the slab\_t is offslab. There |
947 |
offslab. There is a limit to how many objects can be managed off-slab. If |
is a limit to how many objects can be managed off-slab. If it's hit, the |
948 |
it's hit, the order is reduced and kmem\_cache\_estimate is called again. |
order is reduced and kmem\_cache\_estimate is called again. |
949 |
|
|
950 |
\begin{verbatim} |
\begin{verbatim} |
951 |
/* |
/* |
1019 |
cachep->colour = left_over/offset; |
cachep->colour = left_over/offset; |
1020 |
\end{verbatim} |
\end{verbatim} |
1021 |
|
|
1022 |
offset is the offset between each object so that the slab is coloured so that |
offset is the offset between each object so that the slab is coloured so |
1023 |
each object would get different cache lines. |
that each object would get different cache lines. |
1024 |
|
|
1025 |
\begin{verbatim} |
\begin{verbatim} |
1026 |
/* init remaining fields */ |
/* init remaining fields */ |
1049 |
|
|
1050 |
\end{verbatim} |
\end{verbatim} |
1051 |
|
|
1052 |
This just copies the information into the kmem\_cache\_t and initialises it's |
This just copies the information into the kmem\_cache\_t and initialises |
1053 |
fields. The kmem\_find\_general\_cachep despite it's funny name just goes |
it's fields. The kmem\_find\_general\_cachep despite it's funny name just |
1054 |
through the sized caches used by kmalloc until it finds one big enough to |
goes through the sized caches used by kmalloc until it finds one big enough |
1055 |
store the slab\_t . |
to store the slab\_t . |
1056 |
|
|
1057 |
\begin{verbatim} |
\begin{verbatim} |
1058 |
#ifdef CONFIG_SMP |
#ifdef CONFIG_SMP |
1100 |
|
|
1101 |
Add the cache to the chain and return. |
Add the cache to the chain and return. |
1102 |
|
|
|
|
|
|
|
|
1103 |
\section{Growing a Cache} |
\section{Growing a Cache} |
1104 |
\subsection{Function kmem\_cache\_grow()} |
\subsection{Function kmem\_cache\_grow()} |
1105 |
\textit{File: }\url{mm/slab.c} |
\textit{File: }\url{mm/slab.c} |
1112 |
When there is no partial of free slabs left, the cache has to grow by |
When there is no partial of free slabs left, the cache has to grow by |
1113 |
allocating a new slab and placing it on the free list. It is quiet long but |
allocating a new slab and placing it on the free list. It is quiet long but |
1114 |
not too complex. |
not too complex. |
1115 |
|
|
1116 |
\begin{verbatim} |
\begin{verbatim} |
1117 |
|
|
1118 |
slab_t *slabp; |
slab_t *slabp; |
1132 |
return 0; |
return 0; |
1133 |
\end{verbatim} |
\end{verbatim} |
1134 |
|
|
1135 |
Straight forward. Make sure we are not trying to grow a slab that shouldn't be |
Straight forward. Make sure we are not trying to grow a slab that shouldn't |
1136 |
grown. |
be grown. |
1137 |
|
|
1138 |
\begin{verbatim} |
\begin{verbatim} |
1139 |
if (in_interrupt() && (flags & SLAB_LEVEL_MASK) |
if (in_interrupt() && (flags & SLAB_LEVEL_MASK) |
1141 |
BUG(); |
BUG(); |
1142 |
\end{verbatim} |
\end{verbatim} |
1143 |
|
|
1144 |
Make sure that if we are in an interrupt that the appropriate ATOMIC flags are |
Make sure that if we are in an interrupt that the appropriate ATOMIC flags |
1145 |
set so we don't accidently sleep. |
are set so we don't accidently sleep. |
1146 |
|
|
1147 |
\begin{verbatim} |
\begin{verbatim} |
1148 |
ctor_flags = SLAB_CTOR_CONSTRUCTOR; |
ctor_flags = SLAB_CTOR_CONSTRUCTOR; |
1157 |
\end{verbatim} |
\end{verbatim} |
1158 |
|
|
1159 |
Set the appropriate flags for growing a cache and set ATOMIC if necessary. |
Set the appropriate flags for growing a cache and set ATOMIC if necessary. |
1160 |
SLAB\_LEVEL\_MASK is the collection of GFP masks that determines how the buddy |
SLAB\_LEVEL\_MASK is the collection of GFP masks that determines how the |
1161 |
allocator will behave. |
buddy allocator will behave. |
1162 |
|
|
1163 |
\begin{verbatim} |
\begin{verbatim} |
1164 |
/* About to mess with non-constant members - lock. */ |
/* About to mess with non-constant members - lock. */ |
1176 |
offset *= cachep->colour_off; |
offset *= cachep->colour_off; |
1177 |
\end{verbatim} |
\end{verbatim} |
1178 |
|
|
1179 |
The colour will affect what cache line each object is assigned to on the CPU |
The colour will affect what cache line each object is assigned to on the |
1180 |
cache. colour\_off is how far has to be jumped for each cache line. |
CPU cache. colour\_off is how far has to be jumped for each cache line. |
1181 |
colour\_next is what number line we want to go to. This will calculate the |
colour\_next is what number line we want to go to. This will calculate the |
1182 |
offset to be colour\_next * colour\_off . It will increase colour\_next unless |
offset to be colour\_next * colour\_off . It will increase colour\_next |
1183 |
it reaches the max amount of colouring for this slab, cachep->colour in which |
unless it reaches the max amount of colouring for this slab, cachep->colour |
1184 |
case it'll go back to the first lines |
in which case it'll go back to the first lines |
1185 |
|
|
1186 |
\begin{verbatim} |
\begin{verbatim} |
1187 |
cachep->dflags |= DFLGS_GROWN; |
cachep->dflags |= DFLGS_GROWN; |
1229 |
} while (--i); |
} while (--i); |
1230 |
\end{verbatim} |
\end{verbatim} |
1231 |
|
|
1232 |
The struct page is used to keep track of the cachep and slabs. From |
The struct page is used to keep track of the cachep and slabs. From the head, |
1233 |
the head, search forward for the cachep and search back for the |
search forward for the cachep and search back for the slabp. SET\_PAGE\_CACHE |
1234 |
slabp. SET\_PAGE\_CACHE inserts the cachep onto the front of the |
inserts the cachep onto the front of the list. SET\_PAGE\_SLAB will place |
1235 |
list. SET\_PAGE\_SLAB will place the slab on end of the list. PageSetSlab |
the slab on end of the list. PageSetSlab is a macro which sets the PG\_slab |
1236 |
is a macro which sets the PG\_slab bit on the page flags. The while loop |
bit on the page flags. The while loop will do this for each page that was |
1237 |
will do this for each page that was allocated for this slab. |
allocated for this slab. |
1238 |
|
|
1239 |
\begin{verbatim} |
\begin{verbatim} |
1240 |
kmem_cache_init_objs(cachep, slabp, ctor_flags); |
kmem_cache_init_objs(cachep, slabp, ctor_flags); |
1248 |
cachep->growing--; |
cachep->growing--; |
1249 |
\end{verbatim} |
\end{verbatim} |
1250 |
|
|
1251 |
Lock the cache so the slab can be inserted on the list and say that we |
Lock the cache so the slab can be inserted on the list and say that we are not |
1252 |
are not growing any more so that the cache will be considered for reapin |
growing any more so that the cache will be considered for reapin again later. |
|
again later. |
|
1253 |
|
|
1254 |
\begin{verbatim} |
\begin{verbatim} |
1255 |
/* Make slab active. */ |
/* Make slab active. */ |
1278 |
} |
} |
1279 |
\end{verbatim} |
\end{verbatim} |
1280 |
|
|
1281 |
opps1 is reached if a slab manager could not be allocated. failed is reached if |
opps1 is reached if a slab manager could not be allocated. failed is reached |
1282 |
pages could not be allocated for the slab at all. |
if pages could not be allocated for the slab at all. |
|
|
|
|
|
|
|
|
|
1283 |
|
|
1284 |
\subsection{Function kmem\_cache\_slabmgmt()} |
\subsection{Function kmem\_cache\_slabmgmt()} |
1285 |
\textit{File: }\url{mm/slab.c} |
\textit{File: }\url{mm/slab.c} |
1303 |
\end{verbatim} |
\end{verbatim} |
1304 |
|
|
1305 |
The first check is to see if the slab\_t is kept off the slab. If it is, |
The first check is to see if the slab\_t is kept off the slab. If it is, |
1306 |
cachep-$>$slabp\_cache will be pointing to the cache of memory allocations large |
cachep-$>$slabp\_cache will be pointing to the cache of memory allocations |
1307 |
enough to contain the slab\_t. The different size caches are the same ones used |
large enough to contain the slab\_t. The different size caches are the same |
1308 |
by kmalloc. |
ones used by kmalloc. |
1309 |
|
|
1310 |
\begin{verbatim} |
\begin{verbatim} |
1311 |
} else { |
} else { |
1316 |
} |
} |
1317 |
\end{verbatim} |
\end{verbatim} |
1318 |
|
|
1319 |
Otherwise the slab\_t struct is contained on the slab itself at the beginning of |
Otherwise the slab\_t struct is contained on the slab itself at the beginning |
1320 |
the slab. |
of the slab. |
1321 |
|
|
1322 |
\begin{verbatim} |
\begin{verbatim} |
1323 |
slabp->inuse = 0; |
slabp->inuse = 0; |
1327 |
return slabp; |
return slabp; |
1328 |
\end{verbatim} |
\end{verbatim} |
1329 |
|
|
1330 |
The most important one to note here is the value of s\_mem. It'll be set to be |
The most important one to note here is the value of s\_mem. It'll be set to |
1331 |
at the beginning of the slab if the slab manager is off slab but at the end of |
be at the beginning of the slab if the slab manager is off slab but at the |
1332 |
the slab\_t if it's on slab. |
end of the slab\_t if it's on slab. |
|
|
|
|
|
|
|
|
|
1333 |
|
|
1334 |
\section{Shrinking Caches} |
\section{Shrinking Caches} |
1335 |
|
|
1366 |
drain_cpu_caches(cachep); |
drain_cpu_caches(cachep); |
1367 |
\end{verbatim} |
\end{verbatim} |
1368 |
|
|
1369 |
drain\_cpu\_caches will try and remove the objects kept available for a |
drain\_cpu\_caches will try and remove the objects kept available |
1370 |
particular CPU that would have been allocated earlier with |
for a particular CPU that would have been allocated earlier with |
1371 |
kmem\_cache\_alloc\_batch. |
kmem\_cache\_alloc\_batch. |
1372 |
|
|
1373 |
\begin{verbatim} |
\begin{verbatim} |
1381 |
|
|
1382 |
\end{verbatim} |
\end{verbatim} |
1383 |
|
|
1384 |
As the number of slabs freed is returned, bit shifting it by gfporder will |
As the number of slabs freed is returned, bit shifting it by gfporder |
1385 |
give the number of pages freed. There is a similar function called |
will give the number of pages freed. There is a similar function called |
1386 |
\_\_kmem\_cache\_shrink. The only difference with it is that it returns a |
\_\_kmem\_cache\_shrink. The only difference with it is that it returns a |
1387 |
boolean on whether the whole cache is free or not. |
boolean on whether the whole cache is free or not. |
1388 |
|
|
1392 |
\begin{verbatim} |
\begin{verbatim} |
1393 |
int __kmem_cache_shrink_locked(kmem_cache_t *cachep) |
int __kmem_cache_shrink_locked(kmem_cache_t *cachep) |
1394 |
\end{verbatim} |
\end{verbatim} |
1395 |
|
|
1396 |
This function cycles through all the slabs\_free in the cache and calls |
This function cycles through all the slabs\_free in the cache and calls |
1397 |
kmem\_slab\_destory (described below) on each of them. The code is very straight forward. |
kmem\_slab\_destory (described below) on each of them. The code is very |
1398 |
|
straight forward. |
1399 |
|
|
1400 |
\begin{verbatim} |
\begin{verbatim} |
1401 |
|
|
1402 |
slab_t *slabp; |
slab_t *slabp; |
1403 |
int ret = 0; |
int ret = 0; |
1404 |
|
|
1411 |
break; |
break; |
1412 |
|
|
1413 |
\end{verbatim} |
\end{verbatim} |
1414 |
If the list \texttt{slabs\_free} is empty, then both \textit{slabs\_free.prev} and |
|
1415 |
\textit{slabs\_free.next} point to itself. The above code checks for this condition and quits as there are no empty slabs to free. |
If the list \texttt{slabs\_free} is empty, then both \textit{slabs\_free.prev} |
1416 |
|
and \textit{slabs\_free.next} point to itself. The above code checks for |
1417 |
|
this condition and quits as there are no empty slabs to free. |
1418 |
|
|
1419 |
\begin{verbatim} |
\begin{verbatim} |
1420 |
|
|
1421 |
slabp = list_entry(cachep->slabs_free.prev, slab_t, list); |
slabp = list_entry(cachep->slabs_free.prev, slab_t, list); |
1436 |
list_del(&slabp->list); |
list_del(&slabp->list); |
1437 |
|
|
1438 |
\end{verbatim} |
\end{verbatim} |
1439 |
Since we are going to free this slab, remove it from the \textit{slabs\_free} list. |
|
1440 |
|
Since we are going to free this slab, remove it from the \textit{slabs\_free} |
1441 |
|
list. |
1442 |
|
|
1443 |
\begin{verbatim} |
\begin{verbatim} |
1444 |
|
|
1445 |
|
|
1446 |
spin_unlock_irq(&cachep->spinlock); |
spin_unlock_irq(&cachep->spinlock); |
1447 |
kmem_slab_destroy(cachep, slabp); |
kmem_slab_destroy(cachep, slabp); |
1448 |
ret++; |
ret++; |
1450 |
} |
} |
1451 |
return ret; |
return ret; |
1452 |
\end{verbatim} |
\end{verbatim} |
|
Call \texttt{kmem\_slab\_destroy()} (which is discussed below) to actually do the formalities of freeing the slab. Increment the value of \textit{ret}, which is used to count the number of slabs being freed. |
|
|
|
|
1453 |
|
|
1454 |
|
Call \texttt{kmem\_slab\_destroy()} (which is discussed below) to actually |
1455 |
|
do the formalities of freeing the slab. Increment the value of \textit{ret}, |
1456 |
|
which is used to count the number of slabs being freed. |
1457 |
|
|
1458 |
\subsection{Function \_\_kmem\_slab\_destroy()} |
\subsection{Function \_\_kmem\_slab\_destroy()} |
1459 |
\textit{File: }\url{mm/slab.c} |
\textit{File: }\url{mm/slab.c} |
1462 |
void kmem_slab_destroy (kmem_cache_t *cachep, |
void kmem_slab_destroy (kmem_cache_t *cachep, |
1463 |
slab_t *slabp) |
slab_t *slabp) |
1464 |
\end{verbatim} |
\end{verbatim} |
1465 |
This function cycles through all objects in a slab and does the required cleanup. Before calling, the slab must have been unlinked from the cache. |
|
1466 |
|
This function cycles through all objects in a slab and does the required |
1467 |
|
cleanup. Before calling, the slab must have been unlinked from the cache. |
1468 |
|
|
1469 |
\begin{verbatim} |
\begin{verbatim} |
1470 |
if (cachep->dtor |
if (cachep->dtor |
1471 |
#if DEBUG |
#if DEBUG |
1474 |
) { |
) { |
1475 |
|
|
1476 |
\end{verbatim} |
\end{verbatim} |
1477 |
If a destructor exists for this slab, or if DEBUG is enabled and the necessary flags are present, continue. |
If a destructor exists for this slab, or if DEBUG is enabled and the necessary |
1478 |
|
flags are present, continue. |
1479 |
\begin{verbatim} |
\begin{verbatim} |
1480 |
|
|
1481 |
int i; |
int i; |
1499 |
(cachep->dtor)(objp, cachep, 0); |
(cachep->dtor)(objp, cachep, 0); |
1500 |
|
|
1501 |
\end{verbatim} |
\end{verbatim} |
1502 |
If a destructor exists for this slab, then invoke it on the object. The destructors are *not* used in Linux. It has been kept for some future use. |
|
1503 |
|
If a destructor exists for this slab, then invoke it on the object. The |
1504 |
|
destructors are *not* used in Linux. It has been kept for some future use. |
1505 |
|
|
1506 |
\begin{verbatim} |
\begin{verbatim} |
1507 |
#if DEBUG |
#if DEBUG |
1508 |
if (cachep->flags & SLAB_RED_ZONE) { |
if (cachep->flags & SLAB_RED_ZONE) { |
1518 |
kmem_freepages(cachep, slabp->s_mem-slabp->colouroff); |
kmem_freepages(cachep, slabp->s_mem-slabp->colouroff); |
1519 |
|
|
1520 |
\end{verbatim} |
\end{verbatim} |
1521 |
\texttt{kmem\_freepages()} will call the buddy allocator to free the pages for the slab. |
|
1522 |
|
\texttt{kmem\_freepages()} will call the buddy allocator to free the pages |
1523 |
|
for the slab. |
1524 |
|
|
1525 |
\begin{verbatim} |
\begin{verbatim} |
1526 |
|
|
1527 |
if (OFF_SLAB(cachep)) |
if (OFF_SLAB(cachep)) |
1528 |
kmem_cache_free(cachep->slabp_cache, slabp); |
kmem_cache_free(cachep->slabp_cache, slabp); |
1529 |
|
|
1530 |
\end{verbatim} |
\end{verbatim} |
|
If the slab\_t is kept off-slab, it's cache entry must be removed. |
|
|
|
|
1531 |
|
|
1532 |
|
If the slab\_t is kept off-slab, it's cache entry must be removed. |
1533 |
|
|
1534 |
\section{Destroying Caches} |
\section{Destroying Caches} |
1535 |
|
|
1536 |
Destroying a cache is yet another glorified list manager. It is called |
Destroying a cache is yet another glorified list manager. It is called when |
1537 |
when a module is unloading itself or is being destroyed. This is to prevent |
a module is unloading itself or is being destroyed. This is to prevent |
1538 |
caches with duplicate caches been created if the module is unloaded and |
caches with duplicate caches been created if the module is unloaded and |
1539 |
loaded several times. |
loaded several times. |
1540 |
|
|
1541 |
First the cache is removed from the cache chain. Then \_\_kmem\_cache\_shrink |
First the cache is removed from the cache chain. Then \_\_kmem\_cache\_shrink |
1542 |
is called to do the release of slabs. It works the same as |
is called to do the release of slabs. It works the same as kmem\_cache\_shrink |
1543 |
kmem\_cache\_shrink does except it returns a boolean indicating if all |
does except it returns a boolean indicating if all slabs were free or not. If |
1544 |
slabs were free or not. If they were, the cache is freed. If they were not, |
they were, the cache is freed. If they were not, the cache is added back to |
1545 |
the cache is added back to the cache chain and an error is printed. |
the cache chain and an error is printed. |
1546 |
|
|
1547 |
\section{kmalloc/kfree} |
\section{kmalloc/kfree} |
1548 |
|
|