193 |
|
|
194 |
\subsection{Allocation Policy} |
\subsection{Allocation Policy} |
195 |
|
|
196 |
|
\subsubsection{Guaranteed Pages and Extra Pages} |
197 |
|
|
198 |
The physical memory server maintains a concept of \keyword{guaranteed |
The physical memory server maintains a concept of \keyword{guaranteed |
199 |
pages} and \keyword{extra pages}. The former are pages that a given |
pages} and \keyword{extra pages}. The former are pages that a given |
200 |
task is guaranteed to map in a very short amount of time. Given this |
task is guaranteed to map in a very short amount of time. Given this |
201 |
predicate, the total number of guaranteed pages can never exceed the |
predicate, the total number of guaranteed pages can never exceed the |
202 |
total number of frames in the system. Extra pages are pages which are |
total number of frames in the system. Extra pages are pages which are |
203 |
given to clients who have reached their guaranteed page allocation |
given to clients who have reached their guaranteed page allocation |
204 |
limit. The phsyical memory server may request that a client |
limit. The physical memory server may request that a client |
205 |
relinquish a number of extant extra pages at any time. The client |
relinquish a number of extant extra pages at any time. The client |
206 |
must return the pages to the physical memory (i.e. free them) in a |
must return the pages to the physical memory (i.e. free them) in a |
207 |
short amount of time. Should a task fail to do this, it risks having |
short amount of time. The task should not assume that it has enough |
208 |
all of its memory dropped (i.e. not swapped out or saved in anyway) |
time to send pages to backing store. As such, extra pages should only |
209 |
and reclaimed by the physical memory server. |
contain remanufacturable data (i.e. cached data). Should a task fail |
210 |
|
to return the pages in a reasonable amount of time, it risks having |
211 |
Readers familiar with VMS will see a striking difference between these |
all of its memory dropped---not swapped out or saved in anyway---and |
212 |
two systems. This is not without reason. Yet, differences remains: |
reclaimed by the physical memory server. Note that the physical |
213 |
VMS does not have extra pages and the number of pages is fixed at task |
memory server does not know if a given page is considered guaranteed |
214 |
creation time. VMS than maintains a dirty list of pages thereby |
or extra: it knows that a given task has $G$ guaranteed pages and $G + |
215 |
having a very fast backing store and essentially allowing tasks to |
E$ allocated pages, it has $E$ extra pages. The distinction between |
216 |
have more than their quota of memory if there is no memory pressure. |
guaranteed and extra pages must be made by the task itself. One |
217 |
One reason that this is copied in this design is that unlike in VMS, |
strategy is to remember which pages can be remanufactured (e.g. reread |
218 |
the file systems and device drivers are in user space. Thus, the |
from disk or recalculated) and promote them to guaranteed pages when |
219 |
caching that was being done by VMS can not be done intelligently by |
the page becomes dirty being careful to never have less than $E$ clean |
220 |
the physical memory server. |
pages in the task. Given these semantics, guanteed pages should not |
221 |
|
be thought of as wired (e.g. \fuction{mlock}ed in the POSIX |
222 |
|
sense)---although they can have this property---but as frames which |
223 |
|
the task itself must multiplex. Thus the idea of self-paged tasks. |
224 |
|
|
225 |
|
Readers familiar with VMS will see striking similarities with the |
226 |
|
self-paging and guaranteed page paradigms. This is not without |
227 |
|
reason. Yet, differences remain: VMS does not have extra pages and |
228 |
|
the number of guaranteed pages is fixed at task creation time. Pages |
229 |
|
returned to VMS (in order to allocate a new page) are placed in a |
230 |
|
dirty list (thus the actual multiplexing of frames is done in VMS, not |
231 |
|
in user space) thereby simulating a two level backing store: a fast |
232 |
|
memory backing store where pages are waylaid and swap, where they are |
233 |
|
sent when memory pressure forces out. It is in this way that a given |
234 |
|
task may get at more than its quota of memory when there is low memory |
235 |
|
contention. Our divergence from VMS is motivated by the location of |
236 |
|
file systems and device drivers in the Hurd: unlike in VMS, the file |
237 |
|
systems and device drivers are in user space. Thus, the caching that |
238 |
|
was being done by VMS cannot be done intelligently by the physical |
239 |
|
memory server. |
240 |
|
|
241 |
|
\subsubsection{An External Memory Policy Server} |
242 |
|
|
243 |
The number of guaranteed pages that a given task has access to is not |
The number of guaranteed pages that a given task has access to is not |
244 |
determined by the physical memory server but by the \keyword{memory |
determined by the physical memory server but by the \keyword{memory |
245 |
policy server}. This division allows the physical memory server to |
policy server}. This division allows the physical memory server to |
246 |
only concern itself with the mechanisms and means that it must know |
concern itself primarily with the allocation mechanisms and delegate |
247 |
essentially nothing about how the underlying operating system |
all the policy decisions to the underlying operating system. (An |
248 |
functions. (The implication is that although tailored for Hurd |
important implication is that although tailored for Hurd specific |
249 |
specific needs, the physical memory server is completely separate from |
needs, the physical memory server is completely separate from the Hurd |
250 |
the Hurd and can be used by other operating systems running on the |
and can be used by other operating systems running on the L4 |
251 |
microkernel.) Thus, it is the memory policy server's responsibility |
microkernel.) It is the memory policy server's responsibility to |
252 |
to determine who gets how much memory. This may be determined as a |
determine who gets how much memory. This may be determined as a |
253 |
function of the user or looking in file on disk for e.g. quotas. As |
function of the user or looking in a file on disk for e.g. quotas. As |
254 |
can be seen this type of data acquisition could add significant |
can be seen this type of data acquisition could add significant |
255 |
complexity to the physical memory server and require blocking states |
complexity to the physical memory server and require blocking states |
256 |
(e.g. waiting for a read operation on file i/o) and could create |
(e.g. waiting for a read operation on file i/o) and could create |
257 |
circular dependencies. |
circular dependencies. The default memory policy server's mechanisms |
258 |
|
and policies will be discussed later. |
259 |
|
|
260 |
The physical memory server and the memory policy server will contain a |
The physical memory server and the memory policy server will contain a |
261 |
shared buffer of tupples indexed by task id containing the number of |
shared buffer of tupples indexed by task id containing the number of |
266 |
may only be written to by the physical memory server. This scheme |
may only be written to by the physical memory server. This scheme |
267 |
means that no locking in required. (On some architectures where a |
means that no locking in required. (On some architectures where a |
268 |
read of a given field cannot be performed in a single operation, the |
read of a given field cannot be performed in a single operation, the |
269 |
read may have to be done twice). |
read may have to be done twice.) The memory policy server must not |
270 |
|
over commit the number of frames, i.e. the total number of guaranteed |
271 |
|
pages must never exceed the number of frames avilable for allocation. |
272 |
|
|
273 |
Until the memory policy server makes the intial contact with the |
Until the memory policy server makes the intial contact with the |
274 |
physical memory server, memory will be allocated on a first come first |
physical memory server, memory will be allocated on a first come first |
276 |
procedure call to contact the physical memory server: |
procedure call to contact the physical memory server: |
277 |
|
|
278 |
\begin{code} |
\begin{code} |
279 |
error\_t physical\_memory\_server\_introduce (void) |
error\_t pm\_get\_control (out hurd\_cap\_t control) |
280 |
\end{code} |
\end{code} |
281 |
|
|
282 |
\noindent |
\noindent |
283 |
This function will succeed the first time it is called. It will fail |
This function will succeed the first time it is called and return a |
284 |
all subsequent times. The physical memory server will record the |
control capability. It will fail all subsequent times. By using a |
285 |
sender of this rpc as the memory policy server and begin allocating |
capability, the acquiring task may move or copy the capability to |
286 |
memory according to the previously described protocol. |
another task. This permits replacing the policy server on a live |
287 |
|
system. At this point, the physical memory server will begin |
288 |
|
allocating memory according to the previously described protocol. |
289 |
|
Note that the inital buffer will be initialized with the current total |
290 |
|
allocations but the guaranteed pages will be set to zero. The memory |
291 |
|
policy server must request the shared policy buffer as soon as |
292 |
|
possible and adjust these values. |
293 |
|
|
294 |
The shared policy buffer may be obtained from the physical memory |
The shared policy buffer may be obtained from the physical memory |
295 |
server by the policy by calling: |
server by the policy by calling: |
296 |
|
|
297 |
\begin{code} |
\begin{code} |
298 |
error\_t physical\_memory\_server\_get\_policy\_buffer (out l4\_map\_t buffer) |
error\_t pm\_get\_policy\_buffer (out l4\_map\_t buffer) |
299 |
\end{code} |
\end{code} |
300 |
|
|
301 |
\noindent |
\noindent |
302 |
The returned buffer is mapped with read and write access into the |
The returned buffer is mapped with read and write access into the |
303 |
policy memory server's address space. It may need to be resized. If |
policy memory server's address space. It may need to be resized due |
304 |
this is the case, the physical memory server shall unmap the buffer |
to the number of tasks in the system. When this is the case, the |
305 |
from the policy memory server's address space, copy the buffer |
physical memory server shall unmap the buffer from the memory policy |
306 |
internally as required. The policy memory server will fault on the |
server's address space and copy the buffer internally as required. |
307 |
memory region on its next access and it may repeat the call. This |
The memory policy server will fault on the memory region on its next |
308 |
call will succeed when the sender is the memory policy server, it will |
access and it may rerequest the buffer. This call will succeed when |
309 |
fail otherwise. |
the sender is the memory policy server, it will fail otherwise. |
310 |
|
|
311 |
\subsection{Allocation Mechanisms} |
\subsection{Allocation Mechanisms} |
312 |
|
|
313 |
Applications are able allocate memory by Memory allocation will be |
Applications are able allocate memory into containers. Containers may |
314 |
|
be created using: |
315 |
|
|
316 |
|
\begin{code} |
317 |
|
error\_t pm\_container\_create (out container\_t container) |
318 |
|
\end{code} |
319 |
|
|
320 |
|
Memory allocation does not allocate physical frames: if so, it would |
321 |
|
be impossible to move memory around and memory would have to be |
322 |
|
returned to the same spot after being swapped out and back in. |
323 |
|
Containers are useful for grouping and then moving memory around. |
324 |
|
|
325 |
|
How to get pages. Type of pages (e.g. DMA) or fixed physical address. |
326 |
|
|
327 |
|
Memory is not allocate until map time (and not always then, |
328 |
|
e.g. logical copies). |
329 |
|
|
330 |
|
\section{Mapping Memory from Containers} |
331 |
|
|
332 |
|
A map will occur in a short amount of time: this is not guaranteed to |
333 |
|
happen immediately as the pages are only allocated at this point. |
334 |
|
They may currently be in the form of extra pages for another task. |
335 |
|
|
336 |
|
Pages may be unmapped at any time by the physical memory server. This |
337 |
|
allows pages to be moved in memory by the physical memory server |
338 |
|
thereby permitting page blocking (i.e. construct superpages), opening |
339 |
|
up DMAable memory areas or other specific physical address, and to |
340 |
|
create contiguous blocks of memory (i.e. to defragment the memory). |
341 |
|
The task must always be read to reestablish a mapping. |
342 |
|
|
343 |
|
Mapping can be made readable. A task may request that a mapping be |
344 |
|
read/write, however, the physical memory server may not grant a |
345 |
|
read/write mapping if the page is copy on write, COW. In this case, a |
346 |
|
read mapping will be returned. In order to get a read/write mapping, |
347 |
|
the task must request add the enforced flag. |
348 |
|
|
349 |
|
\section{Moving Data} |
350 |
|
|
351 |
|
In a monolithic kernel, little data is exchanged between tasks. In a |
352 |
|
multiserver system, file systems live in their own tasks and thus |
353 |
|
reading and writing involve servers. Thus, powerful primatives for |
354 |
|
moving memory around with the least number of physical copies, |
355 |
|
i.e. using virtual copy mechanisms which preserve COW pages, etc. |
356 |
|
|
357 |
|
It is important that an fs does not map from one task to another |
358 |
|
directly: the client may not trust the source or the source may die, |
359 |
|
etc. Doing the mapping via the phys memory server means all trust |
360 |
|
issues are resolved at the time of mapping and can be reported to the |
361 |
|
user: the fs cannot pretend to be nice and then revoke mappings and |
362 |
|
silently harm the client. |
363 |
|
|
364 |
|
Data will be moved around using containers. Describe how to read and |
365 |
|
write. Task -> FS -> Device drivers. Locking memory. Caching. |
366 |
|
|
367 |
|
It is important that clients do the allocation for the memory which |
368 |
|
they use: not the servers doing allocations on behalf on clients: in |
369 |
|
the latter, there is no way to do resource tracking. |
370 |
|
|
371 |
|
Discuss mmap: local function call. RPC is done when a page is |
372 |
|
faulted: do a read from the fs (into a container), then map the data |
373 |
|
from the container into the AS as required. |
374 |
|
|
375 |
|
MAP_COPY sucks: fs must save all modified data. What happens when a |
376 |
|
100MB file is completely rewritten (or 1GB, etc)? can we use upcalls? |
377 |
|
If we do, the fs still needs to hold the data in the intern. Can we |
378 |
|
copy the file on disk and use that as backing store (think how |
379 |
|
deleting an open file works). |
380 |
|
|
381 |
|
Can a readonly private mapping once faulted be dropped or must we |
382 |
|
promote it to anonymous memory and send it to swap fearing that the |
383 |
|
underlying block might change between dropping it and rereading it |
384 |
|
(e.g. by another task modifying the file)? |
385 |
|
|
386 |
\section{Caching Store Accesses} |
\section{Caching Store Accesses} |
387 |
|
|
519 |
container's name space. |
container's name space. |
520 |
\end{comment} |
\end{comment} |
521 |
|
|
522 |
|
\subsection{Caching Interfaces} |
523 |
|
|
524 |
|
The physical memory server will do an up call to a victim task |
525 |
|
requesting a number of pages back. The physical memory server may do |
526 |
|
this at any time for any reason and it expects to receive the pages |
527 |
|
back from the task within a short amount of time (the victim task |
528 |
|
should not expect to be able to send the pages to backing store in |
529 |
|
that amount of time). The physical memory server will never request |
530 |
|
guaranteed pages. As such, this number will always be less than or |
531 |
|
equal to the number of allocated pages minus the number of guaranteed |
532 |
|
pages. |
533 |
|
|
534 |
|
\begin{code} |
535 |
|
void pm_return_pages (in int count); |
536 |
|
\end{code} |
537 |
|
|
538 |
|
The physical memory send this message to the task's memory control |
539 |
|
thread. The thread must always be ready to receive: the physical |
540 |
|
memory server will never wait (thus, the thread must be in the |
541 |
|
receiving state). If the thread is not ready, the physical memory |
542 |
|
server assumes that the task is misbehaving. The physical memory |
543 |
|
server does not wait for a reply, instead, the client must free the |
544 |
|
pages using \function{pm_release_pages} as described above. |
545 |
|
|
546 |
|
\section{The Memory Policy Server} |
547 |
|
|
548 |
|
At task creation time, the task must negotiate a medium-term contract |
549 |
|
for guaranteed pages and determine if it shall have access to extra |
550 |
|
pages. This may be renegotiated later. It must be renegotiated when |
551 |
|
the contract expires. The policy server will give the task enough |
552 |
|
time to send pages to swap before committing if the number of |
553 |
|
guaranteed pages is reduced. |
554 |
|
|
555 |
|
\section{Sending Data to Swap} |
556 |
|
|
557 |
|
Data must be sent to swap. The swap server must be in the phsyical |
558 |
|
memory server in order to preserve logical copies in swap (if not, X |
559 |
|
tasks swap a page to the swap server thus X writes/reads to swap |
560 |
|
instead of 1 when all tasks release their references to the page). |
561 |
|
|
562 |
|
Swap quotas (put the policy in the memory policy server). |
563 |
|
|
564 |
|
Memory kept on an inactive list thus allowing recover before a page is |
565 |
|
flushed to swap (i.e. a swap operation is not synchronous). |
566 |
|
|
567 |
|
\section{Self Paging} |
568 |
|
|
569 |
|
Tasks multiplex guaranteed pages. Must manage their own memory. How |
570 |
|
to get data (e.g. extend malloc via the slab mechanism, extend fopen). |
571 |
|
|
572 |
|
|
573 |
% Traditionally, monolithical kernels, but even kernels like Mach, |
% Traditionally, monolithical kernels, but even kernels like Mach, |