/[hurd]/hurd-l4/doc/vmm.tex
ViewVC logotype

Diff of /hurd-l4/doc/vmm.tex

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.6 by neal, Tue Oct 21 19:18:27 2003 UTC revision 1.7 by neal, Thu Oct 23 01:13:34 2003 UTC
# Line 199  The physical memory server maintains a c Line 199  The physical memory server maintains a c
199  frames} and \keyword{extra frames}.  The former are virtual frames  frames} and \keyword{extra frames}.  The former are virtual frames
200  that a given task is guaranteed to map in a very short amount of time.  that a given task is guaranteed to map in a very short amount of time.
201  Given this predicate, the total number of guaranteed frames can never  Given this predicate, the total number of guaranteed frames can never
202  exceed the total number of physical frames in the system.  Extra frames  exceed the total number of physical frames in the system.  Extra
203  are frames which are given to clients who have reached their guaranteed  frames are frames which are given to clients who have reached their
204  frame allocation limit.  The physical memory server may request that a  guaranteed frame allocation limit.  The physical memory server may
205  client relinquish a number of extant extra frames at any time.  The  request that a client relinquish a number of extant extra frames at
206  client must return the frames to the physical memory (i.e. free them)  any time.  The client must return the frames to the physical memory
207  in a short amount of time.  The task should not assume that it has  (i.e. free them) in a short amount of time.  The task should not
208  enough time to send frames to backing store.  As such, extra frames  assume that it has enough time to send frames to backing store.  As
209  should only contain remanufacturable data (i.e. cached data).  Should  such, extra frames should only contain remanufacturable data
210  a task fail to return the frames in a reasonable amount of time, it  (i.e. cached data).  Should a task fail to return the frames in a
211  risks having all of its memory dropped---not swapped out or saved in  reasonable amount of time, it risks having all of its memory
212  anyway---and reclaimed by the physical memory server.  Note that the  dropped---not swapped out or saved in any way---and reclaimed by the
213  physical memory server does not know if a given frame is considered  physical memory server.  Note that the physical memory server does not
214  guaranteed or extra: it knows that a given task has $G$ guaranteed  know if a given frame is considered guaranteed or extra: it knows that
215  frames and $G + E$ allocated frames, it has $E$ extra frames.  The  a given task has $G$ guaranteed frames and $G + E$ allocated frames,
216  distinction between guaranteed and extra frames must be made by the  and $E$ extra frames.  The distinction between guaranteed and extra
217  task itself.  One strategy is to remember which frames can be  frames must be made by the task itself.  One strategy is to remember
218  remanufactured (e.g. reread from disk or recalculated) and promote  which frames can be remanufactured (e.g. reread from disk or
219  them to guaranteed frames when the frame becomes dirty being careful to  recalculated) and internally promote them to guaranteed frames when
220  never have less than $E$ clean frames in the task.  Given these  the frame becomes dirty being careful to never have less than $E$
221  semantics, guanteed frames should not be thought of as wired  clean frames in the task.  Given these semantics, guanteed frames
222  (e.g. \function{mlock}ed in the POSIX sense)---although they can have  should not be thought of as wired (e.g. \function{mlock}ed in the
223  this property---but as frames which the task itself must multiplex.  POSIX sense)---although they can have this property---but as frames
224  Thus the idea of self-paged tasks.  which the task itself must multiplex.  Thus the idea of self-paged
225    tasks.
226    
227  Readers familiar with VMS will see striking similarities with the  Readers familiar with VMS will see striking similarities with the
228  self-paging and guaranteed frame paradigms.  This is not without  self-paging and guaranteed frame paradigms.  This is not without
229  reason.  Yet, differences remain: VMS does not have extra frames and  reason.  Yet, differences remain: VMS does not have extra frames and
230  the number of guaranteed frames is fixed at task creation time.  Frames  the number of guaranteed frames is fixed at task creation time.
231  returned to VMS (in order to allocate a new frame) are placed in a  Frames returned to VMS (in order to allocate a new frame) are placed
232  dirty list (thus the actual multiplexing of frames is done in VMS, not  in a dirty list (thus the actual multiplexing of frames is done in
233  in user space) thereby simulating a two level backing store: a fast  VMS, not in user space) thereby simulating a two level backing store:
234  memory backing store where frames are waylaid and swap, where they are  a fast memory backing store where frames are waylaid and swap, where
235  sent when memory pressure forces out.  It is in this way that a given  they are sent to when sufficient memory pressure forces them out.  It
236  task may get at more than its quota of memory when there is low memory  is in this way that a given task may access more than its quota of
237  contention.  Our divergence from VMS is motivated by the location of  memory when there is low memory contention (e.g. if there are two
238  file systems and device drivers in the Hurd: unlike in VMS, the file  tasks each with 100 frames and there are 1000 frames in the system for
239  systems and device drivers are in user space.  Thus, the caching that  tasks, the remaining 800 are not dormant).  Our divergence from VMS is
240  was being done by VMS cannot be done intelligently by the physical  motivated by the location of file systems and device drivers in the
241  memory server.  Hurd: unlike in VMS, the file systems and device drivers are in user
242    space.  Thus, the caching that was being done by VMS cannot be done
243    intelligently by the physical memory server.
244    
245  \subsubsection{An External Memory Policy Server}  \subsubsection{An External Memory Policy Server}
246    
247  The number of guaranteed frames that a given task has access to is not  The number of guaranteed frames that a given task has access to is not
248  determined by the physical memory server but by the \keyword{memory  determined by the physical memory server but by the \keyword{memory
249  policy server}.  This division allows the physical memory server to  policy server}.  This division means the physical memory server need
250  concern itself primarily with the allocation mechanisms and delegate  only concern itself with allocation mechanisms; all policy decisions
251  all the policy decisions to the underlying operating system.  (An  are delegated to the policy server provided by the underlying
252  important implication is that although tailored for Hurd specific  operating system.  (An important implication is that although tailored
253  needs, the physical memory server is completely separate from the Hurd  for Hurd specific needs, the physical memory server is essentially
254  and can be used by other operating systems running on the L4  separate from the Hurd and can be used by other operating systems
255  microkernel.)  It is the memory policy server's responsibility to  running on the L4 microkernel.)  It is the memory policy server's
256  determine who gets how much memory.  This may be determined as a  responsibility to determine who gets how much memory.  This may be
257  function of the user or looking in a file on disk for e.g. quotas.  As  calculated as a function of the user or looking in a file on disk for
258  can be seen this type of data acquisition could add significant  e.g. quotas.  As can be seen this type of data acquisition could add
259  complexity to the physical memory server and require blocking states  significant complexity to the physical memory server and require
260  (e.g. waiting for a read operation on file i/o) and could create  blocking states (e.g. waiting for a read operation on file i/o) and
261  circular dependencies.  The default memory policy server's mechanisms  could create circular dependencies.  The default memory policy
262  and policies will be discussed later.  server's mechanisms and policies will be discussed later.
263    
264  The physical memory server and the memory policy server will contain a  The physical memory server and the memory policy server will contain a
265  shared buffer of tupples indexed by task id containing the number of  shared buffer of tupples indexed by task id containing the number of
# Line 286  control capability.  It will fail all su Line 289  control capability.  It will fail all su
289  capability, the acquiring task may move or copy the capability to  capability, the acquiring task may move or copy the capability to
290  another task.  This permits replacing the policy server on a live  another task.  This permits replacing the policy server on a live
291  system.  At this point, the physical memory server will begin  system.  At this point, the physical memory server will begin
292  allocating memory according to the previously described protocol.  allocating memory according to the described protocol.  Note that the
293  Note that the inital buffer will be initialized with the current total  inital buffer will be initialized with the current total allocations
294  allocations but the guaranteed frames will be set to zero.  The memory  while the guaranteed frames will be set to zero.  The memory policy
295  policy server must request the shared policy buffer as soon as  server must request the shared policy buffer as soon as possible and
296  possible and adjust these values.  adjust these values.
297    
298  The shared policy buffer may be obtained from the physical memory  The shared policy buffer may be obtained from the physical memory
299  server by the policy by calling:  server by the policy by calling:
# Line 309  The memory policy server will fault on t Line 312  The memory policy server will fault on t
312  access and it may rerequest the buffer.  This call will succeed when  access and it may rerequest the buffer.  This call will succeed when
313  the sender is the memory policy server, it will fail otherwise.  the sender is the memory policy server, it will fail otherwise.
314    
315  \subsection{Allocation Mechanisms}  \subsubsection{Containers}
316    
317    Containers are the basic abstraction used for allocating, addressing
318    and sharing memory.  Conceptually, containers contain a set of
319    integers identifying \keyword{virtual frame}s in the physical memory
320    server.  A virtual frame references a physical frame but is not bound
321    to a particular physical frame (this allows the physical memory server
322    to move the contents of frames around).  Multiple physical frames may
323    reference the same physical frame in which case the memory is shared.
324    Sharing may be either real, e.g. System V shared memory, or logical,
325    e.g. copy on write.
326    
327    When a virtual frame is allocated into a container, there may be no
328    physical frame associated with it.  The physical memory server
329    guarantees that when the contents of the virtual frame is accessed a
330    physical frame will be provided in a short amount of time
331    (cf. guaranteed virtual frames above).
332    
333    Each virtual frame in a container counts against the container's
334    owner's total allocated frames.  Only the owner of a container may
335    allocate frames into a container.
336    
337    Containers only hold virtual frames.  When the contents of a frame are
338    copied to backing store, no association between the data on the
339    backing store and the the frame identifier in the container is
340    maintained by the physical memory server.
341    
342    When a task starts, it will allocate an initial contain and several
343    frames into it.  Typically, the total amount of memory used by an
344    application will exceed the total number of guaranteed frames.  When
345    the task reaches its maximum permitted allocation, it must reuse an
346    available frame.  Typically, the task will choose a victim page, unmap
347    any pages that point to the associated frame, swap the frame out, mark
348    the frame as swapped out and save the swap identifier in the mapping
349    database.  At this point, the task may reuse the frame.  This example
350    illustrates that imagining a virtual frame as bound to a page in a
351    task's address space for its entire lifetime is incorrect.  It should
352    also now be clear that when the data is eventually brought back into
353    memory from backing store, it may reside in a different virtual frame.
354    
355    Containers are used for passing data between tasks.  Typically there
356    will be two tasks, a client and a server.  L4 provides a mechanism to
357    map pages from one address space to another.  This mechanism could be
358    used when a file is mapped into a task's address space, however, this
359    can present several problems.  If the server dies before the client,
360    the mappings in the client's address space will suddenly disappear.
361    Similarly, if the server is malicious, it may revoke the mappings at
362    some inconvenient (i.e. unrecoverable) time for the client causing it
363    to crash.  If a server allocates resources on behalf of the the client
364    it becomes impossible to do system wide resource accounting as many
365    servers are not trusted by the system.  All of these problems are
366    solved by containers.  When a client needs to obtain a memory mapping
367    from a server, it creates a container and adds to it container the
368    number of frames that the server will require for the operation.  It
369    then shares the container with the server and the server copies the
370    data into the frames.  It is important to understand that the server
371    does not ``fill'' the container: the number of frames remains constant
372    but the state of the bits changes.  When the server returns to the
373    client, the client unshares the container and is now able to map the
374    frames into its address space by contacting the physical memory
375    server.  Should the server die, the client remains uneffected as the
376    data is cached in the physical memory server.  The physical memory
377    server is also trusted thus if a task is malicious, it can only be
378    malicious during the initial copy of the data into the container,
379    i.e. before the client starts using the data.  Finally, as the
380    resources are allocated by the client via system servers, resource
381    accounting is possible.
382    
383    \subsection{Creating Containers}
384    
385  Applications are able allocate memory into containers.  Containers may  Applications are able allocate memory into containers.  Containers may
386  be created using:  be created using:
# Line 328  How to get frames.  Type of frames (e.g. Line 399  How to get frames.  Type of frames (e.g.
399  Memory is not allocate until map time (and not always then,  Memory is not allocate until map time (and not always then,
400  e.g. logical copies).  e.g. logical copies).
401    
402  \section{Mapping Memory from Containers}  \section{Mapping Memory in Containers}
403    
404  A map will occur in a short amount of time: this is not guaranteed to  The physical memory server guarantees that a mapping operation will
405  happen immediately as the frames are only allocated at this point.  take a short amount of time: this is not guaranteed to happen
406  They may currently be in the form of extra frames for another task.  immediately as the virtual frames may only be allocated at this point
407    and they may have to be reaped from other tasks' extra frame
408  Pages may be unmapped at any time by the physical memory server.  This  allocations.
409  allows vitual frames to be moved between physical frames by the  
410  physical memory server thereby permitting page blocking (i.e. the  The physical memory server may unmap pages at any time.  This allows
411  construction of superpages), opening up DMAable memory areas or other  the contents of vitual frames to be moved between physical frames by
412  specific physical address, and to create contiguous blocks of memory  the physical memory server which permits page blocking (i.e. the
413  (i.e. to defragment the memory).  The task must always be read to  construction of superpages), the creation of DMAable memory areas or
414  reestablish a mapping.  other specific physical address and the creation of contiguous blocks
415    of memory (e.g. to defragment physical memory).  Tasks must be
416  Mapping can be made readable.  A task may request that a mapping be  prepared to reestablish a mapping with the physical memory server at
417  read/write, however, the physical memory server may not grant a  any time.
418  read/write mapping if the frame is copy on write, COW.  In this case, a  
419  read mapping will be returned.  In order to get a read/write mapping,  Mappings may be granted readonly evil if the a read/write mapping was
420  the task must request add the enforced flag.  requested: the physical memory server will not grant a read/write
421    mapping if the frame is marked copy on write.  In order to get a
422    read/write mapping (and thus force the copy on write early), the task
423    must request add the enforced write flag when mapping.
424    
425    \begin{code}
426    error\_t pm\_container\_map (in container\_t container, in frame\_t
427    start, in int nr\_frames, in int flags)
428    \end{code}
429    
430    \noindent
431    Flags may is a bit wise or of: CONTAINER\_MAP\_READ,
432    CONTAINER\_MAP\_WRITE and CONTAINER\_MAP\_ENFORCE\_WRITE.
433    
434  \section{Moving Data}  \section{Moving Data}
435    

Legend:
Removed from v.1.6  
changed lines
  Added in v.1.7

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26