Module System Design Notes
**************************

This chapter documents module system design history.  At the moment (guile-1.5.4, 2002-02-08), the module system is supposedly undergoing redesign; the provisional implementation works but has problems, notably making compilation difficult.

Because module systems design is (was?) an area of active research and development in the Scheme community, many different features are possible. This section is a historical record of the selection process used by the Guile hackers (if one can be discerned).

Wishlist
--------

In the guile-related mailing lists of yore, discussion resulted in the following desirable traits.  Note that some of these contradict each other.

  * support separate compilation

  * hierarchical module names

  * support relative references within the module name space (so that a module within a package can use a sibling module without knowing the prefix of the module name)

  * support re-use of code (the same implementation can be presented to the world through several interfaces)

  * support individual and group renaming of bindings when using other modules

  * easy to import and re-export entire interfaces (so that a main interface in a package can function as a "relay" and publish bindings from many modules in the package)

  * support both variable and syntactic bindings (these should be clearly separated, though) and mesh well with hygienic macro systems

  * hygienic implies that we shouldn't need to care to export bindings which an exported macro introduces at the point of its use

  * autoloading

  * unmemoization protocol

  * cleanliness

     A module should be able to be totally clean.  There should be no need to have *any* extra bindings in a module (a la %module-interface or `define-module').

     Therefore, we should have at least one dedicated "command" or "config" or "repl" module.

     It would probably be a good idea to follow other good Scheme interpreters' lead and introduce the ,<command> syntax for walking around modules, inspecting things, entering the debugger, etc. Such commands can be looked up in this repl module.

     If we insist on not using ,<command> syntax, we are forced to let the module use list consist of a "sticky" part and the rest, where the "sticky" part is only available at the repl prompt and not to the code within the module, and which follows us when we walk around in the system.

  * well integrated with the translator framework

     We should be able to say that a module uses a different syntax or language.

     Note here the nice config language of the Scheme48 module system where it is possible to separate code from module specification: the module specification can use scheme syntax and rest in one file, while the module itself can use the syntax of another language.

     This config language also supports re-use of code in a neat way.

  * examine connection with object system: how easy is it to support Java and other class-centered object systems?

  * easy to export the same module under several different names

  * easily supports both compiled and interpreted modules

  * compiled modules can by dynamically loaded or statically linked in (libltdl might provide this automatically)

  * convenient syntax for referencing bindings in modules that are loaded but not used

     (Assuming this distinction exists.)  But maybe group renaming is a better solution to a similar class of problems.

  * ability to unuse a module (and have it get collected when there are no more references)

  * orthoganality between source files, directories and modules. i.e.  ability to have multiple modules in one source file and/or multiple source files in one module

  * backward compatibility

  * whenever possible the module's meta-information should be stored within the module itself (only possible for scheme files)

  * the compiler stores the meta-information into a single file and updates it accordingly

     (FIXME: per module, per package, directory?, per project?)  This meta-information should still be human readable (Sun's EJB use XML for their deployment descriptors).

  * use the file system as module repository

     Since guile is a GNU project we can expect a GNU (or Unix) environment.  That means that we should use the file system as the module repository.  (This would help us avoid modules pointing to files which are pointing to other files telling the user "hey, that's me (the module) over there".)

  * every module has exactly one owner who is responsible for the module and its interface (this contradicts with the "more than one interface" concept)

  * support module collections

     Provide "packages" with a package scope for people working together on a project.  In some sense a module is a view on the underlying package.

  * ability to request (i.e. import or access) complete packages

  * support module "generations" (or "versions")

     Whenever a new module fails to handle a request (due to an error) it will be replaced by the old version.

  * help the user to handle exceptions (note: exceptions are not errors, see above)

  * no special configuration language (like ,in etc.)

     You can always press Control-D to terminate the module's repl and return to the config module.

  * both C and Scheme level interfaces

  * programming interface to module system primitives

     One should be able to make customized module systems from the low-level interface, as an alternative to using the default module system.  The default module system should use the same low-level procedures.

  * Here are some features Keisuke Nishida desires to support his VM/compiler [snarfed directly from post <m37l33z0cl.wl.cwru.edu> dated 2001-02-06, and requires formatting]:

     * There is no "current module".

     * Module variables are globally identified by an identifier like "system::core::car".

     * Bindings are solved syntactically, either by a translator or a compiler.  If you write just "car", it is expanded to "system::core::car" by a translator or compiler, depending on what modules you use.

     * An interpreter (repl) may memorize the "current working module". It tells the translator or the compiler what modules should be used to identify a variable.  So, during interactive sessions, a user may feel as if there *is* the current module.

     * But the fact is, all variables are globally identified at syntax level.  Therefore, the compiler can identify all variables at compile time.  This means the following code is not allowed:

     ;; I want to access `foo' in the module named some-module-name (let ((module (lookup-module some-module-name))) (set! (current-module) module) (foo x)) -> ERROR: Unbound variable: current-module

     (let ((module (lookup-module some-module-name))) (module::foo x)) -> ERROR: There is no variable named "module::foo"

     Instead, you should write as follows if you need a dynamic access:

     (let ((module (lookup-module some-module-name))) ((module-ref module 'foo) x))

     (let ((module (lookup-module some-module-name))) ((module 'foo) x))      ;; if module is an applicable smob

Findings
--------

This section briefly describes module system truths that are one step more detailed than "module systems are hairy".  These truths are not self-evident; we rely on research, active experimentation and lots of discussion.  The goal of this section is to save ourselves from rehashing that which was hashed previously.

  * Kent Dybvig's module system

     A paper is available at http://www.cs.indiana.edu/~dyb/papers/popl99.ps.gz.

     This was discussed in 2000-11 and 2000-12.

  * Distinction between Top-Level Environment and Module

     These two are different beasts!  Each of the following needs to be well-defined with respect to both of these concepts: eval, define, define-public, define-module, load, working from REPL, top-level begin, [add here].

     In guile-1.4, the distinction was not clear.

  * Current module system internals

     See Module Internals (module-snippets), for implemetation details of the module system up to and including guile-1.6.x.

  * [add here]

Selection Criteria
------------------

Rationale Statements
--------------------

Specification
-------------