/[gzz]/manuscripts/FutureVision/vision.rst
ViewVC logotype

Diff of /manuscripts/FutureVision/vision.rst

Parent Directory Parent Directory | Revision Log Revision Log | View Patch Patch

revision 1.183 by benja, Thu Nov 13 20:52:18 2003 UTC revision 1.184 by benja, Thu Nov 13 21:30:49 2003 UTC
# Line 579  Readings Line 579  Readings
579    
580  Fenfire is a free software project aiming at implementing  Fenfire is a free software project aiming at implementing
581  the applitude-oriented user interface concepts on top of an RDF graph.  the applitude-oriented user interface concepts on top of an RDF graph.
582    Fenfire is a research prototype.
583    
584  4.1 Structures  4.1 Structures
585  --------------  --------------
586    
587  Fenfire provides two intertwingled structures for applitudes:  Fenfire provides two interwoven structures for applitudes:
588  RDF (`Lassila and Swick 1999`_)  RDF (`Lassila and Swick 1999`_)
589  and Xanalogical referential fluid media (`Nelson 1999c`_).  and Xanalogical referential fluid media (`Nelson 1999c`_).
590    
# Line 631  We usually show only a small subset of a Line 632  We usually show only a small subset of a
632  relationships at every time: Depending on the task at hand,  relationships at every time: Depending on the task at hand,
633  the user can "switch" individual relationship types on and off.  the user can "switch" individual relationship types on and off.
634    
635    This is entirely different from existing
636    RDF visualization tools, which create
637    a fixed 2D layout for a graph (sometimes shown in a distorted,
638    hyperbolic-like view), and then only allow
639    the user to zoom and scroll through it.
640    
641    If there are too many connections along the active
642    relationship types, we show only as many as fit on the screen
643    and allow the user to scroll through the list. To make this fast
644    even for very large lists, we plan to employ fisheye sampled
645    lists (`Furnas 1997`_), in which every element of
646    an n-elememnt list can be reached with O(log(n)) clicks.
647    
648  RDF diagrams usually label nodes with the string that is  RDF diagrams usually label nodes with the string that is
649  used internally to identify the node. This is not acceptable  used internally to identify the node. This is not acceptable
650  for showing a network of items; items need to be labeled  for showing a network of items; items need to be labeled
# Line 699  focus location is obvious, as shown in t Line 713  focus location is obvious, as shown in t
713      Figure 7: a diagram of buoy animation (`larger image`__).      Figure 7: a diagram of buoy animation (`larger image`__).
714      The "screenshots" with black edges represent      The "screenshots" with black edges represent
715      keyframes, and the others the animation.      keyframes, and the others the animation.
716        The connecting lines between the focus and the buoys have
717        been emphasized in this diagram; in real life, they are
718        translucent.
719            
720      __ buoysMotion.png      __ buoysMotion.png
721    
# Line 714  sees** - going back from the second keyf Line 731  sees** - going back from the second keyf
731  to the orange document is trivial without  to the orange document is trivial without
732  an explicit "back button".  an explicit "back button".
733    
734    Also, buoys move automatically when the document
735    they are connected to is scrolled. Buoys are arranged
736    on an ellipse around the viewport, so that the focused
737    part of the document, inside the ellipse, is not obscured.
738    Additionally, only buoys whose anchors are visible are shown,
739    and the buoys whose anchors are close to the focus
740    are shown larger. This way, the largest buoys are those
741    connected to things at the focus of the view.
742    
743  The intent of buoys is to reduce the cognitive overhead  The intent of buoys is to reduce the cognitive overhead
744  of browsing and editing the structure  of browsing and editing the structure
# Line 747  A mock-up example of a similar scenario Line 771  A mock-up example of a similar scenario
771  4.3 Libvob  4.3 Libvob
772  ----------  ----------
773    
774  Libvob is a subproject of Fenfire, providing a flexible user-interface  We have suggested that users should be able to add
775  toolkit with some novel features that allow the independent  connections to views programmed by somebody else.
776  applitudes to work on the same user interface.  For example, when showing meetings on a timeline,
777    it should be easy to add buoys showing the participants
778  Instead of a static structure of widgets or a scene graph,  of each meeting. To allow such connections to be added
779  the scene is regenerated each time the user presses a key [#libvob-speed]_.  without changing the view's code, we have developed
780    Libvob, a flexible user-interface toolkit.
781  The scene consists of a DAG of coordinate systems,  
782  a keyed tree for *identifying* coordinate  Libvob provides functionality to not only construct
783  systems and a set of renderable objects (vobs) placed into  a scene graph (i.e., specify what to draw on the screen)
784  one or more coordinate systems each.  but to also specify which parts of the scene graph
785    correspond to which objects in the application model,
786  This structure allows  the RDF graph in the case of Fenfire.
787    
788  1) Automatically generated animations between successively  The timeline view, for example, would specify which
789     generated scenes: the coordinate systems are matched  parts of the scene graph represent each event drawn
790     to each other based on the keyed tree  (events being nodes in the RDF graph). A different view,
791    programmed independently, can iterate through these nodes
792  2) More importantly, **post-processing** of the view generated  and connect buoys to each of them. Through the association
793     by other applitudes -- applitudes can **add** new coordinate  with part of the scene graph, it knows *where* to connect
794     systems and vobs on the view, based on what is already shown.  the buoys to.
795    
796     The buoys are added this way: a connector object goes through  Libvob also uses a directed acyclic graph as the scene graph,
797     a generated vobscene and adds buoys to the nodes in it - regardless  rather than the conventional tree; i.e., its graphical model
798     of who placed those nodes in there.  is not hierarchical like that of other toolkits. This provides
799    natural support for transpointing windows (`Nelson 1995`_),
800    i.e., connections between different parts of the view hierarchy:
801    Showing a connection between two different windows is just
802    as simple as showing a box inside a single window, the only
803    difference being that the connection has *two* parents
804    in the scene graph instead of only one.
805    
806    This is necessary for the implementation of buoys, which
807    are an instance of transpointing windows (a connection
808    is drawn between the buoy and the part of the main view
809    it is related to).
810    
811    
812  4.4 An example applitude combining multiple structures: FenPDF  4.4 An example applitude combining multiple structures: FenPDF
813  --------------------------------------------------------------  --------------------------------------------------------------
814    
815  FenPDF is the first concrete prototype of our architecture.  FenPDF is the first concrete prototype of our architecture,
816  It is a tool to make sense of academic literature.  using buoys, Libvob, RDF, and Xanalogical structure.
817    It is a tool for making sense of academic literature.
818    
819  ..  figure:: fenpdf-shot-small.png  ..  figure:: fenpdf-shot-small.png
820    
# Line 787  It is a tool to make sense of academic l Line 824  It is a tool to make sense of academic l
824    
825      __ fenpdf-shot.png      __ fenpdf-shot.png
826    
827    FenPDF is used to structure a set of articles
828    in PostScript or PDF format. Users can transclude
829    pieces of articles onto *spatial canvases*: infinite,
830    scrollable papers.
831    Transclusions are automatically bidirectionally connected
832    to the article they are from; a buoy shows a shrunk version
833    of the article, and clicking on the buoy brings the article
834    to the center for the user to read.
835    
836    Additionally, the user can type text onto the canvases,
837    and link two pieces of text on different canvases
838    (linked canvases are shown as buoys).
839    
840  In the figure, there are two foci.  In the figure, there are two foci.
841  The **upper focus** shows **a PDF article**  The **upper focus** shows **a PDF article**
842  and its buoys there show the places of  and its buoys there show the places of
# Line 801  The canvases and papers use unique backg Line 851  The canvases and papers use unique backg
851  (`Kujala and Lukka 2003`_) to endow them *identity*  (`Kujala and Lukka 2003`_) to endow them *identity*
852  in the eyes of the viewer.  in the eyes of the viewer.
853    
854    We use FenPDF collaboratively in our research group
855    (synchronizing the RDF graph through CVS).
856    In practice, we have:
857    
858    - A canvas for each source (for example, conference,
859      journal issue), with transclusions of the titles
860      and author lists of the articles published there.
861    - Canvases for different topics, such as open hypermedia,
862      spatial hypertext, and so on. These canvases
863      contain transclusions of particularly relevant parts
864      of articles, allowing us to collect the central ideas
865      from several different articles.
866    - Canvases for each article we are working on,
867      containing notes and transclusions from important
868      references.
869    - Canvases for communicating specific ideas. These
870      contain Memex-like "trails" of transclusions from
871      different articles, intersersed with text discussing these.
872    - A central canvas that has links to the other canvases.
873    
874    In our experience, we have found the transclusion facility
875    very useful: copying a well-chosen region of an article
876    to the a canvas allows the user (either the one who made the
877    transclusion, or a collaborator) reading the canvas to quickly
878    just read the emphasized part, going back to the article
879    for the details if desired. The ease with which the transclusions
880    are created (simply cut and paste) has allowed users to quickly summarize
881    the most relevant points of the article on a separate canvas for
882    easy perusal and for contrasting with points from other articles.
883    
884    In FenPDF, articles, spatial canvases, transclusions
885    of an article and pieces of text on a canvas are all
886    represented by RDF nodes.
887  On the structure side, FenPDF uses four small RDF vocabularies:  On the structure side, FenPDF uses four small RDF vocabularies:
888    
889  - ``FF``: Associating Xanalogical referential fluid media  - ``FF``: Associating Xanalogical referential fluid media
# Line 810  On the structure side, FenPDF uses four Line 892  On the structure side, FenPDF uses four
892    This is used both for text content and transclusions    This is used both for text content and transclusions
893    from PS/PDF files.    from PS/PDF files.
894    
   
895  - ``CANVAS2D``: Placing nodes on a canvas - as in  - ``CANVAS2D``: Placing nodes on a canvas - as in
896    spatial hypertext (`Marshall and Shipman 1995`_)    spatial hypertext (`Marshall and Shipman 1995`_)
897    
# Line 828  Unlike in most annotation-related projec Line 909  Unlike in most annotation-related projec
909  no directly annotation-specific program code in FenPDF.  no directly annotation-specific program code in FenPDF.
910  The different orthogonal  The different orthogonal
911  structures combined just give the user the opportunity  structures combined just give the user the opportunity
912  to create any kinds of structure. One common pattern of use we've noticed  to create any kinds of structure.
913  is transcluding small regions from articles related to a single  
914  topic canvas.  Explicit support for taxonomic hypertext (`Parunak 1991`_)
915    and hierarchies is currently being planned.
916    
917  FenPDF will first be released as a separate application,  FenPDF will first be released as a separate application,
918  but it is designed to become one applitude when the Fenfire system  but it is designed to become one applitude when the Fenfire system
919  is complete. The RDF graphs written by the application  is complete. The RDF graphs written by the application
920  will be importable into Fenfire.  will be importable into Fenfire.
921    
922  Explicit support for taxonomic hypertext (`Parunak 1991`_)      Let us briefly explore how a user could extend
923  and hierarchies is currently being planned.      this applitude using the techniques presented in
924        `Section 2`_.
925    
926        First, metadata about articles, such as author
927        and publication date, could be represented through simple
928        RDF connections. Using our RDF views, one could then
929        for example browse the list of articles of one particular author,
930        sorted by date.
931    
932        Similarly, it is possible to switch from spatial hypertext (`Marshall and Shipman 1995`_)
933        to taxonomic hypertext (`Parunak 1991`_) by replacing the canvases
934        categorizing articles by source with RDF metadata,
935        and browse the articles from one source related
936        to certain subject matters sorted
937        by e.g. author or date.
938    
939    
940        Since the structure is open, it is easy to add new metadata
941        such as the DOI (Digital Object Identifier,
942        `Paskin 1999`_) of the articles to allow interoperability
943        with WWW services such as ACM's web site for, e.g.,
944        seeing new articles not yet entered to the system by the user
945        that cite the current article. This would be a good
946        way to interface our system to the Semantic Web `(Berners-Lee 1998)`_.
947    
948    
949  .. _`Section 5`:  .. _`Section 5`:
950    
# Line 885  The VIKI system is able to find relation Line 991  The VIKI system is able to find relation
991  structures based on spatial placements  structures based on spatial placements
992  of objects (`Marshall and Shipman 1993`_, `Marshall et al 1994`_).  of objects (`Marshall and Shipman 1993`_, `Marshall et al 1994`_).
993    
994    The approach in VIKI was later extended to non-linear
995    views `(Shipman et al 1999)`_ and, in VKB, to support other kinds
996    of media and navigational links `(Shipman et al 2001)`_
997    and an agent approach `(Shipman et al 2002)`_.
998    
999  `Haake et al (1994)`_ separate the dimensions of "user defines"  `Haake et al (1994)`_ separate the dimensions of "user defines"
1000  and "system uses" for how object types are embodied in a system.  and "system uses" for how object types are embodied in a system.
1001  The item-based systems are delocalized over this graph, just  The item-based systems are delocalized over this graph, just
# Line 892  as their Dolphin system is: the types ar Line 1003  as their Dolphin system is: the types ar
1003  For items, the types of the *associations* are more important  For items, the types of the *associations* are more important
1004  than the types of the items themselves.  than the types of the items themselves.
1005    
1006    A different kind of flexible structure, the Frame-Axis model,
1007    which focuses on structure that is based on *attribute values*
1008    of items, is
1009    introduced in `(Masuda et al 1994)`_.
1010    
1011    A non-spatial approach to coexistence of strongly typed
1012    and weakly typed information by using a special Semi Frame
1013    object type is given in _`(Furtado and Madeira 1998).
1014    A similar approach which allows *incremental formalization*
1015    of data is given in _`(Shipman and McCall 1999)`.
1016    
1017    Recently, `Kim (2002)`_ proposed to use a graph-based model
1018    as the basis for Engelbart's proposed Open Hyperdocument System,
1019    to allow several underlying data structures to be combined.
1020    This is essentially a proposal for using a hyperstructure,
1021    even though Kim does not discuss visualization.
1022    
1023    
1024  5.2 Structural computing  5.2 Structural computing
1025  ------------------------  ------------------------
# Line 948  which show the same kind of information Line 1076  which show the same kind of information
1076  RDF vocabularies, a data conversion tool could convert the  RDF vocabularies, a data conversion tool could convert the
1077  information into the vocabularies expected by either view.  information into the vocabularies expected by either view.
1078    
1079    Also, schema languages developed for the Semantic Web
1080    could find applications in Fenfire. For example, if
1081    the system knows that every person has an address, it can
1082    automatically create an address node when a person node is
1083    created; the user then only needs to "fill in the blanks."
1084    
1085    
1086    5.4 Visualizations of graph structures
1087    --------------------------------------
1088    
1089    Work on graph visualizations has mostly concentrated on *graph drawing*,
1090    laying out graphs on 2D planes or hyperbolic planes while minimizing
1091    crossing edges and possibly under other criteria. While graph drawing
1092    has opened several interesting problems in theoretical computer science,
1093    we are more interested in a more Focus+Context (`Furnas 1985`_)
1094    approach,
1095    due to the size of the graphs we are dealing with as well as their
1096    structure which appears to be badly suited for plane drawing, owing
1097    to the large numbers of interconnections.
1098    
1099    Other RDF visualization packages we have found are
1100    of the graph drawing type: RdfViz (XXX) and IsaViz (XXX), both based
1101    on the same 2D plane graph-drawing package (GraphViz, XXX) ,
1102    and
1103    Ontorama (`Eklund 2002`_)
1104    uses a hyperbolic-like planar visualization.
1105    
1106    Work on structure-based focus+context visualizations of graphs was
1107    more popular with semantic networks in the late '80s, for
1108    example in the Cyc (XXXCite) and SemNet (XXXCite) projects, which allowed the user
1109    to navigate through the graph locally, with one node as the *focus*,
1110    and showing the neighbouring nodes as context.
1111    
1112    Later, a similar style of navigation and structural
1113    editing was also used in the commercial
1114    product TheBrain(XXXCite), to allow users to organize files
1115    and items on their computers.
1116    
1117    One of the most important things that is different in the visualizations
1118    presented in this article and the ones for example shown in
1119    (`Utting and Yankelovich 1989`_) is that icons are not used
1120    to represent the neighborhood of an item or a document.
1121    In FenPDF, the academic articles are never represented by
1122    anything other than their image - no file icon, no title
1123    but just the text image itself. With the unique backgrounds, which
1124    allow the user to recognize fragments from articles they
1125    know well, this
1126    yields in our experience an extremely comfortable system to use,
1127    avoiding the representational complexity of the icons.
1128    
1129    
1130    
1131  5.5 Open Hypermedia  5.5 Open Hypermedia
1132  -------------------  -------------------
# Line 961  OHM tries to span the whole desktop by Line 1140  OHM tries to span the whole desktop by
1140  focusing on hyperlink interoperability between existing  focusing on hyperlink interoperability between existing
1141  applications (`Pearl 1989`_).  applications (`Pearl 1989`_).
1142    
1143  However, the FOHM model (`Millard et al 2000`_) is an interesting  However, Callimachus (`Tzagarakis et al 2003`_)
1144  step towards a similar approach: the FOHM model combines the different  and the FOHM model (`Millard et al 2000`_)
1145    are interesting steps
1146    towards a similar approach: they combines the different
1147  hypertext domains into a single conceptual structure.  hypertext domains into a single conceptual structure.
1148    
1149    
# Line 1024  and it will help you to structure everyt Line 1205  and it will help you to structure everyt
1205  in your computer around *them*.  in your computer around *them*.
1206    
1207  That's why we believe that this will be  That's why we believe that this will be
1208  "the next big thing." (You will not want to go back.)  "the next big thing."
1209    
1210    
1211  Acknowledgments  Acknowledgments
# Line 1085  with Nelson: Line 1266  with Nelson:
1266    treatment of PS/PDF files    treatment of PS/PDF files
1267  - the Fenfire codebase  - the Fenfire codebase
1268    
1269    We'd like to thank the anonymous reviewers for their comments.
1270  In addition, we'd like to thank (alphabetically)  In addition, we'd like to thank (alphabetically)
1271  Toni Alatalo,  Toni Alatalo,
1272    Katariina Ervasti,
1273  Tuukka Hastrup,  Tuukka Hastrup,
1274  Hermanni Hyytiälä,  Hermanni Hyytiälä,
1275  Antti-Juhani Kaijanaho,  Antti-Juhani Kaijanaho,
1276  Janne Kujala,  Janne Kujala,
1277  Matti Katila  Matti Katila,
1278    Rauli Ruohonen,
1279    Asko Soukka,
1280  and  and
1281  Asko Soukka  Kimmo Wideroos
1282  for discussions.  for discussions, and we'd like to thank Christel
1283    Fallenstein for providing us with the real-world
1284    example presented in `Section 3.3`_.
1285    
1286  This work was supported by the InBCT 2.1 project.  This work was supported by the InBCT 2.1 project.
1287    
# Line 1153  Computer-Supported Cooperative Work, 143 Line 1339  Computer-Supported Cooperative Work, 143
1339  **Furnas, G.W.,** (1985) "Generalized fisheye views".  **Furnas, G.W.,** (1985) "Generalized fisheye views".
1340  ACM CHI'86 Proceedings, 16-23.  ACM CHI'86 Proceedings, 16-23.
1341    
1342    .. _`Furnas 1997`:
1343    
1344    **Furnas, G.W.,** (1997) "Effective View Navigation".
1345    ACM CHI'97 Proceedings, 367-374
1346    
1347  .. _`Haake et al 1994`:  .. _`Haake et al 1994`:
1348    
1349  .. _`Haake et al (1994)`:  .. _`Haake et al (1994)`:
# Line 1234  Sausalito, CA). Line 1425  Sausalito, CA).
1425  Partially available online at  Partially available online at
1426  ``http://xanadu.com.au/ted/TN/PUBS/LM/LMpage.html``  ``http://xanadu.com.au/ted/TN/PUBS/LM/LMpage.html``
1427    
1428    .. _`Nelson 1995`:
1429    
1430    **Nelson, T.H.,** (1995) "The heart of connection: hypermedia unified
1431    by transclusion". Communications of the ACM, Vol. 38, No. 8.
1432    
1433  .. _`Nelson 1999a`:  .. _`Nelson 1999a`:
1434    
1435  **Nelson, T.H.,** (1999a)  **Nelson, T.H.,** (1999a)
# Line 1300  ACM Hypertext '89 proceedings, 137-146. Line 1496  ACM Hypertext '89 proceedings, 137-146.
1496  "Hypermedia and Cognition: Designing for Comprehension".  "Hypermedia and Cognition: Designing for Comprehension".
1497  CACM 38(8) August, 37-66.  CACM 38(8) August, 37-66.
1498    
1499    .. _`Tzagarakis et al 2003`:
1500    
1501    **Tzagarakis, M.,  Avramidis, D.,  Kyriakopoulou, M.,  Schraefel, M.M.C.,  Vaitis, M.,  and Christodoulakis, D.,** (2003)
1502    "Structuring primitives in the Callimachus component-based open hypermedia system".
1503    J. Netw. Comput. Appl. 26(1), 139-162.
1504    
1505  .. _`Zellweger et al 1998`:  .. _`Zellweger et al 1998`:
1506    
1507  .. _`Zellweger et al (1998)`:  .. _`Zellweger et al (1998)`:

Legend:
Removed from v.1.183  
changed lines
  Added in v.1.184

savannah-hackers-public@gnu.org
ViewVC Help
Powered by ViewVC 1.1.26