579 |
|
|
580 |
Fenfire is a free software project aiming at implementing |
Fenfire is a free software project aiming at implementing |
581 |
the applitude-oriented user interface concepts on top of an RDF graph. |
the applitude-oriented user interface concepts on top of an RDF graph. |
582 |
|
Fenfire is a research prototype. |
583 |
|
|
584 |
4.1 Structures |
4.1 Structures |
585 |
-------------- |
-------------- |
586 |
|
|
587 |
Fenfire provides two intertwingled structures for applitudes: |
Fenfire provides two interwoven structures for applitudes: |
588 |
RDF (`Lassila and Swick 1999`_) |
RDF (`Lassila and Swick 1999`_) |
589 |
and Xanalogical referential fluid media (`Nelson 1999c`_). |
and Xanalogical referential fluid media (`Nelson 1999c`_). |
590 |
|
|
632 |
relationships at every time: Depending on the task at hand, |
relationships at every time: Depending on the task at hand, |
633 |
the user can "switch" individual relationship types on and off. |
the user can "switch" individual relationship types on and off. |
634 |
|
|
635 |
|
This is entirely different from existing |
636 |
|
RDF visualization tools, which create |
637 |
|
a fixed 2D layout for a graph (sometimes shown in a distorted, |
638 |
|
hyperbolic-like view), and then only allow |
639 |
|
the user to zoom and scroll through it. |
640 |
|
|
641 |
|
If there are too many connections along the active |
642 |
|
relationship types, we show only as many as fit on the screen |
643 |
|
and allow the user to scroll through the list. To make this fast |
644 |
|
even for very large lists, we plan to employ fisheye sampled |
645 |
|
lists (`Furnas 1997`_), in which every element of |
646 |
|
an n-elememnt list can be reached with O(log(n)) clicks. |
647 |
|
|
648 |
RDF diagrams usually label nodes with the string that is |
RDF diagrams usually label nodes with the string that is |
649 |
used internally to identify the node. This is not acceptable |
used internally to identify the node. This is not acceptable |
650 |
for showing a network of items; items need to be labeled |
for showing a network of items; items need to be labeled |
713 |
Figure 7: a diagram of buoy animation (`larger image`__). |
Figure 7: a diagram of buoy animation (`larger image`__). |
714 |
The "screenshots" with black edges represent |
The "screenshots" with black edges represent |
715 |
keyframes, and the others the animation. |
keyframes, and the others the animation. |
716 |
|
The connecting lines between the focus and the buoys have |
717 |
|
been emphasized in this diagram; in real life, they are |
718 |
|
translucent. |
719 |
|
|
720 |
__ buoysMotion.png |
__ buoysMotion.png |
721 |
|
|
731 |
to the orange document is trivial without |
to the orange document is trivial without |
732 |
an explicit "back button". |
an explicit "back button". |
733 |
|
|
734 |
|
Also, buoys move automatically when the document |
735 |
|
they are connected to is scrolled. Buoys are arranged |
736 |
|
on an ellipse around the viewport, so that the focused |
737 |
|
part of the document, inside the ellipse, is not obscured. |
738 |
|
Additionally, only buoys whose anchors are visible are shown, |
739 |
|
and the buoys whose anchors are close to the focus |
740 |
|
are shown larger. This way, the largest buoys are those |
741 |
|
connected to things at the focus of the view. |
742 |
|
|
743 |
The intent of buoys is to reduce the cognitive overhead |
The intent of buoys is to reduce the cognitive overhead |
744 |
of browsing and editing the structure |
of browsing and editing the structure |
771 |
4.3 Libvob |
4.3 Libvob |
772 |
---------- |
---------- |
773 |
|
|
774 |
Libvob is a subproject of Fenfire, providing a flexible user-interface |
We have suggested that users should be able to add |
775 |
toolkit with some novel features that allow the independent |
connections to views programmed by somebody else. |
776 |
applitudes to work on the same user interface. |
For example, when showing meetings on a timeline, |
777 |
|
it should be easy to add buoys showing the participants |
778 |
Instead of a static structure of widgets or a scene graph, |
of each meeting. To allow such connections to be added |
779 |
the scene is regenerated each time the user presses a key [#libvob-speed]_. |
without changing the view's code, we have developed |
780 |
|
Libvob, a flexible user-interface toolkit. |
781 |
The scene consists of a DAG of coordinate systems, |
|
782 |
a keyed tree for *identifying* coordinate |
Libvob provides functionality to not only construct |
783 |
systems and a set of renderable objects (vobs) placed into |
a scene graph (i.e., specify what to draw on the screen) |
784 |
one or more coordinate systems each. |
but to also specify which parts of the scene graph |
785 |
|
correspond to which objects in the application model, |
786 |
This structure allows |
the RDF graph in the case of Fenfire. |
787 |
|
|
788 |
1) Automatically generated animations between successively |
The timeline view, for example, would specify which |
789 |
generated scenes: the coordinate systems are matched |
parts of the scene graph represent each event drawn |
790 |
to each other based on the keyed tree |
(events being nodes in the RDF graph). A different view, |
791 |
|
programmed independently, can iterate through these nodes |
792 |
2) More importantly, **post-processing** of the view generated |
and connect buoys to each of them. Through the association |
793 |
by other applitudes -- applitudes can **add** new coordinate |
with part of the scene graph, it knows *where* to connect |
794 |
systems and vobs on the view, based on what is already shown. |
the buoys to. |
795 |
|
|
796 |
The buoys are added this way: a connector object goes through |
Libvob also uses a directed acyclic graph as the scene graph, |
797 |
a generated vobscene and adds buoys to the nodes in it - regardless |
rather than the conventional tree; i.e., its graphical model |
798 |
of who placed those nodes in there. |
is not hierarchical like that of other toolkits. This provides |
799 |
|
natural support for transpointing windows (`Nelson 1995`_), |
800 |
|
i.e., connections between different parts of the view hierarchy: |
801 |
|
Showing a connection between two different windows is just |
802 |
|
as simple as showing a box inside a single window, the only |
803 |
|
difference being that the connection has *two* parents |
804 |
|
in the scene graph instead of only one. |
805 |
|
|
806 |
|
This is necessary for the implementation of buoys, which |
807 |
|
are an instance of transpointing windows (a connection |
808 |
|
is drawn between the buoy and the part of the main view |
809 |
|
it is related to). |
810 |
|
|
811 |
|
|
812 |
4.4 An example applitude combining multiple structures: FenPDF |
4.4 An example applitude combining multiple structures: FenPDF |
813 |
-------------------------------------------------------------- |
-------------------------------------------------------------- |
814 |
|
|
815 |
FenPDF is the first concrete prototype of our architecture. |
FenPDF is the first concrete prototype of our architecture, |
816 |
It is a tool to make sense of academic literature. |
using buoys, Libvob, RDF, and Xanalogical structure. |
817 |
|
It is a tool for making sense of academic literature. |
818 |
|
|
819 |
.. figure:: fenpdf-shot-small.png |
.. figure:: fenpdf-shot-small.png |
820 |
|
|
824 |
|
|
825 |
__ fenpdf-shot.png |
__ fenpdf-shot.png |
826 |
|
|
827 |
|
FenPDF is used to structure a set of articles |
828 |
|
in PostScript or PDF format. Users can transclude |
829 |
|
pieces of articles onto *spatial canvases*: infinite, |
830 |
|
scrollable papers. |
831 |
|
Transclusions are automatically bidirectionally connected |
832 |
|
to the article they are from; a buoy shows a shrunk version |
833 |
|
of the article, and clicking on the buoy brings the article |
834 |
|
to the center for the user to read. |
835 |
|
|
836 |
|
Additionally, the user can type text onto the canvases, |
837 |
|
and link two pieces of text on different canvases |
838 |
|
(linked canvases are shown as buoys). |
839 |
|
|
840 |
In the figure, there are two foci. |
In the figure, there are two foci. |
841 |
The **upper focus** shows **a PDF article** |
The **upper focus** shows **a PDF article** |
842 |
and its buoys there show the places of |
and its buoys there show the places of |
851 |
(`Kujala and Lukka 2003`_) to endow them *identity* |
(`Kujala and Lukka 2003`_) to endow them *identity* |
852 |
in the eyes of the viewer. |
in the eyes of the viewer. |
853 |
|
|
854 |
|
We use FenPDF collaboratively in our research group |
855 |
|
(synchronizing the RDF graph through CVS). |
856 |
|
In practice, we have: |
857 |
|
|
858 |
|
- A canvas for each source (for example, conference, |
859 |
|
journal issue), with transclusions of the titles |
860 |
|
and author lists of the articles published there. |
861 |
|
- Canvases for different topics, such as open hypermedia, |
862 |
|
spatial hypertext, and so on. These canvases |
863 |
|
contain transclusions of particularly relevant parts |
864 |
|
of articles, allowing us to collect the central ideas |
865 |
|
from several different articles. |
866 |
|
- Canvases for each article we are working on, |
867 |
|
containing notes and transclusions from important |
868 |
|
references. |
869 |
|
- Canvases for communicating specific ideas. These |
870 |
|
contain Memex-like "trails" of transclusions from |
871 |
|
different articles, intersersed with text discussing these. |
872 |
|
- A central canvas that has links to the other canvases. |
873 |
|
|
874 |
|
In our experience, we have found the transclusion facility |
875 |
|
very useful: copying a well-chosen region of an article |
876 |
|
to the a canvas allows the user (either the one who made the |
877 |
|
transclusion, or a collaborator) reading the canvas to quickly |
878 |
|
just read the emphasized part, going back to the article |
879 |
|
for the details if desired. The ease with which the transclusions |
880 |
|
are created (simply cut and paste) has allowed users to quickly summarize |
881 |
|
the most relevant points of the article on a separate canvas for |
882 |
|
easy perusal and for contrasting with points from other articles. |
883 |
|
|
884 |
|
In FenPDF, articles, spatial canvases, transclusions |
885 |
|
of an article and pieces of text on a canvas are all |
886 |
|
represented by RDF nodes. |
887 |
On the structure side, FenPDF uses four small RDF vocabularies: |
On the structure side, FenPDF uses four small RDF vocabularies: |
888 |
|
|
889 |
- ``FF``: Associating Xanalogical referential fluid media |
- ``FF``: Associating Xanalogical referential fluid media |
892 |
This is used both for text content and transclusions |
This is used both for text content and transclusions |
893 |
from PS/PDF files. |
from PS/PDF files. |
894 |
|
|
|
|
|
895 |
- ``CANVAS2D``: Placing nodes on a canvas - as in |
- ``CANVAS2D``: Placing nodes on a canvas - as in |
896 |
spatial hypertext (`Marshall and Shipman 1995`_) |
spatial hypertext (`Marshall and Shipman 1995`_) |
897 |
|
|
909 |
no directly annotation-specific program code in FenPDF. |
no directly annotation-specific program code in FenPDF. |
910 |
The different orthogonal |
The different orthogonal |
911 |
structures combined just give the user the opportunity |
structures combined just give the user the opportunity |
912 |
to create any kinds of structure. One common pattern of use we've noticed |
to create any kinds of structure. |
913 |
is transcluding small regions from articles related to a single |
|
914 |
topic canvas. |
Explicit support for taxonomic hypertext (`Parunak 1991`_) |
915 |
|
and hierarchies is currently being planned. |
916 |
|
|
917 |
FenPDF will first be released as a separate application, |
FenPDF will first be released as a separate application, |
918 |
but it is designed to become one applitude when the Fenfire system |
but it is designed to become one applitude when the Fenfire system |
919 |
is complete. The RDF graphs written by the application |
is complete. The RDF graphs written by the application |
920 |
will be importable into Fenfire. |
will be importable into Fenfire. |
921 |
|
|
922 |
Explicit support for taxonomic hypertext (`Parunak 1991`_) |
Let us briefly explore how a user could extend |
923 |
and hierarchies is currently being planned. |
this applitude using the techniques presented in |
924 |
|
`Section 2`_. |
925 |
|
|
926 |
|
First, metadata about articles, such as author |
927 |
|
and publication date, could be represented through simple |
928 |
|
RDF connections. Using our RDF views, one could then |
929 |
|
for example browse the list of articles of one particular author, |
930 |
|
sorted by date. |
931 |
|
|
932 |
|
Similarly, it is possible to switch from spatial hypertext (`Marshall and Shipman 1995`_) |
933 |
|
to taxonomic hypertext (`Parunak 1991`_) by replacing the canvases |
934 |
|
categorizing articles by source with RDF metadata, |
935 |
|
and browse the articles from one source related |
936 |
|
to certain subject matters sorted |
937 |
|
by e.g. author or date. |
938 |
|
|
939 |
|
|
940 |
|
Since the structure is open, it is easy to add new metadata |
941 |
|
such as the DOI (Digital Object Identifier, |
942 |
|
`Paskin 1999`_) of the articles to allow interoperability |
943 |
|
with WWW services such as ACM's web site for, e.g., |
944 |
|
seeing new articles not yet entered to the system by the user |
945 |
|
that cite the current article. This would be a good |
946 |
|
way to interface our system to the Semantic Web `(Berners-Lee 1998)`_. |
947 |
|
|
948 |
|
|
949 |
.. _`Section 5`: |
.. _`Section 5`: |
950 |
|
|
991 |
structures based on spatial placements |
structures based on spatial placements |
992 |
of objects (`Marshall and Shipman 1993`_, `Marshall et al 1994`_). |
of objects (`Marshall and Shipman 1993`_, `Marshall et al 1994`_). |
993 |
|
|
994 |
|
The approach in VIKI was later extended to non-linear |
995 |
|
views `(Shipman et al 1999)`_ and, in VKB, to support other kinds |
996 |
|
of media and navigational links `(Shipman et al 2001)`_ |
997 |
|
and an agent approach `(Shipman et al 2002)`_. |
998 |
|
|
999 |
`Haake et al (1994)`_ separate the dimensions of "user defines" |
`Haake et al (1994)`_ separate the dimensions of "user defines" |
1000 |
and "system uses" for how object types are embodied in a system. |
and "system uses" for how object types are embodied in a system. |
1001 |
The item-based systems are delocalized over this graph, just |
The item-based systems are delocalized over this graph, just |
1003 |
For items, the types of the *associations* are more important |
For items, the types of the *associations* are more important |
1004 |
than the types of the items themselves. |
than the types of the items themselves. |
1005 |
|
|
1006 |
|
A different kind of flexible structure, the Frame-Axis model, |
1007 |
|
which focuses on structure that is based on *attribute values* |
1008 |
|
of items, is |
1009 |
|
introduced in `(Masuda et al 1994)`_. |
1010 |
|
|
1011 |
|
A non-spatial approach to coexistence of strongly typed |
1012 |
|
and weakly typed information by using a special Semi Frame |
1013 |
|
object type is given in _`(Furtado and Madeira 1998). |
1014 |
|
A similar approach which allows *incremental formalization* |
1015 |
|
of data is given in _`(Shipman and McCall 1999)`. |
1016 |
|
|
1017 |
|
Recently, `Kim (2002)`_ proposed to use a graph-based model |
1018 |
|
as the basis for Engelbart's proposed Open Hyperdocument System, |
1019 |
|
to allow several underlying data structures to be combined. |
1020 |
|
This is essentially a proposal for using a hyperstructure, |
1021 |
|
even though Kim does not discuss visualization. |
1022 |
|
|
1023 |
|
|
1024 |
5.2 Structural computing |
5.2 Structural computing |
1025 |
------------------------ |
------------------------ |
1076 |
RDF vocabularies, a data conversion tool could convert the |
RDF vocabularies, a data conversion tool could convert the |
1077 |
information into the vocabularies expected by either view. |
information into the vocabularies expected by either view. |
1078 |
|
|
1079 |
|
Also, schema languages developed for the Semantic Web |
1080 |
|
could find applications in Fenfire. For example, if |
1081 |
|
the system knows that every person has an address, it can |
1082 |
|
automatically create an address node when a person node is |
1083 |
|
created; the user then only needs to "fill in the blanks." |
1084 |
|
|
1085 |
|
|
1086 |
|
5.4 Visualizations of graph structures |
1087 |
|
-------------------------------------- |
1088 |
|
|
1089 |
|
Work on graph visualizations has mostly concentrated on *graph drawing*, |
1090 |
|
laying out graphs on 2D planes or hyperbolic planes while minimizing |
1091 |
|
crossing edges and possibly under other criteria. While graph drawing |
1092 |
|
has opened several interesting problems in theoretical computer science, |
1093 |
|
we are more interested in a more Focus+Context (`Furnas 1985`_) |
1094 |
|
approach, |
1095 |
|
due to the size of the graphs we are dealing with as well as their |
1096 |
|
structure which appears to be badly suited for plane drawing, owing |
1097 |
|
to the large numbers of interconnections. |
1098 |
|
|
1099 |
|
Other RDF visualization packages we have found are |
1100 |
|
of the graph drawing type: RdfViz (XXX) and IsaViz (XXX), both based |
1101 |
|
on the same 2D plane graph-drawing package (GraphViz, XXX) , |
1102 |
|
and |
1103 |
|
Ontorama (`Eklund 2002`_) |
1104 |
|
uses a hyperbolic-like planar visualization. |
1105 |
|
|
1106 |
|
Work on structure-based focus+context visualizations of graphs was |
1107 |
|
more popular with semantic networks in the late '80s, for |
1108 |
|
example in the Cyc (XXXCite) and SemNet (XXXCite) projects, which allowed the user |
1109 |
|
to navigate through the graph locally, with one node as the *focus*, |
1110 |
|
and showing the neighbouring nodes as context. |
1111 |
|
|
1112 |
|
Later, a similar style of navigation and structural |
1113 |
|
editing was also used in the commercial |
1114 |
|
product TheBrain(XXXCite), to allow users to organize files |
1115 |
|
and items on their computers. |
1116 |
|
|
1117 |
|
One of the most important things that is different in the visualizations |
1118 |
|
presented in this article and the ones for example shown in |
1119 |
|
(`Utting and Yankelovich 1989`_) is that icons are not used |
1120 |
|
to represent the neighborhood of an item or a document. |
1121 |
|
In FenPDF, the academic articles are never represented by |
1122 |
|
anything other than their image - no file icon, no title |
1123 |
|
but just the text image itself. With the unique backgrounds, which |
1124 |
|
allow the user to recognize fragments from articles they |
1125 |
|
know well, this |
1126 |
|
yields in our experience an extremely comfortable system to use, |
1127 |
|
avoiding the representational complexity of the icons. |
1128 |
|
|
1129 |
|
|
1130 |
|
|
1131 |
5.5 Open Hypermedia |
5.5 Open Hypermedia |
1132 |
------------------- |
------------------- |
1140 |
focusing on hyperlink interoperability between existing |
focusing on hyperlink interoperability between existing |
1141 |
applications (`Pearl 1989`_). |
applications (`Pearl 1989`_). |
1142 |
|
|
1143 |
However, the FOHM model (`Millard et al 2000`_) is an interesting |
However, Callimachus (`Tzagarakis et al 2003`_) |
1144 |
step towards a similar approach: the FOHM model combines the different |
and the FOHM model (`Millard et al 2000`_) |
1145 |
|
are interesting steps |
1146 |
|
towards a similar approach: they combines the different |
1147 |
hypertext domains into a single conceptual structure. |
hypertext domains into a single conceptual structure. |
1148 |
|
|
1149 |
|
|
1205 |
in your computer around *them*. |
in your computer around *them*. |
1206 |
|
|
1207 |
That's why we believe that this will be |
That's why we believe that this will be |
1208 |
"the next big thing." (You will not want to go back.) |
"the next big thing." |
1209 |
|
|
1210 |
|
|
1211 |
Acknowledgments |
Acknowledgments |
1266 |
treatment of PS/PDF files |
treatment of PS/PDF files |
1267 |
- the Fenfire codebase |
- the Fenfire codebase |
1268 |
|
|
1269 |
|
We'd like to thank the anonymous reviewers for their comments. |
1270 |
In addition, we'd like to thank (alphabetically) |
In addition, we'd like to thank (alphabetically) |
1271 |
Toni Alatalo, |
Toni Alatalo, |
1272 |
|
Katariina Ervasti, |
1273 |
Tuukka Hastrup, |
Tuukka Hastrup, |
1274 |
Hermanni Hyytiälä, |
Hermanni Hyytiälä, |
1275 |
Antti-Juhani Kaijanaho, |
Antti-Juhani Kaijanaho, |
1276 |
Janne Kujala, |
Janne Kujala, |
1277 |
Matti Katila |
Matti Katila, |
1278 |
|
Rauli Ruohonen, |
1279 |
|
Asko Soukka, |
1280 |
and |
and |
1281 |
Asko Soukka |
Kimmo Wideroos |
1282 |
for discussions. |
for discussions, and we'd like to thank Christel |
1283 |
|
Fallenstein for providing us with the real-world |
1284 |
|
example presented in `Section 3.3`_. |
1285 |
|
|
1286 |
This work was supported by the InBCT 2.1 project. |
This work was supported by the InBCT 2.1 project. |
1287 |
|
|
1339 |
**Furnas, G.W.,** (1985) "Generalized fisheye views". |
**Furnas, G.W.,** (1985) "Generalized fisheye views". |
1340 |
ACM CHI'86 Proceedings, 16-23. |
ACM CHI'86 Proceedings, 16-23. |
1341 |
|
|
1342 |
|
.. _`Furnas 1997`: |
1343 |
|
|
1344 |
|
**Furnas, G.W.,** (1997) "Effective View Navigation". |
1345 |
|
ACM CHI'97 Proceedings, 367-374 |
1346 |
|
|
1347 |
.. _`Haake et al 1994`: |
.. _`Haake et al 1994`: |
1348 |
|
|
1349 |
.. _`Haake et al (1994)`: |
.. _`Haake et al (1994)`: |
1425 |
Partially available online at |
Partially available online at |
1426 |
``http://xanadu.com.au/ted/TN/PUBS/LM/LMpage.html`` |
``http://xanadu.com.au/ted/TN/PUBS/LM/LMpage.html`` |
1427 |
|
|
1428 |
|
.. _`Nelson 1995`: |
1429 |
|
|
1430 |
|
**Nelson, T.H.,** (1995) "The heart of connection: hypermedia unified |
1431 |
|
by transclusion". Communications of the ACM, Vol. 38, No. 8. |
1432 |
|
|
1433 |
.. _`Nelson 1999a`: |
.. _`Nelson 1999a`: |
1434 |
|
|
1435 |
**Nelson, T.H.,** (1999a) |
**Nelson, T.H.,** (1999a) |
1496 |
"Hypermedia and Cognition: Designing for Comprehension". |
"Hypermedia and Cognition: Designing for Comprehension". |
1497 |
CACM 38(8) August, 37-66. |
CACM 38(8) August, 37-66. |
1498 |
|
|
1499 |
|
.. _`Tzagarakis et al 2003`: |
1500 |
|
|
1501 |
|
**Tzagarakis, M., Avramidis, D., Kyriakopoulou, M., Schraefel, M.M.C., Vaitis, M., and Christodoulakis, D.,** (2003) |
1502 |
|
"Structuring primitives in the Callimachus component-based open hypermedia system". |
1503 |
|
J. Netw. Comput. Appl. 26(1), 139-162. |
1504 |
|
|
1505 |
.. _`Zellweger et al 1998`: |
.. _`Zellweger et al 1998`: |
1506 |
|
|
1507 |
.. _`Zellweger et al (1998)`: |
.. _`Zellweger et al (1998)`: |