7/28/2019 215814- Chapter 13

1/62

Chapter 13Documents, Hypertext andMHEGA documenfconsistsof a set of structural information that can be different formsof media, and during presentation can be generated or recorded [App90]. A docu-ment is aimed at the perception of a human, and is accessibleor computer process-ing.

13.1 DocumentsA multimed,ia document is a document which is comprised of information codedin at least one continuous (time-dependent) medium and in one discrete (time-independent) medium. Integration of the different media is given through a closerelation between nformation units. This is also called synchronization. A. multime-dia document is closelyrelated to its environment of tools, data abstractions,basicconcepts and document architecture.Currently, continuous and discrete data are processeddifferently: text is processedwithin an editor program as a type of a programming language (namely the TypeCharacter); a motion picture can be manipulated with the same editor programonly through library calls. The goal of abstracting multimedia data is to achieve

481

7/28/2019 215814- Chapter 13

2/62

482 CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEGintegrated, i.e., uniform, description and processing of all media. This reducesthe complexity of program generation and maintenance that process multimediadata. Since Chapter 16 discusses n detail different approaches o programmingabstractions, we will not concentratehere on this issue. Abstractions of multimediadata serveas the fundamental building block for programming different multimediaapplications, especiallyeditors and other document processing ools.Basic system conceptsfotdocument processinguse multimedia abstractions and a,lsoserve as concepts or the information architecture in a document. Thus, we use theterms document architecture and information architecture nterchangeably.

13.1.1 Document ArchitectureExchanging documentsentails exchanging he document content as well as the doc-ument structure. This requires hat both documentshave the same document archi-tecture. The current standardized, respectively in the progressof standardization,architecturesare Ihe Standard Generalized,Markup Language SGMI) andthe OpenDocument Architecture (ODA). There are also proprietary document architectures,such as DEC's Document Content Architecture (DCA) and IBM's Mired ObjeclDocument ContentArchitecture MO :DCA).Information architectures use their data abstractions and concepts. A documentarchitecture describes he connections among the individual elements representedas mod,els e.g., presentation model, manipulation model). The elements in thedocument architectureand their relations are shown n Figure 13.1. Figure 13.2shows a multimedia document architecture including relations between individualdiscretemedia units and continuousmedia units.The manipulation model describesall the operations allowed for creation, changeand deletion of multimedia information. The representationmodel defines: (1) theprotocols or exchanging this information among different computers; and, (2) theformats for storing the data. It includes the relations between the individual infor-mation elementswhich need o be consideredduring presentation. It is important tomention that an architecture may not include all describedproperties, respectivelymodels.

7/28/2019 215814- Chapter 13

3/62

13.1.DOCUMENTS

Figure I3.I; Document architecture and, ts elements.13.L.2 Manipulat ion of Mult imedia DataThe user becomesmost aware of multimedia documents through tools for manip-ulati,onof multimedia data, such as editors, desktoppublishing progranlsand othertert processingprograrns.A document undergoes he processshown n Figure 13.3. The information includedin a document belongs to a certain document type, e.g., a business etter or an in-ternal memorandurn. The same document can belong to other types which mainlyinfluence the final representation. The transformation from the actual informationto its final representationbehavesaccording to rules specific to the document archi-tecture.The processingcycles (Figure 13.3) of a traditional document and an interactivemultimedia presentation are analogous, as shown in Pigure 13.4. Currently, anauthor edits a document with a text editor. Thus, he or she uses he system'scharacter set (e.g., ASCII) as the actual content of a document, as well as a hid-den language available in most interactive editors for structural description (e.g.,

483

Representation Model

7/28/2019 215814- Chapter 13

4/62

CHAPTER 13. DOCUMENTS. HYPERTEXT AND MHEG

Figure 13.2: A multimedia documentarchitectureand, ,ts constituent elements.

SGMI). At this point, the document exists n a processableepresentation. Thesubsequent ormatting processdetermines the layout of the document. The resultis a final representation of the document. A typical example of this representationis the typesetting language PostScripirM. The availability of hypertext and mul-timedia technology have changed the representation of documents. Although theprocessingcycle of document generation will remain the same, it is apparent thatthere will be major changes n how documents are displayed. The output of inter-active hypermedia documents wili be mostly computer-supported. Therefore, thepresentation of a document will have to be not only fi,nal,but also erecutable.Whilethere are a broad range of processableormats, there are too few final representationformats. It has been nternationally recognized hat such a final representation (ex-change ormat) is very important, especially n a distributed, heterogeneous ystemenvironment. This exchange ormat for interactive multimedia presentation s calledMHEG (Multimedia and Hypermedi,a Information Coding Erpert Group).Using the main concepts of hypermediaand hypertert for multimedia documents,the following sections of this chapter present the document architectures SGMI

Manipulatiorlresentation

Representation Model

7/28/2019 215814- Chapter 13

5/62

13.2. HYPERTEXT AATDHYPERMEDIA

Processing

Figure I3.3: Processingof a document: from the information to the presentation

and ODA. Finally, MHEG is briefly described.

L3.2 Hypertext and HypermediaCommunication reproducesknowledge stored n the human brain via severa,lmedia.Documents are one method of transmitting information. Reading a document is anact of reconstructingknowledge. In an ideal case,knowledge ransmissionstarts withan author and ends with a reconstruction of the same deas by a reader. InformationIoss s minimal. Figure 13.5 shows this communication processbetween an authorand a reader.Today's ordinary documents (excludinghypermedia), with their linear form, supportneither the reconstruction of knowledge,nor simplify its reproduction. Knowledgemust be artificially serializedbefore the actual exchange. Hence, t is transformedinto a linear document and the structural information is integrated into the actualcontent. In the caseof hypertext and hypermedia, a graphical structure is possible

485

SourceDocument

TyPe

7/28/2019 215814- Chapter 13

6/62

CHAPTER 13,DocumentSystems

DOCUMENTS, HYPERTEXT AND MHEGInteractiveyper-/

Multimediayslems

ProcessableForm

Figure I3.4: Problem description.

Figure L3.5: Information transmission [G590].in a document which may simplify the writing and reading processes.

13.2.1 Hypertext, Hypermedia and Mult imediaA book or an article on a paper has a given structure and is represented n a se-quential form. Although it is possible o read individual paragraphswithout readingprevious paragraphs,authors mostiy assume a sequential reading. Therefore manyparagraphs refer to previous learning in the document. Novels, as well as movies,for example, always assume a pure sequential reception. Scientific literature can

FinalForm

SuiptlX,/*Ic0mpose

HyTimeScript*/

- l l - l-l l/^J I- - _ l - | \44A^

7/28/2019 215814- Chapter 13

7/62

13.2. HYPERTEXT AND HYPERMEDIA 487consist of independent chapters, although mostly a sequential reading is assumed.Technical documentation (e.g., manuals) consistsoften of a collection of relativelyindependent nformation units. A lexicon or referencebook about the Airbus, forexample, is generated by several authors and always only parts are read sequen-tially. There also exist many cross references n such documentations which lead tomultiple searchesat different places for the reader. Here, an electronic help facility,consisting of information links, can be very significant.Figure 13.6 shows an example of such a link. The arrows point to such a relation

Figure 13.6: Hypertert d,ata. An exampleof linking information of d,ifferentmedia.

between he information units (tDUs). In a text (top left in the figure), a referenceto the landing properties of aircrafts is given. These properties are demonstratedthrough a video sequence bottom left in the figure). At another place in the text,salesof landing rights for the whole USA are shown (this is visualized n the form ofa map, using graphics - bottom right in the figure). Further information about theairlines with their landing rights can be made visible graphically through a selectionof a particular city. A special nformation about the number of the di fferent airplanessold with landing rights in Washington is shown at the top right in the figure with

7/28/2019 215814- Chapter 13

8/62

488 CHAPTER 13. DOCUMENTS. HYPERTEXT AND MHEGa bar diagram. Internally, the diagram information is presented n table form. Theleft bar points to the plane, which can be demonstrated with a video clip.

Non-linear Information Chain

Hypertext and hypermedia have as a major property a non-linear inJormation linlc.There exists not only a reading sequence,but also the reader decideson his/herreading path. The reader can start in a lexicon with a notion hypertert, then gothrough a cross eference o systemsand finish with a description of AppleTalk. Bythis association, hrough reference inks, the author of the information determinesthe actual links.As another example, let us consider this document (our multimedia book) aboutmultimedia systems. The structure of this work consists,besides he introductoryand closing chapters (first impression) of a set of equiualent chapters. Actuaily,Chapter 2 has a more important position: it should be read before any of the re-maining chapters. The reason is that it includes some fundamentals to explaincommon notions used in the other chapters. All other chapters are relatively inde-pendent and the reader can determine his/her own path. The structure is a treewherethe reading path in this linear document s explained verbaily and not throughthe structure. A hypertext structure is a graph, consisting of nodes and edges.Thereferences o other chapters and literature citations are, for example, such pointerswhich build a tree-similar document to a graph.

r The nodesare the actual information units. They are, for example, the textelements, ndividual graphics, audio or video LDUs. The information unitsare shown at the user interface mostly in their own windows.

o The edgesprovide links to other information units. They are usually calledpointers or links. A pointer is mostly a directed edge and includes its owninformation too.

7/28/2019 215814- Chapter 13

9/62

13.2. HYPERTEXTA/VDHYPERMEDIAAnchor

489

The forward movement in linear sorted documents is called a nauigation throughthe graph. At the user interface, the origin of pointers must be marked, so that theuser can move to a further information unit. This origin of a pointer is called ananchor. A main factor of the user interface is the conceDtof the anchor: how canthe anchor be representedproperly?

o A media-i,ndependentepresentation can happen through the selectionof gen-eral graphical elements, such as buttons. In such an element, informationabout the destination node should be included. If the destination node is atext, a short, descriptive text of the content can be represented. In the caseof an image, the image content can appear in minimized form on the screen.A visual representation of the video content can follow in form of a mouingfcon (Micon). This is a minimized motion picture which representsa charac-teristical portion of the video sequenceof the destination node (MIT ProjectElastic Charles [Bra87]). If the content of the destination node consists ofaudio information, a visual representation of the audio content must follow.For example, n the caseof a music passage,a picture of the composer couldbe displayed.

o In a terf, individual words, paragraphs or text sectionsof different length canbe used for representation. The positioning of the pointer to the marked areaand double clicking in this area leads to a display of the destination node,connectedwith the clicked nformation (e.g.,seeFigure 13.11).

o In images,specificgraphical objects or simply areas are defined as selectionobjects. A specificmarking can occur through a color or stripe.

o In a motion uideo, media-independentrepresentationsof the anchor are pre-ferred. There can also be time-changing areas used. Mostly, no spatial selec-tion occurs and the particular shown image is conclusive. A timely selectionis supported.

o With respect o audio, a media-independent olution s used. In this case,ashort, descriptive text or an image of the sizeof an icon is preferably shown.

7/28/2019 215814- Chapter 13

10/62

490 CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEGFigure 13.7emphasizes he relation among multimedia, hypertext and hypermedia.

Figure 73.7: The hypertert, hypermed,ia nd multimedi,a relationshi,p.

Hypertext SystemA hypertert system s mainly determined through non-linear links of information.Pointers connect the nodes. The data of different nodes can be representedwithone or several media types. In a pure text system, only text parts are connected.We understand hypertert as an information object which includes links to severalmedia.

Multimedia SystemA multimedia systemcontains, according to Section2.3, information which is codedat least in a continuous and discretemedium.For example, if only links to text data are present, then this is not a multimediasystem. It is a hypertert. A video conference,with simultaneous transmission oftext and graphics, generated by a document processing program, is a multimediaapplication. Although it does not have any relation to hypertext and hypermedia.

Hypermedia SystemAs Figure 13.7shows,a hypermediasystern ncludes the non-linear information linksof hypertext systemsand the continuousand discrete media of multimedia systems.

7/28/2019 215814- Chapter 13

11/62

13.2. HYPERTEXTA]VDHYPERMEDIAFor example, if a non-linear link consists of text and video data, then this is ahypermedia, multimedia and hypertext system.As is often the case,we will not use the notion hypermedia in its strongest sense. fnot explicitly specified hypertert and,hypermediaarc used nterchangeably.There have been many international conferencescovering this area since the late1980s: Hypertext'87, Hypertext'89, etc. A good overviewof articleschosen romthe first Hypertext'87 conference s in [ACM88]. ECHT'9O was the first EuropeanConferenceon Hypertert held in Paris. There exists a large number of conferencesand workshops, n addition to these main in ternational events,at the regional andnational levels.

L3.2.2 Hypermedia Systems: An ExampleActually, it is not easy o presenton paper a real hypermedia system. Therefore, tis urgently recommendedo the reader o work with such a system(e.9., he NCSAMosaic@ tool for viewing hypermedia documentswritten in HTML - an applicationof SGML) to get a better understanding of the properties and the advantagesanddisadvantages.The following exampleof a lecture "hypertert', as a hypermedia document, s similarto [Nie9Ob,Nie90a]. It describes he part of a typical manipulation processwith ahypermediasystem.

First ScreenA possiblynatural environment s created for the reader of a hypertext to improvisethe usual setup (Figure 13.8).Therefore,at the beginningof the lecture,a book isshown, which can be opened by clicking on it. The title of the document appearson the cover.Additional information can also be shown, such as when the reader ast opened thebook. The same nformation (after the book was opened) s only shown with respectto the individual nodes.

49r

7/28/2019 215814- Chapter 13

12/62

492 CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEG

ffiffi

Wffi ffiFigure 73.8: The f,rst screenof a hypermedia ecture.

A content-sensitiveelp function (with respect o the particular state of the system)can be made available. The help function describes he actual state and the impli-cationsof possible nteractions with the user. Further, it allows for the possibility toaccess eneralhelp information. An application of hypertext, which is very popular,consistsof such system-wide help functions. Here, the text medium is most oftenused.

Second ScreenUpon opening the book (Figure 13.9), the reader is presentedwith an overviewof the document in the form of a two-dimensional content directory. There is nofirst chapter. Besides he linking nodes with edges i.e., the information units withpointers), the physicalordering of the nodesamong eachother can provide additionalinformation. Hence, chapters that are closely elated to each other can be presentedon the screennear each other. The author can express semantic reiations throughthe layout. For example, if a document describesODA, SGMI and hypertext,

7/28/2019 215814- Chapter 13

13/62

13.2, HYPERTEXTAJVDHYPERMEDIA 493

WWWW ffi

Figure I3.9: The secondscreen- fi,rst-Ieuelcontent directory.

(1) ODA and SGML can be placed close ogether during the presentation of bothdocument architectures, and (2) hypertert can be used n both architectures.The content directory in its most general orm is often representedby a tree graph.The nodes that have already been visited can be marked as such, which simplifiesnavigation. When clicking at a node,only a paragraph is shown becauseof overviewreasons.On one hand, if too many nodes are linked in the content directory, the reader losesthe comprehensiveview. Given a 14" screenwith VGA resolution and a contentdirectory in form of a tree, a practical number of displayednodes s 30. It is assumedthat all nodes are simultaneouslypresented. On the other hand, if too few nodes areshown (e.g., only three), then this may lead to a large number of levelswhich willcause a fragmented perspective. The optimal number of presentednodes dependson the screensize, the number of pointers and the degree of possible evels. Manynodes can be presentedas ttees, few nodes can be part of complex graphs. Mostly,content directories are trees. The content of the particular destination node refinesthe content directory. In our example (Figure 13.9), Lecture Eypertert is divided

7/28/2019 215814- Chapter 13

14/62

494 CHAPTER 13, DOCUMENTS, HYPERTEXT AND MHEGinto four sections: Applications, Systems,,ResearchAreas and Bibli,ography. Thereader selectsSystemshere.

Third Screen

The structure of the four sections on the third screencan be presented n a refinedversion. The four sectionscan be seen as buttons which the user can select.The following Systemssubtopics are offered: Requirements, lser Interface Designand a Classification Figure 13.10). In the presentation (Figure 13-10),a multilevel

WffiFigure 13.10: The third screen second-Ieuelontent directory.

content directory could be hidden, but we assume his is not the case. The user mayget information about the user interface and select his information unit. Here, thereader selectsUser Interface Design.

7/28/2019 215814- Chapter 13

15/62

13.2. HYPERTEXTAITDHYPERMEDIAFourth Screen

495

Typical information about the medium text, used in User Interface Design, s pre-sentedon the fourth screen Figure 13.11). n addition, the book's contentdirectory

Figure13.11: TheJourthscreen detailsaboutUser nterface

and the second evel are visible. The selectedpath Systems/User Interface Designis specially marked. This helps the reader to navigate through the document. Textis displayed on one side of the screen mitating an opened book. Someparts of thescreenare marked as anchors: Orientati,on, OueruiewDiagrams, Hi,storg Files andBacktraclcinq.Each of theseanchorspoints to further information. This can lead tofaster pacing without absorbing previous information. Further information aboutthe topic Orientation is available as a motion video. The user clicks on the anchorOri,entation Figure 13.11)and the screen hown n Figure 13.12appears.

User Interface Design

Resarch howedhat theusersof a

7/28/2019 215814- Chapter 13

16/62

496 CHAPTER 13. DOCUMENTS. HYPERTEXT AND MHEGFifth ScreenThe fifth screen (Figure 13.12) eads the reader to a video clip showing examplesof Orientat'ion. Here, there may aJsobe an additional program for motion picture

W$$wwffi@-L- W

Figure 73.12: The fifth screen- details about Orientation.control. This program provides functions similar to a video recorder, meaning thevideo can be moved forward and backward at different speeds,as well as stopped,and a still image can be displayed. Additionally, certain positions in the video clip(e.g., the beginning of the clip) should be made accessible.The selectionof the symbol at the lower right corner leads the reader back to theprecedingnode, in this case, o the User Interface Design screen(Figure 13.13).

Sixth ScreenThe sixth screen s a return to the User Interface Design screen.The user now selectsHistory Files. The cross-barunder the openedbook is a scroll-bar. It indicates thatthe displayed page representsonly approximately 25% of al l the information stored

7/28/2019 215814- Chapter 13

17/62

13.2. HYPERTEXT ATVDHYPERMEDIA

Figure 13.13: Hypermed,iaexarnple: sirth screen- d,etailsabout the user interface.

in the node about User Interface Design. Additional pagescan be viewed by rollingthe roll-bar or selection of the pointer.Seventh ScreenThe Hi,storyFiles screen Figure 13.14)shows he particular nodes ast read. Specif-ically, the reader read the couerpage5 minutes ago, the overview about the lectureHypertert 2 minutes ago, the chapter Systems4 minutes ago, and Lhe User Inter-face Desi,gnin the chapter systems2 minutes ago. This history list showsall of thetraversed nodes. Each node can be accessed gain by selecting an element in thelist.Often, a graphical assignment s better to keep n mind than a textual description.The user, for example) may remember that there is a picture where a red hat isplaced n the upper left corner. Alternatively, a display of the traversed nodes mayhelp the reader to find h is/her way through the document. This is not a very goodsolution, however f many screenshave been involved.

7/28/2019 215814- Chapter 13

18/62

498 CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEG

HISTORYLISTttt: :it* t:l clck on the nane or an arricle to get back to itread the article

5 min Book cover ofthe lechrre IIYPERTEXT2 min Overview of the lecture HYPERTEXT4 min ChapterOverview - Systm2 min User Interface Design in chapter Systems

Wffi

WWWc

Figure L3.74: The Seuenthscreen- history of preuiously presented nJormation.With this short, virtual navigation through a hypertext lecture, somecharacteristicproperties of a hypertext system have been shown. Various systemsuse differentconcepts, unctions and layout of the user interface.

Further Application AreasA lecture is not the only respectively typical application area. The followinghave shown to be useful domains for hypertext systems:

In someclassicalcomputer applications, hypertext is already state-of-the-art;especially, the help function.In the area of commercial applications, repair and operational manuals can befound. Here, different media are used. This technology eads to a replacementof a microfilm which can be found by the distribution of replacementparts. Arepair instruction can be presentedmuch more flexibly using motion pictures.Here, an interaction in the form of a slow forward and backward rewinding

7/28/2019 215814- Chapter 13

19/62

13.2. HYPERTEXT A.I{D HYPERMEDIA 499is expected. Exhibition and product cataloguescreate, together with otherapplications from the advertisement branch, the basis for a number of diverseapplications.The organization of ideas,brainstorming and the generation of scientific doc-uments count, for example, as intellectual applications. Here, the structureof the document is not clear at the beginning of its generation. During theintellectual processof writing a document, the structure gets clarified.Education and tutorials can be improved through the input of continuousmedia. Foreign language education requires the audio medium. In museums,further explanation of exhibition piecescan be offered to visitors using audioand video.Tourist information systemsand interactive science-fictionmoviescount on theareasof entertainment and free-time activities. A new generation of computergames s going to becomeavailable.

Hypertext provides advantageswhenever the document for a specific appl-icationdoes not require strict linear structure. However, the authors'experiences showthat hypertext will not replace all conventional print material'

13.2.3 HistoryThe history of hypertext goes quite far back, although it has been only recently(in the last couple of years) that hypertext systemscame on the market. Also, theintegration of continuous media was demonstrated n laboratories severalyears ago'In the following paragraph, we will give a short overview of hypermedia/hypertexthistory according to [Nie90b].Vannever Bush is the originator of the main hypertext concept, the linked informa-tion structure. He describedthe first hypertext system Memex (MEMory EXten-der). Memex was never implemented, it exists only on the paper. Vannever Bushdeveloped he ideas for this topic in 1932. He published the first descriptive articleas We May Think in 1945.

7/28/2019 215814- Chapter 13

20/62

500 CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEGLet us imagine all information in the form of microfilm being placed on a table.With proper projectors, areascould be displayed n similar form as using X WindowSystemTM. Here, an associatiuendcris generatedwhich links the different microfilmareas ogether (theseare the pointers).Doug Englebart developeda project to augment the human capability Augment atthe Stanford Research nstitute (SRI) 1962-1976.One part of it is NLS (oN LineSystem),which has hypertext properties. NLS servedas oint document storage orall createddocumentsduring this project. All scientistsworking on this project usedit with its possibilitiesof pointers. At the end, there were approximately 100,000entries.Ted Nelson used the notion Hgpertert for the first time in 1965. In his systemXanadu, all information which human beings describedat any time, was contained.His conceptsdescribedthe access o local, as well as to remote data. Xanadu wasnot implemented with his global information content until now.Since the middle of the 1960s, work on hypertext systems has been going on atBrown University, Providence,RI. In 1967, he Hypertext Editing Systemwasdevel-oped under the leadershipof Andries van Dam. This was the first run-able hypertextsystem. It needed 12O-Kbytemain memory of a small IBM/360 mainframe com-puter. It was sold and used for the documentation of the Apollo mission. Thesuccessor roject was 1885,9 (File Retrieual and Editing SgStem)in 1968. Bothsystems inked documents through pointers, the user interface was implementedthrough text. At Brown University from this time, successful esearch n the areaof hypertext/hypermedia has continued.The Aspen Mouie Map is probably the first important hypermedia system whichsupports continuous media too. It was developedat the MIT Architecture MachineGroup under the intensive cooperation of Andrew Lippman. This group was builtup later on with other scientists and was known as the MIT Med,ia Lab lBra87l.With this application, a virtual drive through the city Aspen (Colorado) could befollowed on the computer screen. The user could move in all four geographicaldirections as s/he desired. L jogstick served as an input of the direction. Thetechniqueconsistedof a large set of individual images which were stored on a videodisk' For this purpose, four cameraswere installed on a pick-up with the angle of

7/28/2019 215814- Chapter 13

21/62

13,2. HYPERTEXTAND HYPERMEDIA 50190o o eachother (with the view: front, back, right, left). The car drove through allstreets of the city and took one image every three meters. The imageswere linkedthrough implicit pointers: therefore, a drive through the city could be simulated.The drive was simulated with a maximal two images per secondl therefore,a speedof approximately 110 km/h could be achieved. The display occurred through twoscreens: the first screen displayed the picture of the street and the secondscreenshoweda street map with the actua,lposition.Successor rojects concentrated on the joint usageof video, individuai images andtext for bike and car repair-manuals.Until now, all hypertext systems mentioned were not developed as products andseldom wereused outside of their researchgroups. In 1982, Symbolicsstarted devel-opment of the SymbolicsDocument Eraminer. It was ready as the first hypertextproduct in 1985. Its main application was the documentation of the SymbolicsWorkstation, which was comprised of about 8,000 pages. It contained approxi-mately 10,000nodes and 23,000pointers. Also, the metaphor a boolc n the screenwas used and an emphasiswas put on a simple user interface.Since 1985, many hypertext systemshave been announced and establishedon themarket. NoteCards rom Xerox and, ntermediafrom Brown University started asresearchprojects and ended up as products. The Guide, implemented by OfficeWorkstation Limited, started asproduct development. It was the first product basedon mini computers (1986). In 1987, he Apple company presented he HyperCard. Itwas installed for free on all Macintosh computers and was therefore widely availabie.

ConeeptsHypertext systemsdiffer from each other in their fundamental concepts:

o (Jnspecifi,c ystems were not developed for any specific application. They aredetermined to be used generally for the generation and reading of hypertextdocuments. The Apple (HyperCard) product is an example.

o Application-specificsgstemswere developed or determined usage. For exam-ple, gIBIS gives explanations for political discussions. It is meant to be a

7/28/2019 215814- Chapter 13

22/62

CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEGdecision help. In gIBIS, three special nodes and nine different pointer typesexist [CB88l.

L3.2.4 Systems: Architecture, Nodes and PointersArchitectureThe architecture ofa hypertext systemcan be divided into three layers with differentfunctionalities [CG87]

o Presentation LaEerAt the upper layer, the presentation layer, all functions connected o the userinterface are embedded. Here, nodes and pointers are mapped to the userinterface. At the user interface, one or several parts of the document arevisualized. This layer determines, based on the given structure and user'sdesired display, which data are presented and how they are presented. Thislayer takes over control of all inputs.

o Hgpertert Abstract MachineThe Hypertert Abstract Machine (HAM) is placed between the presentationand storage ayers. It can expect from the underlying layer database unctionsfor storageof multimedia data in a distributed environment. It doesnot haveto consider nput and output of the upper layer. HAM knows the structureof the document, t has the knowledgeabout the pointers and its attributes.The data structure, respectively a document architecture, is constructed forthe managementof the document. This layer has the least system dependencyin comparisonto the other two layers. Therefore, t is the most suitable layerfor standardization [Nie90bl.

o StorageLayerThe storage layer (also called the database layer) is the lowest layer. Allfunctionsconnectedwith the storageof data, i.e., secondary toragemanage-ment, belong to this layer. The specific properties of the different discreteand continuous media need to be considered.Functionalities from traditional

7/28/2019 215814- Chapter 13

23/62

13.2. HYPERTEXT AI\rD HYPERMEDIA 503database systemsare expected,such as persistence data persist through pro-grams and processes),multi-useroperations synchronization, ocks, ...) andthe restoration of data after a failure (transaction). The nodes and pointersof a hypertext document are processedas data objects without any specialsemantics.

Unfortunately, in most current implementation, there is no clear division betweenthe different layers. The reasonsare: shorter development ime, efficient mplemen-tations and currently an incomplete, respectively unavailable general multimediainterface for the lowest layer,

NodesA node s an information unit (LDU) in a hypertext document. The main classifi-cation criterion of different realizations s the maximal stored data amount in onenode:

The maximal stored data amount can be limited,and mapped onto the screensize. The metaphor of a note card, respectively a frame, was introduced here(e.g., HyperCard). A video clip and audio passage ould be limited to theduration of, for example, 20 seconds.An author is forced eventually to distribute logical connectedtext content toseveralcards, although it is not desired. Applying it to video clips and audiopassages,t would mean that the close nterconnection among the distributedselluences ould get lost easily. An advantage s the overview.Window-based systems with an unlimited data amount per node are the al-ternative. Forward and backward scrolling of pages s offered analogous oother windows at the user interface. Intermedia is such a system. Here, atevery node the amount of data, coded as continuous media, is not limited (inprinciple) with respect to its duration.Therefore, individual nodes can include a very different length, although atfirst they may appear equal. Two different methods at the user interface areused or the presentation of further information: Either it is switched between

7/28/2019 215814- Chapter 13

24/62

504 CHAPTER 13, DOCUMENTS. HYPERTEXT AND MHEGthe nodes, or scrolling is used in one node with the usual mechanismsknownin window systems.

A secondarycriterion applies to the time point of information generatfon. Usually,the author specifies he whole content of the nodes during the generation of thedocument. Alternatively, the author can generate nformation according to his/herprevious input during the presentation time. This approach allows the author toinclude, for example, information about a company, also a pointer, to the actualcourseof the company'sstockson the New York stock exchange.This informationcan be requestedautomatically through the u ideotert seruiceand presented as partof the hypertext document.

PointersPointers are the edgesof a hypertext graph. Hypertext systems are classified ac-cording to different criteria with respect to edges. As a flrst question, one can ask:Which information includes a pointer?

o Simple pointers link two nodes of the graph without containing any furtherinformation. They are visible only through the relation between the nodes.

o Typedpointers include, in addition to the link between two nodes, furtherinformation. Each pointer gets a label. Through this label, commentaries tothe particular labei are possible(e.g., author and creation date).One can use further semantics. For example, in the caseof an educationalunit, the continuation of reading further detaiis could depend on the resultof an exam testing the previous details. The pointers then include a formulato activate the further reading. The formula is dependent on the result ofthe test. Also, access ights can be controlled through pointers. Anotherpossibility consistsof assigning ypes to pointers according o their properties.For example, it can be used to differentiate between the relations destinationnod,e ,s an erample and destination node is a detail. These diferent semanticscan also be expressed hrough different representationsof the anchor at theuser interface.

7/28/2019 215814- Chapter 13

25/62

13,2. HYPERTEXT AND HYPERMEDIAAnother property of the pointers is connected o the question: What d,oeshe pointermean? Often, pointers with very different meaningsare used together. This usagecomplicates he understanding. The author of a hypertext should know about thisproblem and use unambiguous pointers. The following relations can be expressedthrough pointers:

To be; A is part of B. This sentence epresentsa set relation.To present: A is an example of B, A demonstrated B.To infiuence: .4.causesB, B is a result of A. Consequencesrom a behaviorcan be describedmore closely.To needor to be needed: 4 needs B, B is neededby A. This relation expressesa necessity.

o To own: ,4 has B, A is associatedwith B. Here, ownership is expressed.To include: .4 ncludes B, .4 consistsof B, A occurs n B. An inclusion relationis expressed n diferent meanings.To be similar A is similar lo B, A is different from B, ,4 replaces B, -4 is thealternative to B. Using this relation, similarities can be expressed.

Another basicproperty of pointers is describedwith the question: Who is responsiblefor the pointer specification?There are two possibilities:

Implicit PointersA relation between nodes can be established automatically by a hypertextsystem. The author determines the algorithm according to which pointers arecreated. The system Intermedia automatically generatesall pointers whichbelong to one index. A similar approach can be taken by lexicons. Queryreferencesare done automatically using main notions of an entry.Erplicit Poi,ntersThe author creates all links bv him/herself.

505

7/28/2019 215814- Chapter 13

26/62

506 CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEGA pointer can be created at different times. The question follows: When is thedestination of a pointer specifi,ed?

o In the classicalcase, he pointer is created during the generation of a hyper-text document, and hereby the origin of the destination node is determined.The author determines explicitly the links of the information units duringdocument processing.

r A destination node can be determined first by using the pointer, i.e., duringreading. The author specifiesan algorithm for the creation of the pointers,but they are determined first during the reading depending on the context.The systemcomputes he destinationnode.An example s a travel plan which shows he trains to one speciflc destinationstation. A pointer to the next train depends on actual time. This exam-ple comesfrom the information system to the city Glasgow Glasgow Online[Har89].

In the most systems, a pointer has one source and one destination node. One canalso ask the questions: Which direction has a pointer? ar.d What is the number ofoutgoingpointers?The direction is mostly unidirectional. Backtrackino is supported by the systemitself. Hence, the path always eads back to the source. The alternative wouid beto have bidirectional pointers, but then the anchor, as well as the destination node,would have to be specially marked. Introducing bidirectional pointers, it is possibleto have multiple teferences rom severalnodes to the same destination node. Hence,thesekinds of pointers must be marked at the destination node and a further choicecriterion must exist. Most systems support unidirectional pointers with only onedestination node. It is easier o understand and generate.The last question is connected o the form of an anchor at the user interface: Howis the pointer represented,? ection 13.2.1 showsdifferent possibilities.

7/28/2019 215814- Chapter 13

27/62

13.2. HYPERTEXT AAID HYPERMEDIAToolsA hypertext systemconsistsofseveral necessary ooIs. Editor(s/ process nformationrepresentedn different media. Beside his, the generation,management,editing anddeletion of pointers are supported.Search ools allow the search of desired nformation. Also. different media need tobe considered.Browser allows a shortened but clear representation of the nodes and edges. Thenodes are described media-dependently. The structure is presented to the usermostly in a graphical representation. Often, only the previously read text or relevantinformation can be displayed.During the nauigatfon through a document, a proper support of the phenomenaGetting Lost in the Hyperspace s needed. A backtrackingand clear representationof the whole structure with respect to the actual position should be available.

13.2.5 Some Final Comments about Hypertext SystemsThe ordering during a reading of a hypertext document should be pre-specifiedwith respect to the context and reader's interest. Therefore, the structure of thedocument can change depending on the context [Hof91].For example, a textbook about the esophaguscan offer to a medical student inhis/her first semesteran overview of the organ's functionality. Only a limited setof nodes are presented. The most pointers build a tree with a suggestedpath. Anavigation through this document turns out to be simple. In the following semesters,the students are lead to specific topics in the surgery area. Using text, graphics,video and audio, different possibilities of surgical procedurescan be presented. Thehypertext document ncludes n this context, in addition to the fundamental notions,a detailed description of the surgery. Further sections n this document discussresearchaspects. Referencespoint to a number of other articles, and also cutrentunsolved researchaspects are shown. The user has the possibility of navigatingthrough the whole document.

507

7/28/2019 215814- Chapter 13

28/62

508 CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEGThe most current hypertext systems do not allow any references o nodes outsideof their internal data structure. For example, shared electronic letters cannot bestored as documents in one hypertext system, and simultaneously sent throughanother mailing program. Open soluti,ons ould be required in this case o supportlinks between information units of different hypertext applications. This requiresstandardizedexchange ormats and protocols.A hypertext document is stored mostly on one computer. In a d,istributedenui-ronrnent, information units can be located on different computers. The pointersgo beyond the computer boundaries. According to the presentedarchitecture, thestorage ayer would be mostly affected.Further interesting developmentand researchprojects refer to the following aspectsand questions:

Sizeand concept of the information units.(what size s the optimal size? which factors does this specification dependon?)support of distributed documents by migration of information and/or thereorganization of networks.(How can a document be preservedwith respect to its remotely stored con-tent?)Version management.(which elementsshould comply with versionmanagement? what should thismanagement ook like?)Authorization and access ights.(Which elementsshould be subject to an authorization and what should theiraccess ights be?)Cooperative work, joint document processing.(which accesseshould be locked? How is such a management mplemented?)

o Virtual views onto hypermedia documents.

7/28/2019 215814- Chapter 13

29/62

1.3.2.HYPERTEXT AATDHYPERMEDIA(What does he virtual view determine? How are the virtual views managed?)

Concluding, a short evaluation of the hypertext/hypermedia properties is given. Apart of this evaluation s basedon personal experiencesof the authors, further ideascome rom discussionwith experts. Many aspects,which are visible at the beginningof working with such systems,are mostly positive. Operation of most systemscanbe learned quite easily without manual. The user knows quickly and effectively howto find the desired nformation and how to manipulate all data.Many of these enumerated properties are system-dependent,but most hypertextsystemsshow the named properties. The hypertext documents themselvesare verydifferent in nature. Some are structured clearly and easy to read, others are notstructured properly, hence hey are difficult to read.Hypermedia integrates diverse media in a very elegant and simple fashion. Eachrelation between information units is implemented with pointers. Some systemsalso support the joint management of information among several persons. Thistechnique has someproperties which need to be critically evaluated. The most well-known effect s Getting Lost in the Hyperspace.While reading such a document, theperspective and context can get lost. Hypertext documents can easily become socalled spaghettibooksbeca'rsef many pointers with diferent meanings. A hypertextdocument is difficult to bring into paper form without any information loss. Hence,even without audio and video information, the reader needs a computer. Somesystemsuse their own window systems. There is a lack of establishedstandards forinformation exchangeamong current hypertext systems.Different task forces work on standards for hypermedia. Extensions of documentarchitectures ODA and SGML include hypertext techniques. They support a dataexchangeof hypertext-like documents n a heterogeneous nvironment.Further activities are embedded n ISO/IEC JTC1 SC2/WG12 Multimedia and Hy-permedia Information Coding Erpert Group (MHEG). This group works onthe codedrepresentationof multi,mediaand hypermed,ianformatfon [MHE93].The ANSI group X2Vl.8M represents he Music Information ProcessingStandards(MIPS) Committee. This committee works on HyTime with respect to hypertextand the Standaril Music Description Language (SMDL).

509

7/28/2019 215814- Chapter 13

30/62

5i0 CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEGIn summarS hypertext is not suitable for al l kinds of documents and applications.This technique is very good with lexicons. The information units can be linked toeachother through references pointers). In comparison to an accessand search na book, the usageof hypertext system makesfinding information faster.video and audio can be included in such a lexicon easily. For example, passagesfrom music or animal voicescan be stored under a certain entry's explanation. Videoallowsa short representationof typical movementprocesses spart of a lexiconentry.Another area of using hypermedia is education and tutorials. Coursework maybe supportedby audio-visualmeans. Thesemeans can be used according o thelearning behavior of the individual participants. Por the instructors, the hypertextresearchworks toward a didactical support for course design. For example, such asystemwas developedat the University of Karlsruhe [Mueg9l.

13.3 Document Architecture SGMLThe standard Generalized,Markup Language (]GML)[Gol91b], [019g6] was sup_ported mostly by American publishers.Authors prepare the text, i.e., the content.They specify in a uniform way the title, tables, etc., without a description of theactual representation e.g., script type and line distance). The publisher speciflesthe resulting layout.The basic idea is that the author uses ags for marking certain text parts. SGMLdetermines he form of tags. but it doesnot specify their location or meaning. Usergroups agreeon the meaningof the tags. SGML makesa frame availablewith whichthe user specifies he syntax description in an object-specific system. Here, classesand objects, hierarchiesof classesand objects, inheritance and the link to methods(processingnstructions)can be usedby the specification.SGML specifieshe syn-tax, but not the semantics. For example,

Mult imedia-Systems< /t i t le )< author>FeIix Gatou(/author>

7/28/2019 215814- Chapter 13

31/62

13.3.DOCUMENTARCHITECTT]RE GML(s ide) IBM( /s ide>This exceptional paper fron Peter .. .

This example shows an application of SGML in a text document.

13 .3 .1 Some Deta i lsFigure 13.15shows he processing f an SGML document. It is divided into two

Figure 13.15: SGML: Document processing from the information to the presenta-tion.processes.Only the formatter knows the meaning of the tag and it transforms thedocument into a formatted document. The parser uses the tags, occurring in thedocument, in combination with the corresponding document type. Specificationof the document structure is done with tags. Here, parts of the layout are linkedtogether. This is based on the joint context betweenthe originator of the documentand the formatter process. t is not defined through SGMI.

User GroupSpecific Definitions

Generic Mark-upDefinition(Document TypeReference)

Semanti6 ofAttribut6 TargetDocument

7/28/2019 215814- Chapter 13

32/62

5r2 CHAPTER 13. DOCUMENTS. HYPERTEXT AND MHEGTags are divided into different categories:

o The descriptiue markup (tags) describes the actual structure always in theform:

respectively also (/end-tag)

An example is the definition of a paragraph at its beginning:

< !ELEI.{ENTreanble (titte, author, side) )

7/28/2019 215814- Chapter 13

33/62

13.3.DOCUMENTARCHITECTURE GML

< I E L E M E N To d y ( . . . ) >

513

o Instructions for other programs in a text are entered through processing n-structions. They can be meant, for example, or the formatter. Using process-ing instructions, different media can be inserted.

SGML defines a syntax for tags through a grammar which needs to be followed.SGMI doesnot define he semantics f these ags.The information or documer,rt rchitectureof SGML is shown n Figure 13.16. SGMI

Figure 13.16: SGML: Document architecture- emphasison the representationmodel.with its tags possesses representation model. Objects, classes nd inheritancebe used for the definition of the structure.

Structure \

Object Irheritance. clss -/

Representationmodel

7/28/2019 215814- Chapter 13

34/62

5I4 CHAPTER 13. DOCUMENTS. HYPERTEXT AND MHEG13.3.2 SGML and Multimedia

Multirnedia data arc supported in the SGML standard only in the form of graph-ics. A graphical image as a CGM (Computer Graphics Metafile) is embedded n anSGMI document. The standard does not refer to other media [01986].

A link to concrete data can be specified through #NDATA. The data are storedmostly externally in a separate ile.The aboveexampleshows he definition of video which consistsof audio and motionpictures.Multimedia information units must be presented properly. The synchronizationbetween the components s very important here. HyTime [Gol91a] and MHEG[MHE93] work on these ssues.

7/28/2019 215814- Chapter 13

35/62

13.3,DOCUMENTARCHITECTURE GML13.3.3 Closing Comments about SGML

515

A standardizeddocument exchange s necessarywith respect to the communication.Sender writer) and receiver(reader) can be distributed in t ime, as well as n space.Often, documents are processedautomatically. This requires a joint context. Thesyntax is transmitted and the semanticsmust be discussed n SGML separately.The Document Type Definitions form the basis for these discussion.SGML as a standard will stay in its current form [01986],but extensions(additions)are also worked out. A standardized layout semantics s necessary. This will sim-plify interactions of user groups. The Document Style Semanti,csand SpecificationLanguage (DSSSL) is such an extension to the standard. Based on PostScript, aStandard Page Descri,ptionLanguage(SPDL) is specified.With respect to multimedia, pointers are included as non-readable. An extensionfor the descriptionof music representsStandardMusic Description language(SMDL)and HyTime.Another application conforming to International Standard ISO 8879- SGML - is theHyperTed Marlcup Language (Hf ML). HTML is a mark-up language for hypertextwhich is understood by all WWW (World Wide Web) clients. HTMI is persuadedto becomea standard for interchangeof hypertext information on the network. It isproposed o be registeredas a MIMB (RFC1521)content type. HTML can be usedto represent:

o Hypertext news, mail, online documentation and collaborative hypermedia.

o Menusof options.o Database query results.

o Simple structured documents with in-lined graphics.

o Hypertext views of existing bodies of information.

7/28/2019 215814- Chapter 13

36/62

51613.4

CHAPTER 13, DOCUMENTS, HYPERIEXT AND MHEGDocument Architecture ODA

The OpenDocument Archi,tecture ODA)[Org89] was nitially called the Office Doc-ument Architecture because t supports mostly ofrce-oriented applications. Themain goal of this document architecture is to support the exchange,ptocessingandpresentation of documents n open systems. ODA has been endorsedmainly by thecomputer industry, especially n Europe.

13.4.1 Some Detai ls on ODAThe main property of ODA is the distinction among content, logical structure andlayout structure. This is in contrast to SGML where only a logical structure and thecontents are defined. ODA also definessemantics. Figure 13.17shows these threeaspects inked to a document. One can imagine these aspects as three orthogonal

LayoutStructure

LogicalStructure

Figure 13.77: ODA: Content, layout and logical uiew.views of the same document. Each of theseviews represent one aspect, ogether weget the actual document.

7/28/2019 215814- Chapter 13

37/62

13.4. DOCUMENT ARCHITECTURE ODAContent Portions

517

The content of the document consistsof Content Portions. These can be manipu-lated accordingto the correspondingmedium.A content architecturedescribes or each medium: (1) the specificationof the ele-ments' (2) the possibleaccessunctions and, (3) the data coding. Individual elementsare the Logical Data Units (tDUs), which are determined or each medium. Theaccess unctions serve for the manipulation of individual elements. The coding ofthe data determines he mapping with respect to bits and bytes.ODA hascontentarchitectures or media tert, geometricalgraphics and raster graph-ics. Contents of the medium tertare deflned through the Character Content Archi-tecture' The Geometric Graphics Content Architecture allows a content descriptionof still images' It also akes nto account ndividual graphical objects. This is similarto CGM. Pixel-oriented still images are described hrough RasterGraphics ContentArchitecture. It can be a bitmap as well as a facsimile.

Layout Structure and Logical StructureThe structure and presentation models describe- according to the information ar-chitecture - the cooperationof information units. These kinds of meta informationdistinguish layout and logical structure.The layout structure specifiesmainly the representationof a document. It is relatedto a two dimensional epresentationwith respect o a screenor paper. The presenta-tion model is a tree. Figure 13.18shows he contentof a document together with thelayout structure. Using frames the position and size of individual layout elementsis established.For example, the page size and type style are also determined.The logical structure includes the partitioning of the content as shown in Figure13.19. Here, paragraphsand individual headingsare specifiedaccording to the treestructure. Lists with their entries are deflned (example) as:

paper = preamble body postanble

7/28/2019 215814- Chapter 13

38/62

518 CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEGLayout-Structure

Generic-Spwificlleader Frme

Body Frame

F@aer Frme

Figure 13.18:Layout uiew and,contenl of an ODA document.

bodY = chaPterl chaPter2chapterl = heading paragraph picturechapter2 = heading paragraph Picture ParagraPh

The aboveexampledescribes he logical structure of an article. Each article consistsof a preamble,a bodyand a postamble.The body includes two chapters,both of themstart with headings. A content is assigned o each element of this logical structure.An example, shown in Figure 13.20,,demonstrates the relation among a content,logical structure and layout structure. Figure 13.21shows the contentsand thelogicaland ayout structuresof the document rom Figure 13.20.This figure consistsof a title, a describing text and a figure. The title and describing text are mappedonto the content architecture Character Content Architecture. The figure belongsto the content architecture Raster Graphics Content Architecture.The logical structure of the example dictates for each section at least one title, oneparagraph and one flgure. This behavior is shown in Figure I3.2L

7/28/2019 215814- Chapter 13

39/62

Gmphic ObjetTIIE N-TTMES TRANSFORM COMMANDBy dE k N @t||md you oD pduce . quetr ofcbryd @pl ol . gnphlc obJd ngur S shE.hcx.ryle dheedof bG thtn be.dlevd.

Flsure,l5: N-TIMES TRANSroRM ExaDplcm.n you bv. @Bttltd hbgnphlc objd.hd dnsn|q w ah EF.t k.y (PF, b m.lc tb 6d opy.TB@py b the drehaobjd Yd en ch.nae 6eobjd by dol4 .ny nmb.r of tolloehA adbc:r &f6! th obreru4 rb drJ. drd dt

o I y F Rbeub lLo Youo tF N mr trE lr he mlydErd tuLo To @py & obl.4 |+rr & d olr.6br

The @mmd en b. peilod .b.doh!{elly:

70 PENNAL TBLBH|NG sysEM rNmoDufrtoN

13.4.DOCUMENTARCHITECTUREODALogical StructureGeneric-SIEciIlcHeaderSetion Title

Figure:-Illustration-Caption

Paragraph

5i9

Unordercd Lisl

Ordercd Llst

AutomaticPage Numbering

Figure 13.19: Logicaluiew and content of an ODA document.

n the upper right paragraph of Figure 13.21, he layout structure of the documents displayed. It consistsof severa,lrames which are ordered n a certain style on thetwo-dimensional surface.The ordering is presented hrough individual lines between the elements of the dif-erent areas. A. content portion is assigned o each leaf of the logical tree (a basic

object) and of the layout tree.ODA distinguishes he following layout and logical structures:

The generic ogicaland generic layout structures nclude a set ofdefault values.For example, a paragraph can be specified with LeftHandOffset = 0.

The speci,ficogical and specifi,cayout structure describea concretedocument.They are linked to the generic structure. For example, a concrete paragraphcan be defined with LeftHandOffset : 7cm. The following example presentsaspecific ayout object:

7/28/2019 215814- Chapter 13

40/62

CHAPTER 13. DOCUMENTS. HYPERIEXT AND MHEG

brief instructions: Setting TV StationsTV stations broadcGa ac@rding to differena standardsin different coutries (se ieble of standards).Prcced c follows ao set a station which @nfo|ruto r differena st ndard (multi-stand.rd version only):

Selet the dcsired progmn nmber. Press the "C".button24 to sct the ch.nncl function .nd ilter the desiredchannel number.

Figure 13.20: An ODA d,ocumenterample).

object type:object identifier:position:dimensions:content architecture class:content portions:

blocktitle(x=111, y=222)(height=333 width=444)formatted charactercontent architecture(POTNTERTO TEXT)

The information architecture ODA includes he cooperativemodels shown n Figure13.22. The fundamental descriptive means of the structural and presentationalmodels are linked to the individual nodes which build a document. The documentis seenas a tree. Each node (also a document) is a constituent) or an object. Itconsistsof a set of attributes, which represent he properties of the nodes. A nodeitself includes a concrete value or it defines elations between other nodes. Herebv.relations and operators, as shown in Table 13-1, are allowed.If we consider document processingaccording to Figure 13.3, in ODA it can bepresentedas shown in Figure 13.23.

7/28/2019 215814- Chapter 13

41/62

13.4.DOCUMENTARCHITECTUREODA

Figure 73.2L: Relations arnong content and logical and layout structures (eramplefrom Figure 13.20).

The simplified distinction is between the editing, formatting (Document LayoutProcess and Content Layout Process)and actual presentation (Imaging Process).Current WYSIWYG (What You See Is What You Get) editors include these inone single step. It is important to mention that the processingassumesa linearreproduction. Therefore, this is only partially suitable as a document architecturefor a hypertext system. Hence,work is occurring on Hyper-ODA. For the documentexchange,different document architecture classescan be used, as shown in Figure13.23:

o L formatted document ncludes the specific layout structure, and eventuallythe generic layout structure. It can be printed directly or displayed, but itcannot be changec.

o A processable ocumenlconsistsof the specific ogical structure, eventually thegeneric ogical structure, and later of the generic ayout structure. The docu-ment cannot be printed directly or displayed. Changeof content is possible.

521

7/28/2019 215814- Chapter 13

42/62

522 CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEG

Figure 13.22: ODA information architecture with structure, content, presentationand representationmodel.

o A formatted processable ocumenl s a mixed form. It can be printed, displayedand the content can be changed.

For the communication of an oDA document, the representation model, shown inFigure 73.22, is used. This can be either lhe Open Document Interchange For-rnat ()DIF)(based on ASN.1), or the open DocumentLanguage oDL) (basedonsGMr).The manipulationmodeln oDA, shown n Figure rs.2z,makesuseof DocumentApplicationProfi,les DAPs). Theseprof,lesare an ODA extension or documentmanipulation.Because f the ODA structure, he extension efines hreedifierentlevels,which representa subsetof oDA (Tert only, Tert * RasterGraphics+GeometricGraphics,Aduanced,euel).

Content

t - lGeomtricGmphics

tr mRepresentationModel

7/28/2019 215814- Chapter 13

43/62

13.4. DOCUMENT ARCHITECTURE ODA 523

Sequence all child nodes are ordered sequentiallyAggregate no ordering among the child nodesChoice one of the child nodes has a successorOptional one or no (operator)Repeat one .... any times (operator)Optional Repeat 0 ... any times (operator)

Table L}.l: Relations among nodes.L3,4.2 ODA and MultimediaMultimedia requires, besidesspatial representationaldimensions, he time as a mainpart of a document. If ODA should include continuous media, further extensions nthe standard are necessary.Currently, multimedia is not part of the standard. Allfurther paragraphs discussonly possibleextensions,which formally may or may notbe included in ODA in this form.

ContentsThe content portions will change to timed content portions [RG90]. Hereby, theduration does not have to be specifieda priori. These types of content portions arecalled Open Timed Content Portions. Let us consider an example of an animationwhich is generatedduring the presentation time depending on external events. Theinformation, which can be included during the presentation time, is images takenfrom the camera. In the caseof a Closed Timed Content Portion, the duration isfixed. An example is a song.

StructureOperations betweenobjects must be extended with a time dimensionwhere the timerelation is specified n the father node r.rn proportion to the child nodes k1, k2. Thefollowing additional examplesand described elations can be found in chapter 15 on

7/28/2019 215814- Chapter 13

44/62

CHAPTER 13, DOCUMENTS. HYPERIEXT AND MHEG

Figure 13.23: ODA documentprocessing from the information to the presentation.

synchronization:

o before:k1 before kz, lq I kz ) ao meetst h,kz), k1 exactly before kz,kt + kz = oo ouerlapso duringo startt endo midd,leo a to equal

7/28/2019 215814- Chapter 13

45/62

13,4.DOCUMENTARCHITECTUREODAContent Architecture

525

Additional content architectures for audio and video must be defined. Hereby, thecorrespondingelements,LDUs, must be specified. For the access unctions, a set ofgenerally valid functions for lhe control of the media streamsneeds o be specified.Such functions are, for example, Start and Stop. Many functions are very oftendevice-dependent. One of the most important aspects s a compatibility provisionamong different systems mplementing ODA.During data coding, a link to the exist ing de-jure and d,e-facto tandards s required.Especially mportant standards are: JPEG for image compression,MPEG for videoand audio, H.261 for video and CD technology. The possibility of an oryn archi-tecture should be considered. With this approach, further developments could beconsidered oday without changing or extending the standard.

Logical StructuresExtensions for multimedia of the logical structure also need to be considered. Forexample, a film can include a logical structure. It could be a tree with the followingcomponents:

1. PreludeIntroductory movie segmentParticipating actors in the secondmovie segment

2. Scene13. Scene24 . . . .5. Postlude

Such a structure would often be desirable for the user. This would allow one todeterministically skip some areas and to show or play other areas.

aa

7/28/2019 215814- Chapter 13

46/62

526 CHAPTER 13. DOCUMENTS. HYPERTEXT AND MHEGA time relation between different objects is also relevant for the definition of thelogical structure. This time relation should be specifiedonly to the extent that theuser perceivescontent portions to be in sync. A relation can be definedbetween hesubtitles and the sceneas a whole, although the exact time cannot be specified bythe logical structure. This would rather count the layout structure.A spatial relation betweendifferent objects is defined n the sameway as t is done byother discrete media. Instead of a tree structure in an ODA multimedia document,it should be possible o definea graph. This would allow for any kind of hypermediatechniques.

Layout Structure

The layout structure needsextensions or multimedia. The time relation by a motionpicture and audio must be included. Further, questionssuchas When will somethingbeplayed?, From which point? and With which attributes and dependencies?mtstbe answered.The spatial relation can specify, or example, relative and absolutepositions by theaudio object. Additionally, the volume and al l other attributes and dependenciesshouldbe determined.Especialiy by the continuousmedia, interactiuity needs o be considered.The docu-ment is not only anymore a paper, the linear processingwill become the interactiueprocessing. In the caseof ODA, Ihe i,magingprocessshould not be left out, as wewill discuss n the next section n terms of MHEG.If al l extensions of ODA with respect to the integration of continuous media aresummarized, the result is the multimedia document architecture shown in Figure13.2.

7/28/2019 215814- Chapter 13

47/62

13.5, MHEG13.5 MHEGThe committee ISO/IEC JTCl/SC29 (Cod,ingof Audio, Picture, Multimedia andHypermed,ianformation) works on the standardization of the exchange ormat formultimedia systems. The actual standards are developedat the international levelin three working groups cooperatingwith researchand industry. Figure 13.24showsthat the three standards deal with the coding and compressionof individual media.The results of the working groups: ihe Joi,nt Photographic Erpert Group (JPEG)

WG1Coding f StillPictures

(JBrG/JPEG)

WG11Coding f MovingPicturesndAudioMPEG)WGl2Coding fMultimediaandHypermediaInformalionMHEG)Structure_ content _------------- Strudure

Figure 13.24: Working Groups withi,n the ISO-5G29'and the Motion Picture Expert Group (MPEG) are of special mportance in the areaof multimedia systems(seeChapter 6 on compression).In a multimedia presentation, the contents, in the form of individual informationobjects, are describedwith the help of the abovenamed standards. The structure(e.g., processing n time) is specified irst through timeiy spatial relations betweenthe information objects. The standard of this structure description is the subjectof the working gloup WG12, which is known as the Multi,media and HypermediaInformation Coding Erpert Group (MHEG). The name of the deveioped standardis officially called Information Technology- Coding of Multimedia and HypermediaInformation (MHEG). The final MHEG standard will be described n three docu-ments. The first part will discuss he concepts,as well as the exchange ormat. Thesecondpart describesan alternative, semantically to the first part, isomorph syntaxof the exchange ormat. The third part should present a referencearchitecture for

rso/rEcTcl/sc29Coding f Audio, icture,MultimediandHypermedianformation

7/28/2019 215814- Chapter 13

48/62

528 CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEGa linkage to the script languages.The main conceptsare covered n the flrst docu-ment, and the last two documents are still in progress; herefore, we will focus onthe first document with the basic concepts. Further discussionsabout MHEG arebasedmainly on the committee draft version, because: (1) uJl related experienceshave been gained on this basis [MEg+]; (Z) the basic concepts between the finalstandard and this committee draft remain to be the samel and, (3) the finalizationof this standard is still in progresswhile printing this book. Note that the followingdiscussion s based on [ME9a] designing, mplementing and improving the MHEGstandard.

13.5.1 Example of an rnteractive Multimedia PresentationBefore a detailed description of the MHEG objects is given, we will briefly exam-ine the individual elements of a presentation using a small scenario. Figure 13.25presentsa time diagram of an interactive multimedia presentation. The presentationstarts with somemusic. As soon as the voiceof a news-speakers heard in the audiosequence, graphic should appear on the screen or a couple of seconds.After thegraphic disappears,the viewer carefully reads a text. After the text presentationends, a Stop button appears on the screen. With this button the user can abortthe audio sequence. Now, using a displayed input field, the user enters the title

Selection; StopVideo

lmageTextAudio

[lilililililllillliliN

Modification

Figure 13.25: Time iliagram of an interactiue presentation.of a desired video sequence.These video data are displayed mmediately after the

Start ofPresentation

7/28/2019 215814- Chapter 13

49/62

13.5. MHEGmodification.

529

Content

A presentation consistsof a sequenceof information representations. For the rep-resentation of this information, media with very different properties are available.Becauseof later reu,se)t is useful to capture each nformation LDU as an individualobject. The contents n our example are: the video sequence, he audio sequence'the graphics and the text.

Behavior

The notion behauiormeansal l information which specifies he representationof thecontents as well as defines he run of the presentation. The first part is controlledby the actions start, set uolume, set posit'ion, etc. The last part is generated bythe definition of timely, spatial and conditional links between ndividual elements.If the state of the content's presentation changes, hen this may result in furthercommandson other objects (e.g., he deletion of the graphic causes he display of thetext). Another possibility, how the behavior of a presentation can be determined,is when external programs or functions (script) are called.

User InteractionIn the discussed cenario, he running animation could be aborted by a correspond-ing user interaction. There can be two kinds of user interactions. The first oneis the simple selection, which controls the run of the presentation through a pre-specifiedchoice(e.g.,push the Stop button). The secondkind is the more complexmodifi,cation,which gives he user the possibility to enter data during the run of thepresentation e.g.,editing of a data input field).

7/28/2019 215814- Chapter 13

50/62

530 CHAPTER 13. DOCUMENTS' HYPERTEXT AND MHEGContainerMerging together severalelements as discussedabove, a presentation, which pro-gressesn time, can be achieved.To be able to exchange his presentation betweenthe involved systems,a compositeelement is necessary.This element s comparableto a container. It links together ai l the objects into a unit. With respect to hyper-text/hypermedia documents,suchcontainerscan be ordered to a complexstructure,if they are linked together through so-calledhypertext pointers.

13.5.2 Derivation of a Class HierarchvFigure 13.26summarizes he individual elements n the MHEG classhierarchy n theform of a tree. Instancescan be createdfrom all leaves(roman printed classes).AI I

Action ""^Alxpo.it"Modification

Figure 13.26: Class hierarchy of MHEG objects.internal nodes, including the root (itatic printed classes),are abstract classes,.e.,no instancescan be generated rom them. The leaves nherit someattributes fromtheroot ofthe tree as an abstract basic class. The internal nodesdo not include anyfurther functions. Their task is to unify individual classes nto meaningful groups.The action, the link and the script classesare grouped under the behavior class,which deflnes he behavior n a presentation. The interaction class ncludes he userinteraction, which is again modeledthrough the selectionand modification class.Al lthe classes ogether with the content and composite classesspecify the individual

Link

7/28/2019 215814- Chapter 13

51/62

1.3.5.MHEG 531components n the presentation and determine the componentclass. Somepropertiesof the particular MHEG engine can be queried by the descriptor class. The macroclass servesas the simplification of the access, espectively reuse of objects. Bothclassesplay a minor rolel therefore, they will not be discussed urther.The deveiopmentof the MHEG standard uses he techniquesof object-oriented de-sign. Although a class hierarchy is considereda kernel of this technique, a closerlook shows that the MHEG classhierarchy does not have the meaning it is oftenassigned. Methods of the classesare not specified n the standard, therefore onlysome attributes of the MH-object class are passed from their derived classes. Inthis context, a problem of using ASN.1 syntax for MHEG classdescription becomesobvious. ASN.I d.oesnot support inheritance mechanisms; herefore, the reuse ofthe attributes in the remaining classes,which are defined n the MH object class, sapplied. It needs o be mentioned that the MHEG classhierarchy is used mainlybecauseof didactical reasons. For the implementation of a MHEG run-time envi-ronment, it does not have any importance.

MH-Object-ClassThe abstract MH-Object-Class inherits both data structures MHEG ldentifier and'Descriptor.MHEG lilentifier consists of the attributes MHEG ldentifier and. Object Numberand it servesas the addressingof MHEG objects. The first attribute identifies aspecificapplication. The Object Numberisa number which is definedonly within theapplication. The data structtre Descriptor provides the possibility to characterizemore precisely each MHEG object through a number of optional attributes. Forexample, this can becomemeaningful f a presentation s decomposednto individualobjects and the individual MHEG objects are stored in a database. Any author,supported by proper search unctions, can leuse existing MHEG objects.In the following paragraphs, he MHEG classesare presentedand some main mech-anisms are discussed. For more technical details, the interested reader should seethe committee draft [MHE93].

7/28/2019 215814- Chapter 13

52/62

532 CHAPTER 13. DOCUMENTS, HYPERTEXT AND MHEG13.5 .3 ContentsContent ClassThe content class differs from the other classesbecause t provides the link to theactual contents. Through this content class, his information becomes lexible andis linked together in an open way in the system. Each content object representsexactly one information within a presentation.The type of the particular medium is defined in a content object through the at-tribute MHEG Classi,ficationand he coding s specified hrough the attribute Hook.The actual data can be included either in the object (included data), or they canbe referenced hrough an unambiguous identifier (referenceddata ). The includeddata are meaningful only when a small amount of data is present. The reason isefrciency because he content object itself is transformed, before any exchangeoc-curs, through the encoder/decoder. The referenceddata have the advantage thatthey can be reusedoutsideof MHEG.At the time of content object processing, t must be guaranteed hat the referenceddata were requestedthrough proper application services. For the presentation ofdata, one medium-correspondingrepresentationcomponent is used.The standard includes a set of codings. Besides he existing standards, future stan-dard (Non-MHEG Standardized Catalogue),as well as application-specific codingmethods (Proprietary Catalogue), are considered. This is done through an opendefinition of the individual formats.

Virtual Coordination SystemsThe so-calledGeneric Spacedefinesa virtual coordination system. Content objectscanbe defined elative in dimension and ordering to eachother. There are three axes:X (width)' Y (height) and Z (depth). A value from -32768to 32767 s assigned oeachaxis. During the run-time, a translation from the virtual MHEG coordinates othe physical coordinate system is performed in the particular presentation service(e.g', the number of pixels which cover a Motif window). Additionallv. a time

7/28/2019 215814- Chapter 13

53/62

13,5. MHEGcoordinate ystemexistswith its axis T.interval rom 0 to infinity where he sca,le

Virtual Views

The defined value set for this axis is anunit is a millisecond.

Until now we assumed hat the presentation of the contents occurs exactly as orig-inated. Actually, MHEG provides a set of possibilities which can control a presen-tation of the content objects through proper parameters. For example, a movie canbe played according to certain time specifications, he volume of an audio sequencecan be set or the visual area of a graphic can be specified. The manipulation oftheseparameters s determined through correspondingcommands (seeaction class)in the coding. The following example of a computer simulated basketball gameshows that the same player of each team can occur at different positions of thefield at the same time. Instead of storing all possiblepresentation combinations asseparate content objects, the changeof the parameter is modeled as a call of anobject's method. Using this approach, so-calledvirtual views (Presentable) o eachcontent object are created during run-time. These virtual views are specifiedat thepresentation-composition hrough unambiguousnumbers. During run-time, they fixthe parameters according to the representation. Figure 13.27shows someexamplesof virtual views and illustrates the reusepossibilities.Nffi

13.27: Eramples of airtual aiews.

Such views exist for all component-classes, s well as for objects of the selection,modification and composite classes.However, they have within each classdifferentspecifications. n the next paragraphs, f MHEG objects are addressed, irtual viewsare meant. Only in the caseof a different handling, the exclusion of virtual viewswill be stated explicitly.

Positionontnt biect

Figure

7/28/2019 215814- Chapter 13

54/62

7/28/2019 215814- Chapter 13

55/62

13.5. MHEG

Prparation Status Prcsentation Status

Figure 13.28: An erample of two state transition graphs.

object can be displayedby a virtual view. With the action d,estroy,he resources,a,llocatedby the object, can be freed. The display of this object dependson thepresentation status of a correspondingvirtual view.Further state transitions are defined or the composi,tionstatus, modificationstatus,time-stone status and script actiuation status. An exception is the selectionstatus.The different statusesare determined first with the modeling of the individual userinteractions in a presentation. The previously describedactions are basicoperationsof an MHEG object.The construction of an action object also ncludes compler runs thal are defined bygrouping severalbasic actions. Depending on requirements, single actions can beperformed in parallel or sequentially . The data structure of the action classdefinesfor this purpose a Parallel Action List. The elementsof this ljst consistagain of a listof basic actions. DelayedSequentialAction is defined as a Sequence f Actions. Thebasic actionsin Delayed SequentialAction are processedonly sequentially. Figure13.29gives a graphical overview of the data structure.

535

+IDelayed-Sequential-Action+Delay t"f:ftlfl+

Figure 13.29: An enampleof an acti,on object.

7/28/2019 215814- Chapter 13

56/62

Trigger onditions

536 CHAPTER 13. DOCUMENTS. HYPERTEXT AND MHEGLink ClassSo far, no relation which specifies he run of a presentation could be establishedbetween an action object and a content object, respectively a virtual view. The linkclassmust fulfill two tasks. First, it specifieswhich actions are sent to which MHBGobjects. Second, he conditions are specifiedunder which this processoccurs. Fromthese asks, t can be derived that the processingof an MHEG-coded presentation sbasedon an event-drivenprocessingmodel. This model is suitable for the mapping ofparaliel running, synchronizedprocesseswhich exist often in interactive multimediapresentations. Their sequentializationdepends on the performing system.A link object consistsof a set of iinks. The semanticsof a link are shown in Figure13.30. A link connects a sourceobject with one or several destination objects. The

SourceObjects TargetObjectsFigure 13.30:Construction f a link.

source can be MHEG objects, as well as virtual views. The performance of a link isalways dependenton a condition (trigger condition), which can be expressedhroughthe possiblestate transition in the source object. Only if this condition is satisfied,other conditions (additional conditions) are checked.If all conditions are satisfied,the link is active. In this case, he action objects, specified n the action /isf of thelink, are sent to al l destination objects.Although the standard does not specify the implementation, it may be appropri-ate to mention that the link is checkedonly if a state transition of the particularsource objects occurred. This implementation is driven by efficiencyreasons. Theattachment point is used to position destination objects relative to the sourceob-ject. It means hat coordinates n an action object expressa relative position of the

7/28/2019 215814- Chapter 13

57/62

13.5. MHEGdestination bject with respect o the source bject.

Script ClassAnother possibility to determine the behavior of objects or the run of a presentationis the script c/ass. This class was considered o support an MHEG presentation inother run-time environments e.g.,Script/X), externalprograms ".g., r C-program)or functions calls. Comparable with the content class, different languages eitherstandardized MHEG-Catalogue)or non-standardizedNon-MHEG-Catalogue),aresupported.

13.5 .5 lJser ln teract ionUsing the previously described classes, he user cannot interact with a runningpresentation. The introductory scenario has discussed hat the MHEG standarddistinguishes wo kinds of user interactions: selectionand modification.

Selection ClassThe selectionclassprovides he possibility to model an interaction as a selectionof avaluefrom a pre-definedvalue set. The explanation of the link class has shown thatthe run of a presentation s controlled by the occurrenceof events which are givenby the standard. With a selectionobject, it is possible o consider user interactionsalso in the form of such events. A selection object takes over the definition ofthese application-specificevents. A correspondingevent value is prepared for theparticular selectionpossibilities. At a certain point in the user interaction, thereexists a virtual view on the selectionobject (similar to a content object). The eventof the user interaction is stored in the view. The storage s performed through theassignmentof the particular event value onto a selection status field, which is pre-determined. The changeof this status field should lead to the following situation:the condition of a link object is satisfied and therefore actions are sent to other

7/28/2019 215814- Chapter 13

58/62

538 CHAPTER13, DOCUMENTS, HYPERTEXT AND MHEGThe changeof the status field, as well as the display and control of proper primitives(the following primitives are specified n the standard: n'Lenu, ull-down rnenu,pop-up menu, button list, key list, d,eaiceist, scrolling list, sw'itch, item, button, key,d,euice) t the user interface are suborders of the user interface services. With thecoding of a selection object, it is not specified which kind of primitives should beused or the interaction. The primitive and its properties e.g., ext on the knob) areset through actions (seeselection style presentability) which are sent to the virtualviewsof a selectionobject.

Modification Class

The secorrd orm of user interaction servesas the input and.manipulation of data.In contrast to a selectionobject, no value set s pre-definedby a modification object .The event of an interaction is represented hrough an entered content object . Itscontent is represented hrough the virtual view, which is defined for the contentobject, and it is modified through the primitive of the user interface. The actualprocessingstate of the content object is captured in these virtual views in a mod-ifi,cation status field before the modification (modifi,able),during the modification(modifuing) and after the modification (modified). Hence, it is possible, similar tothe selection, o query the status field through alink object, and therefore to controlthe run of a presentation.During the international agreementprocess, t was recognized hat the ASN.1 codingis incomplete in the CD-document. Exactly, the deal concerneda variable in thecontent object. The value of this variable can serve again as a parameter for anaction (e.g., volume by sef uolume). In the current ASN.I coding, a reference othis variable for action objects is not possible. Therefore, so-calledgeneric ualuesare included in the later version of the standard. They can contain the event of amodification according to type. The following types have been specified: Boolean,Character String, Numeric Value, Numeric Vector, Spatial Vector, Temporal and.Volume. Further, they can be reused by action objects. In some cases e.g., fill outa sheet), it will be necessary o return the values to the application. They can beperformed by the action return.

7/28/2019 215814- Chapter 13

59/62

13.5. MHEG13.5.6 ContainerComposite ClassThe composite class has the task of composing a,ll the necessaryobjects from thepreviously described classesnto a presentation. Independently, if a single object isincluded or referenced,each composite object behavesas a container. It means hatit representsa closedunit for the exchangeof presentations among systems. Thefunctionality of a composite object must satisfy some.assumptionsor the synchro-nized output through the MHEG engine. Through pointers (compare hypertextIinks) between different composite objects, any complex presentation can be cre-ated. At this point, the meaning of the abstract classesBehauior and Component(Figure 13.26) should be clear. Figure 13.31 shows graphically the structure of acompositeobject, including defined virtual views (Presentables)and two link objects( Cont ainer- start- up, P resent abto -st art - up).

Figure 13.31: Construction of a compositeobject.

Before we describe the individual parts, let us mention that the performance of

539

7/28/2019 215814- Chapter 13

60/62

540 CHAPTER13. DOCUMENTS. HYPERTEXT AND MHEGeach compositeobject, as by all other MHEG objects, runs in two phases. Thefirst phase,started by the action prepare) nitializes the composite object. It meansthat the object itself, both start-up-