"Physical hypermedia": Organising collections of mixed physical and digital material

Kaj Grønbæk , Jannie F. Kristensen , Peter Ørbæk , Mette Agger Eriksen, "Physical hypermedia": organising collections of mixed physical and

digital material, Proceedings of the fourteenth ACM conference on Hypertext

and hypermedia, August 26-30, 2003, Nottingham, UK

[doi>10.1145/900051.900056]

Dealing with information Information exists in a mixture

Physical Digital

Much information is available digitally but people don’t always want work with it that way Print papers and photos out

•Revise, annotate, sketch and diagram

Dealing with information Other information or objects may only exist in a physical form with no realistic solution for fully digital information solutions There needs to be a way to annotate and link information to these objects

When working on a project professionals need to be able to relate these physical and digital objects together

Dealing with information Although he domain focus for this paper is architecture and design, this problem exists in many domains and fields (such as medicine and law)

Examples from this domain Wood and foam models Building material samples

These objects need to be able to be related back to the project description, drawings and specifications.

Ubiquitous Computing as a solution to the mix of media So far in the ubiquitous computing field the focus has been Developing infrastructure for display enabled devices on difference scales• PDAs, cell phones, watches

Making digital information available everywhere and shared

Much less research has focused on how to relate computing to physical objects such as paper, binders, samples, materials, containers

The presents an opportunityAugmented Reality Field studies linking digital information to physical objects, spaces and locations This is practically completed through an ID code (barcode), readers, and RFID (radio frequency identification)

The authors focus on the ordinary physical artifacts of work

In the architecture field And how these objects are used

Hypermedia and Spatial Relationships A recent trends in hypermedia research, which studies links, composites, hierarchies, groupings and metadata has been to examine the aspect of spatial organization in digital information. Described as a 2d canvas with cards being paced on a table• Cards can be changes in size, shape and color• May contain pointers to external content• Some cards can be “opened” to reveal a new space and different cards

Hypermedia and Spatial Relationships Topos uses this approach in 3d, which was used by the authors in this paper.

The central concept Tops uses is “workspace” Sets of spatially related and placed materials(drawings, models, notes, documents)

Supports internet sharing and collaboration Supports 3d organization of models and documents Can support abstract spatial hypermedia and concrete spatial hypermedia such as:• Open unfurnished spaces• Buildings or landscapes and backgrounds for relative material placement

Bridging physical and digital material In the physical work environment people utilize things like paper, files, clipboards, clips, photos, sketches, wood and cardboard in their work. Some of these items can be related digitally via folders and directories, but there isn’t much support for keeping these relationships constant.

Bridging physical and digital material For this study the author’s utilize a RFID tag system and integrate it with the 3d Topos system Provides support for linking physical material a digital environment•Through linking, grouping, annotating and tracking where physical objects were last located

How architects/designers manage physical materials The process for organizing material can range: From very formal (legal medial fields)

To open to definition by the individual or group

How architects/designers manage physical materials Use of physical work space:

“nomadic” movements though the space for both collaborative and individual work

Space is not just used for organizing but also for exhibition of work and as a creative and inspirational atmosphere.

This implies a tagging system in this environment may have to be made up of many tag readers integrated into desks, worktops, walls and shelves

Organizing Physical Objects Much of what this work entails is finding, collecting, producing and organizing many different types of physical materials Paper, bricks, plants, plans, cardboard, models, samples, drawings•For inspiration, documentation, design process, reference, prototype

Meta-data Annotation The three main purposes were discovered for organizing physical materials and objects Documentation Inspiration Management

For all 3 purposes it is important to be able to able to re-find and recognize material for it’s later use

Meta-data Annotation The ability to add annotations, and keep them related to the physical objects is also important in work situations Example a series of time stamped models placed in order to represent progress and the ability to backtrack in a project

Collectional Artifacts Can be described by the range of materials they are suited for organizing

In terms of the way they afford different aspects of collecting and organizing• Flexible(shelves), specific, mobile(stapler), link loosely(paperclip) or permanently

• For this study collectional artifacts included:• Shelves, walls, boxes, folders, tape, clips, staplers

Abstractions of Collectional Artifacts How do we map handling of physical materials to digital materials? Notion of space in a mixed environment

•Topos provides features supporting spatial organization of digital materials, the infrastructure supports spatial memory• the workspace concept is the main structuring mechanism

•The next step is to map these physical organizational actions into semantics for the hypermedia system

Abstractions of Collectional Artifacts How do we map handling of physical materials to digital materials? Understanding collectional actions

• Designers use a multitude of objects to organize physical collections together • From very formal to very informal

• Regardless, these methods related to explicit and implicit meta-data actions• Example: Attaching a post it to a collection of papers while stapling it

• This implies some reasoning based on category, associating materials by purpose over time tightly or loosely connected• Stapling more permanent that paper clipping

We wish to create open space of possibilities for coupling actions

Relationships between Physical and Digital Worlds Based on abstractions of collections the following are possibilities: Physical Only

• Digital world has no trace of object Physical w/ digital ID

• Digital world has an ID plus meta-data relating to the object but no digital representation • Example: a brick with an RFID tag

Physical w/ low resolution digital• Example: a pen tracked drawing, scanned document or photo of an object

Relationships between Physical and Digital Worlds Based on abstractions of collections the following are possibilities: Physical generated from digital

•A printed map, drawing or report Digital Only

•Content cannot be printed or made tangible outside of the digital• Video or sound source files

Relationships between Physical and Digital Worlds Collections of materials from one category differ greatly from collections in other categories Items from category 1 are difficult to map semantically into the digital environment• We can do this by description and properties and related the item to a tag or stamp so that is it transformed into cat 2.

Items from cat 2 can be identified ad described by meta-actions and grouped through collective actions• They still do not have distinct visual or auditory appearance in the digital environment

Items in cat 3 can have low res digital representation in the environment allowing them to appear in workspaces, but cannot be worked with dynamically• Allow collectional and metadata actions in the physical world to be mapped directly into digital actions

Relationships between Physical and Digital Worlds

Items from cat 4 correspond into common documents in Topos• Physical representations can be mapped digitally in mirrored actions

• These materials may have physical representations with the same unique ID• Annotations or metadata added to any should be associated with the unique ID in digital world representing the material

For collectional actions for items in cat 5 mapping is done through an indirect relationship to the digital item augmented with a metadata ID referencing the actual digital material• Screen dump, transcription of a video or audio file

Physical Hypermedia Prototypes The previous observations are the starting point for the prototype The real world design took place in a workshop form over 3 days with 4 landscape architects• They were presented with a design task (real world) related to a site nearby• Given material and time to prepare pre-workshop

Asked to solve task in normal fashion utilizing the physical and digital work environments and prototypes with in the project

Discussion, observation or their work gave first iteration of user feedback on design issues and concepts of relating the the gap between physical and digital environments

Tagging- Registering and Detecting Collectional Actions RFID tags were used on physical materials

• Inexpensive, adhesive, placed in a cup on a table or workspace• Single or collectional

• Tags are associated with an item through selection of the digital item and associating the tag

Extended the concept to support grouping into collections and link between materials

Topos pops up workspaces or objects when tagged items are placed near the tag reader antennas

RFID readers consist of:• a desktop model that recognizes several tags at once • Handheld models that see one tag at a time

• Two modes: Tag arrived, tag left when events occur

Tagging- Registering and Detecting Collectional Actions Easy tagging to track within office and electronic workspace Tagging can take place after meetings as new documents are created• Metadata and annotations can be added to physical items through the digital representations

The RFID allows for concurrent contact less detection of numbers of tags containing unique IDs and can record that a collectional action has taken place in the vicinity

If physical objects have been removed from collectional artifact their digital counterpart displays this (with a red mark) to show it is missing from the collection

Representing Physical Material Digitizing physical documents is important in the design process in bridging the two environments Tagging physical material and linking it to digital material is ne way to do this

Other times having a digital “snapshot” in the electronic environment as a reminder is important.

A Snapscanner allows quickly adding a digital representation with an ID for a physical to digital link. Many images found to be too low res to be helpful Should be portable

Tool Tags Physical cues for linking and grouping

Tool tag is an RID which is couples to a Topos command in hypermedia system rather than piece of info.• Commands are then issues by placing item on reader with or with out a physical item

• Allow for tangible interaction, making a number of commands available even when mouse & keyboard are not• Link group much like clips and staplers

• Users found it difficult to use tooltags on pure digital objects or accidentally applied more then one conflicting tags

Conclusions Need to extend hypermedia to better integrate physical world

Have identified a set of abstract collectional and meta-data actions to support this

Goal to develop a physical hypermedia system to be at hand for users and pervasive To be familiar in a common metaphor

Questions In an environment like construction, or architecture, does this make sense to use this technology?

This process is very people labor intensive (add work) does it add the required value in this situation? Ex. Many large business use similar processes (with barcodes) for inventory management, saving millions of dollars.

Where would it make more sense to use this technology?