Download - Unit 1
Principles Of Multimedia
610905MCA Semester III
Faculty : Swapna Kolhatkar
MCA Department
Unit 1
UNIT I: Introduction 8 HrsWhat is multimedia, goals and objectives, characteristics of multimedia presentation,multimedia application, multimedia building blocks, multimedia and internet.
Unit 1 ...
Multimedia Architecture:User interface: GTK +, Qt, Windows multimedia support, hardware support, distributed multimedia application, streaming technologies, multimedia database systems, multimedia authoring tools, overview of multimedia software tools, multimedia document architecture (MHEG, SGML, ODA, OMF, etc.)
What is Multimedia
Multimedia means multiple intermediaries between the source and sink of information or multiple means by which information is stored, transmitted, presented or perceived.
The multiple means by which we perceive information are :
1.Text 3. Sound2. Images 4. Video
Characteristics of Multimedia
Multimedia is any combination of text, graphic, art, sound, animation and video delivered by any electronic means.
Multiple medianon linearityscope of interactivityintegrity Digital Representation.
1. Multiple Media
Pictures1.images – captured by camera2.graphics – hand drawn
Time1.static – text, images, graphics2. variable – sound, movies (motion pictures or animation)
2. Non Linearity
It is the capability of jumping or navigating from one point within a presentation to another point without apparent delay.
Eg. Multimedia based presentation
Non linear eg. - TV shows, motion pictures.
3. Interactivity
Interaction is made possible through a set of interactivity elements embedded within the presentation like buttons, menu items or hyperlinks. It helps in taking user inputs.
4. Digital Representation
Analog – audio and video cassette on magnetic tapes. Also called sequential access.
Digital – random access, hard disk, floppy disk and compact disc.
5. Integrity
Several media types present and playing continuously, need to be integrated or be a
part of a single entity which is the presentation.
Purpose of Multimedia
To present visualizations of concepts
To present animations
To require interactive participation of learner
Multimedia Application
Home EntertainmentEducational PurposeIndustrial TrainingInformation kioskscorporate PresentationsBusinessTourism and travel industryElectronic shoppingcommunication and networks.Medicineengineering applicationscontent based storage and retrieval system (CBSR)
Multimedia Building Blocks
TextSound ImagesGraphicsAnimationVideo
Multimedia and Internet
The Internet has been home to multimedia -- graphics, animation, and sound files -- for years and years.
The intertwining of multimedia and the Internet (slowly and irrevocably) makes sense. People like multimedia on their computers. It's engaging, entertaining, and often makes otherwise complex computers a little easier to use. Certainly it works: Some multimedia educational CD-ROMs are meant to be enjoyed by children who are barely old enough to walk. The Internet, once an ugly, text-only system, certainly needed some pictures, sound, and point-and-click simplicity to make it understandable to the masses.
This example is only the beginning of how multimedia is changing the Internet.
Consumer products and services, such as Mosaic in a Box and America Online, are making the Net still easier for all to enjoy.
GoogleEarth
User interface: GTK +, Qt,
GTK (GIMP Toolkit) is a library for creating graphical user
interfaces. It is licensed using the LGPL (Lesser General
Public License) license, so you can develop open software,
free software, or even commercial non-free software using
GTK without having to spend anything for licenses or
royalties.
Hello World in GTKNow for a program with a widget (a button).
#include <gtk/gtk.h>
static void hello( GtkWidget *widget, gpointer data ){ g_print ("Hello World\n");}static gboolean delete_event( GtkWidget *widget, GdkEvent
*event, gpointer data ) { g_print ("delete event occurred\n"); return TRUE;}
GTK+ contd..
The GTK+ library itself contains widgets, that is, GUI
components such as #GtkButton or #GtkTextView.
The GtkButton widget can hold any valid child widget. That is
it can hold most any other standard GtkWidget. The most
commonly used child is the GtkLabel.
Syntax : GtkWidget *gtk_label_new( const char *str );
Qt+
Widgets are the basic building blocks for graphical user
interface (GUI) applications built with Qt. Each GUI
component (e.g. buttons, labels, text editor) is a widget that
is placed somewhere within a user interface window, or is
displayed as an independent window. Each type of widge is
provided by a subclass of QWidget, which is itself a
subclass of QObject.
Distributed Multimedia Applications
In a true distributed system not only the hardware but the software also remains dispersed.
The reasons for distribution of multimedia are those of traditional data processing objects plus some new ones.
i.e. Moving and sharing.
Moving and sharing...
Moving data to the point of useSharing of data and resources.
Eg. An architectural blueprint can be worked upon by different groups of construction engineers, dispersed geographically.
More Examples
Companies can communicate product information to customers in format and location they prefer.
Advertising companies Medical professionals can collaborate
Architecture
Creation of subsystem where the content is created.
A storage subsystem where the created content is stored.
A distribution subsystem using which the content may be distributed to multiple users.
A management subsystem where the entire operations are managed.
Distributed Multimedia Applications
Videophone, video conference and multimedia conference
Video on Demand (VoD)Movie on Demand (MoD)Interactive Television (ITV)Voice over IP (VoIP)
Streaming Technologies
Streaming media is media that is consumed (read, heard, viewed) while it is being delivered.
It is more a property of the delivery system than the media itself.
2 Techniques1. Progressive download2. Real time streaming.
Progressive download
A media file containing audio or video is downloaded and stored on the clients local file system. As the file is being downloaded, the client is able to play back the media file without having to wait for the file to be downloaded in its entirety.
Mostly used for short video clips.
Real time Streaming
The media file is streamed to the client but is only played and not stored by the client.
Mostly used for long videos, internet radio and TV broadcasts.
A streaming media system is made of many interacting technologies. Video cameras and audio recorders
Multimedia Database Systems
Multimedia database systems (MMDBS) can be defined as collections of media elements like text, image, audio and video that can be stored, manipulated and retrieved.
1. Storage and retrieval of multimedia
a) Textual NotationsMMDB needs to be populated first before
it can be utilized.For retrieval, text notations were manually
generated by individuals. Disadvantages :
Subjective nature of manual desc.More time and effortAll multimedia may not have text desc.
1. Storage and retrieval of multimedia
b) Low-Level Featurescolorshapeloudnesspitch.
c)Semantic featuresA Semantic Object (SO) is a collection of
image pixels or audio samples that corresponds to the projection of a real object or event in an image, audio, video sequence eg a cluster of trees, a car moving on a road.
1. Storage and retrieval of multimedia
d) Knowledge based InterpretationExtraction and representation of low level
features and semantic objects is not sufficient to appropriately handle user queries because the system still cannot interpret what the features or objects mean.
e) Queries and searching.'display all images containing the color red''display all audio files containing a laugh'
General Architecture of MMDBS
A multimedia database consists of 2 levels of abstractions
1. physical level consisting of the media objects (Media DB)
2. logical level consisting of the features characterizing the media objects. (Features DB)
Basic overview of an MMDBS
Symbolic links
Media file
Media DB
Feature ExtractionFeature Info
Feature DB
Extended view of an MMDBS
Media DB Feature DB
Media element
Result list of query
Feature Extraction
Query Interface
Semantic Info
Query
Symbolic Link
Prototype Implementations
1. MINOS , MULTOS2. REDI3. MIM4. QBIC5. OVID6. CORE7. VisualSEEk8. SCORE9. CHABOT10. CANDID11. VISION12. JACOB13. FACE etc
Multimedia Authoring Tools
Also known as authorware, a program that helps you write hypertext or multimedia applications. Authoring tools usually enable you to create a final application merely by linking together objects, such as a paragraph of text, an illustration, or a song. By defining the objects' relationships to each other, and by sequencing them in an appropriate order, authors (those who use authoring tools) can produce attractive and useful graphics applications. Most authoring systems also support a scripting language for more sophisticated applications.
Examples
AuthorwareFormula Multimedia
HyperStudioiShell 3
MaxMediaMediaMixer
OpusPlatypus
Revolution
Overview of Multimedia Software Tools
* Digital Audio * Music Sequencing and Notation * Image/Graphics Editing * Image/Graphics Editing * Animation * Multimedia Authoring
Ref : http://www.cs.cf.ac.uk/Dave/ Multimedia/node40.html
Digital Audio
Macromedia Soundedit -- Edits a variety of different format audio files, apply a variety of effects
CoolEdit -- Edits a variety of different format audio files
Music Sequencing and Notation
Cakewalk
Cubase
Logic Audio
Marc of the Unicorn Performer
Image/Graphics Editing
Adobe Photoshop
Adobe Premiere
Macromedia Freehand
Animation
Avid SoftImage
Animated Gif building packages e.g.
GifBuilder
Multimedia Authoring
Macromedia Director
Authorware
Microcosm
Flash : Macromedia
HyperCard
Multimedia Document Architecture
Document
A document consists of a set of structural information using which some content is displayed.
A document is aimed at the perception of a human and is accessible for computer processing.
A multimedia document is a document which comprises of information coded in at least one continuous (time dependent) medium and in one discrete (time independent) medium.
Document Architecture
It specifies the content of a document, the physical layout of document like pages and frames and logical layout of the document like paragraph, sections, headings etc.
For exchanging of documents between different applications, the documents themselves must adhere to a standard architecture.
Standardized Architectures
SGML (Standard Generalized Markup Language)
ODA (Open Document Architecture)specifically for multimedia ....
OMF (Open Media Framework)
MHEG (Multimedia and Hypermedia Information Coding Expert Group)
DCA (Document Content Architecture)
MODCA (Mixed Object Document Content Architecture)
SGML
It prescribes a standard format for embedding descriptive markup within a document.
SGML allows you to set up hierarchical models for each type of document produced.
It forces each element in the structure (labeled chapter, title, paragraph) to fit in the logical, predictable structure of document.
A typical document has 3 layers – structure, content and style.
SGML ...
1. StructureDTD – (Document Type Definition) which provides a framework and rules for the definition of the elements that constitute a document.
A DTD accompanies a document wherever it goes.
SGML ...
2. Content
Content is the information itself. Creating an SGML document
involves inserting tags around content.
<topic><par>Multimedia is interesting !! </par></topic>
3. Style
2 standard based style sheets :
FOSI – Formatting Output Specification Instance
DSSSL – Document Style Semantics and Specification Language
SGML Structures1. Elements – element names are always case – insensitive. It has 3
parts, start tag, content, end tag [<pre> example </pre>]
2. Attributes – attribute/value pair appear before the final '>'
[<h1 align = 'center'> ................. </h1>]
3. Entities – named part of a marked up document, irrespective of
any structural considerations. When parser encounters any entity
reference, it immediately substitutes the value declared for the
entity name.
SGML DTD
Rules are the first stage in the creation of a formal specification for the structure of an SGML document ie SGML DTD
Minimization Rules determine whether or not start and end tags must be
present in every occurrence of the element concerned.
Content Model for specifying the content in terms of other elements or
using special reserved words.
Occurrence indicators, group connectors and entity definitions together
make the DTD.
Example
<! ELEMENT poem - - (title?, stanza+)>
<! ELEMENT title - 0 (#PCDATA)>
- : must be present (start tag is present for both)
0 : may be omitted. (for title, end tag may be omitted)
#PCDATA : valid character data
?, *, + : Occurrence indicators
ODA
The main property of ODA is the distinction among content, logical structure and layout structure. This is in contrast to SGML where only logical structure and contents are defined.
1. The ODA Architecture – enables compound documents to be
encoded and interchanged electronically between systems
conforming to the standards.
2. Logical, layout and content information – indept of page layout
3. Generic structures – reports as defining class of the document.
4. Style – Layout (figure + caption) and presentation(indentation of
first line)
5. The document profile – author name, create date, keywords etc
MHEGAn MHEG document can be distributed over the network or in
CD ROM format, but for presentation only, no editing is possible.
MHEG is intended to produce electronic multimedia versions
of printed books.
It supports encoding of interactive multimedia presentations
defining a platform independent set of multimedia objects
like buttons, text fields etc. Facility of an embedded
scripting language for event handling is also provided.
MHEG ...
It is based on object hierarchy where each object has a set of components and behaviours.
MH-Object
Behavior
Action
Link
Script
Component
content
Interaction
Descriptor
HyTIME (extension of SGML)
It is meta language similar to SGML. The structure allows for:
1. The specification of spatial interrelations between
document elements.
2. The specification of temporal interrelations between
document elements (synchronization and time
dependencies)
3. The specification of the overall logical structure
4. The specification of pointers between elements within or
outside the document.
OMF
It is a standard format for the interchange of digital media data among heterogeneous platforms. The primary emphasis is video production.
1. Source (analogue) material object represent videotape and film
so that the origin of the data is readily identified.
2. special track types store (SMPTE) time codes for segments of
data.
3. Transitions and effects for overlapping and sequences of
segments are predefined.
OMF ...
The OMF provides structures for 3 distinct elements :
1. Digital media data – audio, video and graphics for m only part of the
information.
2. Media Sources – describes the original physical sources of data.
3. Composition – describe the arrangement of sections of sources and
how they are played over time.
OMF interchange keeps sources separate from compositions. More than one
composition can reference sections of the same digital media data, reducing
storage requirements.
OMF ...
OMF uses objects called Mobs (media Objects) to uniquely describe compositions and sources. Each Mob contains an identifier called a MobID
It uses SMPTE time code for numbering frames.
It also has support for motion control.