computer assisted manufacturing

26
PISA Production, Indexing and Search of Audio-visual Material Computer Assisted Manufacturing Dieter Van Rijsselbergen, IBBT – MMLab – UGent

Upload: vrt-medialab

Post on 08-May-2015

1.803 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Computer Assisted Manufacturing

PISAProduction, Indexing and Search

of Audio-visual Material

Computer Assisted ManufacturingDieter Van Rijsselbergen, IBBT – MMLab – UGent

Page 2: Computer Assisted Manufacturing

35

PISA - Overview

Abstract

Information

Footage

Concept

Virtual

Model

Model Driven Development:

• Setting (Stage properties, light)

• Character

• Synthetic Speech

• Sound effects

• Character animation

• Virtual camera

VirtualModelling

AutomatedProductionRealisation

• Ingest

• Editing

• Mastering

• Reproduction to alternative distribution channels

Computer Assisted Design

Computer Assisted Manufacturing

Script Editing

• Parse scenario

• Shooting script editor

• Storyboard

Script Editing

Reverse Engineering

• Shot segmentation

• Video footprint and reuse detection

• Biometric face detection

• Background analysis

• Speech-to-text

Interpretation

• Character identification

• Background categorisation

and identification

• Topic and eventdetection

Intelligent Analysis and

Quantization

Quantization

Analysis

Indexing

Retrieval

• Timecode based indexing

• Geo-temporal reference

• Taxonomy based indexing and search

• Facetted search

Search Engine

Page 3: Computer Assisted Manufacturing

36

Computer Assisted Manufacturing

! Computer Assisted Manufacturing is about automation

of media production processes

! Based on extensive production metadata

! high level Product Engineering for narratives

! low level Manufacturing Engineering for actual

manufacturing specifications

! Using clever algorithms to fill in the gaps

! supported by Trouvaille

Page 4: Computer Assisted Manufacturing

37

Drama Manufacturing Processes

Product Engineering

Manufacturing Engineering

Manufacturing

Synopsis

Writing

3-D

Previsualization

Dialogue

Writing

Shooting

ScriptingEditing

Quality

Assurance

Acquisition? ? ?

Page 5: Computer Assisted Manufacturing

38

Objectives

! Extend manufacturing workflow with 3-D previsualization

! Build evolving products

! Re-use previsualization parameters

during manufacturing

! Increase efficiency and automation in production

processes

! 1) Rough Cut Editing based on 3-D previsualization

! 2) Repurposing for multiple output channels

Page 6: Computer Assisted Manufacturing

Computer Assisted

Manufacturing

Rough Cut Editing

Page 7: Computer Assisted Manufacturing

40

Rough Cut Editing

! Inclusion of 3-D previsualization into thedrama manufacturing process

! Use previsualization as shooting script

! Integration into the MAMS and Craft Editing systems

! Building evolving products

!mixture of previsualized and shot footage

! evolve from all-previsualized to all-shot composition

Page 8: Computer Assisted Manufacturing

41

Rough Cut Editing Processes

Product Engineering

Manufacturing Engineering

Manufacturing

Synopsis

Writing

Previz.

Rendering

Dialogue

Writing

EditingQuality

Assurance

Acquisition

3-D

Previsualization

Page 9: Computer Assisted Manufacturing

Computer Assisted

Manufacturing

Repurposing

Page 10: Computer Assisted Manufacturing

43

Computer Aided Repurposing (CAR)

! Automated Repurposing of Drama

! Different output channels

!HDTV

!SDTV

! iPod/Cellular

! True Universal Media Experience

! requires semantically sensible scaling/cropping

Page 11: Computer Assisted Manufacturing

44

Repurposing Concept

! Automated Repurposing of Drama

! Spatial adaptation using semantic Objects of Interest

! as in “focus on Capt. Kirk”

! decided by creative staff in function of the narrative

! is a form of Manufacturing Engineering

! can be defined in early 3-D Previsualization

! preparation not only of shooting script

but also of adaptations

! parameters refined in Quality Assurance

Page 12: Computer Assisted Manufacturing

45

Repurposing: How?

! 1) Determine where the semantic elements are

! Semantic Objects of Interest

are linked with regions of interest

!manual annotation

! automated

algorithms

! aided by

Product

Engineering

metadata!Jenny Waldek

Page 13: Computer Assisted Manufacturing

46

Repurposing: How?

! 2) Provide reframing instructions, e.g.:

! Object Orientation and Range

! “The Bad must be shown on the right…”

! Object Size

! “… and I want it to be a full shot of The Bad.”

0.75

Frame SizeObject of Interest

Size: (30%, 90%)

Page 14: Computer Assisted Manufacturing

47

Repurposing: Channels

! HDTV

! Originally acquired

high definition material

! SDTV

! Legacy applications

! Common pan/scan

! Editing

(AVID)

! Mobile (iPod)

! Semantically

repurposed

Page 15: Computer Assisted Manufacturing

48

Repurposing Processes

Product Engineering

Manufacturing Engineering

Manufacturing

Synopsis

Writing

3-D

Previsualization

Previz.

Rendering

Dialogue

Writing

EditingQuality

Assurance

AcquisitionEDL

RenderingRepurposing

Repurposing

Scripting

Page 16: Computer Assisted Manufacturing

49

Workflow (1): Synopsis

Page 17: Computer Assisted Manufacturing

50

Workflow (2): Management

Page 18: Computer Assisted Manufacturing

51

Workflow (3): 3-D Previz. – Scenario

Page 19: Computer Assisted Manufacturing

52

Workflow (4): 3-D Previz. – Shooting Script

Page 20: Computer Assisted Manufacturing

53

Workflow (5): 3-D Previz. Editing

Page 21: Computer Assisted Manufacturing

54

Workflow (6): Acquisition

Page 22: Computer Assisted Manufacturing

55

Workflow (7): Editing

Page 23: Computer Assisted Manufacturing

56

Workflow (8): Quality Assurance & Analysis

Plane

Page 24: Computer Assisted Manufacturing

57

Workflow (9): Repurposing

Page 25: Computer Assisted Manufacturing

58

Workflow (10): Conform & Publish

Page 26: Computer Assisted Manufacturing

59

Computer Aided Manufacturing: Workflow