clayvision-yuichiro takeuchi and ken perlin-works

Presentation on

ClayVision:The Elastic Image of the City

By,

Darshan Mehta

3/8/2014 1

Contents

Introduction.

History.

Requirements.

Working.

Applications.

Advantages.

Disadvantages.

Future Scope.

Conclusion.

References.

3/8/2014 2

Introduction

Q. What is Clayvision?

It is a new quasi-immersive urban navigation

system that rethinks the design convention of

existing Augmented Reality (AR) applications.

Instead of “information bubbles” onto the urban

scenery, ClayVision communicates through real-

time 3D transformations and video feed of the city

elements.

3/8/20143

In other words, the system reassembles the city into

a better-designed copy of the original, that is both

easier to navigate and tailored to suit the user’s

needs and preferences.

3/8/2014 4

History

There were many experimental systems built in the

mid 1990s ,which were marked by their bulky

setups and low frame rates.

Later on, due to the devices lacking the graphical

capabilities had to send camera images to the server

for each frame and increased computational burden.

3/8/2014 5

Contd..

The further developed AR technique lacks accuracy.

“Information bubble” display , the bubbles do not

have any absolute, exact positions within the 3-D

space.

3/8/2014 6

Requirements

High Speed Wireless Internet Connection.

Hardware Device having Camera and Display

Screen.

A 3-D Graphics Engine

-part of system.

-handles graphical simulation and interfaces.

3/8/2014 7

What is API ?

-protocol used as interface by

software for comm.

-routine , data structure, obj.

class….etc

-libraries in c++ ,java API

3-D Graphics Engine

API

OpenGL ?

GLUT ?

3/8/2014 8

Application Programming Interface

Q. What is API ?

-protocol used as interface by software for

communication.

-routine , data structure, objects,Class etc.

-libraries in c++ ,java etc.

3/8/2014 9

OpenGL

OpenGL –Graphics Library.

OpenGL is used for graphics routine and other

coding.

OS independent.

Widely used in industry.

3/8/2014 10

GLUT is OpenGL Utility Toolkit

◦ Not part of OpenGL.

◦ “GLUT is designed for constructing small to

medium sized OpenGL programs.”

What is GLUT?

3/8/2014 11

OVERALL FLOW:

DATA IS DISPLAYED

USE OF 3-D

GRAPHICS ENGINE

DATA IS SENT TO

HARDWARE

DEVICE

CORRELATION OF

CO-ORDINATES

DATA SENT TO

SATELLITE

IMAGE

PROCESSING

IMAGE CAPTURING

HARDWARE

DEVICE USED AS

INPUT

3/8/2014 12

Image Processing:

Image processing of the video feed is done using

SIFT, which outputs a set of feature points and other

data used to determine the relative position of the

entire frame.

Scale-invariant feature transform (or SIFT) is an

algorithm in computer vision to detect and describe

local features in images.

Output is used to compare the video feed to the

database of pictures and the template pictures are

transformed based on the device’s specifications to

produce the correct pose.

3/8/2014 13

Contd.. After localization, projection and modelview

matrices are calculated to map 3D building models

onto the feed.

These models are then textured using information

from the feed and transformed to communicate

information to the user.

Texturing is done correctly by altering the image

background with template picture information in a

way that doesn’t disrupt the video and allows for

transformations that don’t cause excessive errors.

3/8/2014 14

Each frame of the real-time video feed is compared

to a collection of photos, shot from the same

location using a device.

Attributes such as building shapes , colours ,

materials are modified so that they represent useful

information, thus increase the efficiency of visual

communication.

This is the approach taken by ClayVision.

Working

3/8/2014 15

In Model Mapping, The 3-D model of the requiredsector is generated.

3-DMODEL MAPPING

Applications:

3/8/2014 16

Buildings can be emphasized by strategically

changing their visual attributes.

Attaching fake facades , enhancing the height ,

adding false saturation levels can make us find the

given targeted building more quickly.

Navigation:

3/8/2014 17

Navigation Comparisons:

Contd..

3/8/2014 18

It is will be very useful in the fields of urban

planning and design.

Navigation systems can be upgraded to a new

level using clayvision technology.

3-D Model Mapping is implemented using

Clayvision Technology

Advantages

3/8/2014 19

This design attracts a significant part of the user’s

attention, which may result in the user becoming

less attentive to other pedestrians, cars, etc., creating

a serious safety risk.

Slowdowns may occur, especially because of

network speed.

Disadvantages

3/8/2014 20

Panorama creation: It can converge all major city

elements into a single screen.

Straightening streets: There are streets crooked,

extending in seemingly random directions. Clay

vision provides a clear view of what can be found

further down the road.

Manual interaction: Tapping or drawing on the

screen can be implemented for example cutting a

hole in a building to see what lies beyond.

Future Scope

3/8/2014 21

Thus ClayVision is a novel vision-based augmented

reality system that offers the experience of real-time

urban design. It is a set of techniques to enable

freeform transformations of built elements in the

city, and discussed a range of transformation

operations and their implications on the urban

experience.

Conclusion

3/8/2014 22

Yuichiro Takeuchi and Ken Perlin-Works on Clay

vision technology(IEEE Paper)

References

3/8/2014 23

3/8/201424

Thank You