from 3d scene geometry to human workspace
DESCRIPTION
From 3D Scene Geometry to Human Workspace. Authors: Abhinav Gupta, Scott Satkin , Alexei A. Efros and Martial Hebert The Robotics Institute, Carnegie Mellon University Presenter: Jiapei Zhang. Project Undertaking. Image if I were Abhinav - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/1.jpg)
From 3D Scene Geometry to Human Workspace
Authors: Abhinav Gupta, Scott Satkin, Alexei A. Efros and Martial Hebert
The Robotics Institute, Carnegie Mellon UniversityPresenter: Jiapei Zhang
![Page 2: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/2.jpg)
Project Undertaking
Image if I were Abhinav
I received a project from my boss about indoor scene understanding
![Page 3: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/3.jpg)
Selection of Subject Material at hand
• A set of indoor-scene images• Motion Capture Database• No LiDAR scanner, not as rich as VCC, China• How I wish to be a VCCer one day
VCC V5
![Page 4: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/4.jpg)
Selection of Project Inspirations from Psychology
Where is the “entrance” of my novelty? Where is the “Observer” for traditional Image Understanding? Put Human back to Image Gestalt Psychologists: Objects are best defined not by their
identity but rather by their function. The meaning or a value of a thingconsist of what it affords. -----JJ Gibson(1979) “Affordance” “functionality”
![Page 5: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/5.jpg)
Selection of Project Shoulder of Giants
Bounding Box Level == Object DetectionSingle-View Indoor Geometry EstimationHuman Pose Analysis via Motion Capture
![Page 6: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/6.jpg)
Selection of Project Give a Final Word
Propose a scene representation that combines 3D scene geometry with a set of possible human actions, to create a joint space of human-scene interactions.
Predict the full human workspace of the scene
3D Scene Understanding
HUMAN ACTION MODELING
![Page 7: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/7.jpg)
Implementation Detail Representing Scene Geometry
An indoor scene is modeled by the layout of room(walls, floors, ceilings) and the location/layout of objects.
The objects are modeled by a set of occupied voxels.
![Page 8: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/8.jpg)
Implementation of Detail Geometry Estimation
Sliding Cuboids
Selection
![Page 9: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/9.jpg)
Implementation of Detail Qualitative Human Pose Representation
Each pose is represented by the occupied blocks in discretized 3D space(red) and the required surfaces of interaction(green).
![Page 10: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/10.jpg)
Implementation of Detail Human-Scene Interactions
Constraints
Binary Occupancy Matrix “Space Search” to fit “Constrains”
Free space constrain
Support constrain
![Page 11: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/11.jpg)
Harvest Season Result Showing
![Page 12: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/12.jpg)
Retrospect and Meditation
• Now return back to me• What useful info. for our Indoor-Scene Project
I could extract from this paper ?
![Page 13: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/13.jpg)
Retrospect and Meditation Incorporate images to our projects Will geometry estimation improve our rough mesh model?
Categorize “mobility” of scene into two:a) Extract from Repetitions: Based on Geometry b) Extract from “Human Pose”: Based on Functionality
![Page 14: From 3D Scene Geometry to Human Workspace](https://reader036.vdocuments.site/reader036/viewer/2022062305/568164a6550346895dd69734/html5/thumbnails/14.jpg)