real-time human posture reconstruction in wireless smart camera networks chen wu, hamid aghajan...
TRANSCRIPT
Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks
Chen Wu, Hamid Aghajan
Wireless Sensor Network Lab, Stanford University, USA
IPSN 2008
Speaker Lawrence
Outline
• Background• Motivation• Goal• Challenge• Strategy for Camera Sensor Network• System Overview• Wireless Smart Camera (Hardware)• Human Pose Estimation (Algorithm)• Result• Conclusion
Background
• Traditional Camera network for surveillance & security
• New applications of camera network for multimedia, video conference…etc
• Wireless Camera network – Scalability– Privacy preservation– Flexibility on collaboration scheme between cameras
Motivation
• As pervasive sensors the cameras can free the users from wearable devices.
• Lack of real-time vision algorithm to achieve moderate complexity, robustness and scalability.
Goal
• Implementation of human pose interpretation on a wireless smart camera network.
• Employing distributed processing– Real-time vision & scalability for complex vision algorithms.
Challenge
• A vision sensor network poses three key challenges:
– High computation capacity for real-time performance.
– Wireless links limit image transmission (bandwidth & energy)
– Lack of established vision-based fusion mechanisms (by real time)
Strategy for Camera SN
• Difference between Camera network & Distributed vision processing strategy systems.– Employ cameras as a wireless sensor network.
• Strategy:
1. Video data reducing (Network bandwidth)
2. Level of vision analysis to different PHY processors
Strategy for Camera SN (cont.)
Central PC
SmartCamera
Level of vision analysis to different PHY processors
Scalability : Spatial and functional parallelism
• Each camera video processes its own data(spatial)• Running their own function modules(functional)
Strategy for Camera SN (cont.)
Smart camera communicate with the central PC through ZigBee
System Overview
LCD display
Smart camera
Different ZigBee channels
Data flow in the system
System Overview (cont.)
Semaphore tech for DPRAM
P.S. DPRAM allows multiple r or w to occur at the same time.
Asynchronous
Wireless Smart Camera
• Hardware Platform– VGA color image
sensor– SIMD
processor(IC3D)– Embedded
processor(8051)– ZigBee platform
• Parallel arch power consumption
• LPA(320 PEs) data processing
• GCP control IC3D & DSP operations
• PE # video format, e.g., VGA(640*480)
• The main design factors of SIMD frequency & PE #
Wireless Smart Camera (cont.)
MP-SIMD
• Data sharing between processors– PDRAM functions as an asynchronous connection
between IC3D and 8051– Semaphore tech to prevent mutual access
• Wireless communication– P2P structure offers direct camera to PC communication– Maximum data rate : 100 Kbit/sec
Wireless Smart Camera (cont.)
• Review (Algorithm)– Goal : 2D to 3D– Ambiguity: Perspective views of the camera or self-
occlusion of human body
• Pose Estimation Approach– Top-down– Bottom-up
Human Pose Estimation
Top-down vs Bottom-up
• Top-down– Strength
• Occlusion handling• Contours & body
part association
– Weakness• Search tech
complexity(depth)• Computational
load(projection)
• Bottom-up– Strength
• Much less demands on 3D switch
– Weakness• Complex assemble• Difficult to detect
occlusions
Human Pose Estimation (cont.)
• Challenges & Method– Bandwidth constraint
(100Kbits/sec)/(30frames/sec)/(8bits/Byte) ≈ 400B/frame
solution: Detect body part cancroids coordinates
– Limited image processing capability of the SIMD processorsolution: Color segmentation
– Robustness with varied environment solution: Auto-balancing
filtering & combination
Human Pose Estimation (cont.)
In-node processing
• Detect positions(x, y):– Head, shoulders and hands– 2Bytes for x and y
• Detect mechanism:– Face -> face color model– Head -> skin color model– Shoulders -> shirt color model (low-pass filter)
Human Pose Estimation (cont.)
• Processing on the central PC
– Noise filtering and 2D to 3D reconstruction
Human Pose Estimation (cont.)
Standard Deviation of detected body part coordinates in the smart cameras (in pixels) and those after noise filtering
Results (cont.)
Demo
Original data from the smart cameras and data after noise filtering
Head
Left shoulder Right shoulder
Results (cont.)
Conclusion
• Propose an algorithmic strategy to approach vision problems in a wireless camera sensor network
• Major aspect of the strategy:– reduce video data locally through smart camera
• Implement a prototype system of 3D human reconstruction using a wireless smart camera.
• Wireless camera networks will offer potentials for user-centric applications.