Human Arm TrackingTRAINING A CRS ROBOTIC ARM USING HUMAN ARM IMITATION THROUGH KINECT

MICROSOFT

KINECT SENSOR

It consists of:

• RGB Camera.

• Two Infrared sensors which act as a depth sensor.

• Four element microphone array.

In default range mode, Kinect can see people standing between 0.8 meters and 4.0 meters away.

Users will have to be able to use their arms at that distance, which narrows down to a practical range of 1.2 to 3.5 meters.

In the near mode, Kinect has a practical range of 0.8 to 2.5 meters.

[1]

CRS ROBOTIC

ARM

The robotic arm has 6 degree of freedoms:

We wouldn’t be using JOINT 6 as the human arm doesn’t has this degree of freedom.

[2]

[3]

D-H ParametersThe Denavit–Hartenberg parameters (also called DH parameters) are the four parameters associated with

a particular convention for attaching reference frames to the links of a spatial kinematic chain.

θi - the angle between the axes, Xi-1 and Xi,

about the axis Zi .

di - the distance between Xi-1 and Xi

along Zi.

ai - the distance between the common normals

to axes Zi and Zi + 1

along Xi.

αi - the angle between the axes, Zi and Zi + 1, about

the axis Xi.

Using the above 4 parameters we could define a reference for each link of our robotic arm.

[4]

Kinect Skeletal API

Microsoft Kinect sensor provides us with the API to track a human body. The Kinect keeps track of the joints of our arm.

We are using this to provide us with the 3D coordinate of the joints which we would be further using to calculate the joint angles using the known D-H parameters of the arm.

[5]

Calculating Joint Angles• We obtain the upper extremity joint position measurement from the kinect sensor, ex., [x y z].

• To calculate joint angles at the 6-DOFs robot manipulator given the end-effector position measurement, we apply the iterative Newton method for solving the inverse kinematics problem.

• The algorithm can be described by

• This will be one of the major challenges.

• The robot is currently controlled by either a manual controller or using a windows application RobCoMM (the OS of the robot)

• The Robot was discontinued in around 1995

• We do not have much info on how to control the bot in realtime by sending commands through the serial port

• The data is transmitted to the robot through Serial Communication

Transmitting Data To The Robot

[6]

Local Optimization and Application

Each of the configuration of the robotic arm is a point in a 5-dimensional space.

While we operate the robotic arm using our hand, it is possible that we might have not taken the optimal path from start to end configuration .

So we could remove some redundant states from our path to optimize our path.

Algorithm: We would connect each state with it’s k nearest neighbors. Then in the new graph we would find the shortest path between the start and the end state using the Dijkastra’s Algorithm. After that we would only keep the states found on the above path and discard the other states.

Thus we can now use the robot to efficiently perform the task any number of times.

References• [1] http://gamesforkinect.org/kinect-information/how-does-the-kinect-sensors-work/

• [2] http://cmp.felk.cvut.cz/cmp/hardware/A465/A465.html

• [3] http://cmp.felk.cvut.cz/cmp/courses/ROB/labsmaterial/CRS/CRS-uvod.htm

• [4] Figure 3.4, Page 66 Introduction to Robotics By John J. Craig.

• [5] http://msdn.microsoft.com/en-us/library/hh973074.aspx

• [6] http://www.doom9.org/index.html?/DigiTV/dbox-howto.htm

• Real-Time Human Pose Recognition in Parts from Single Depth Images

http://research.microsoft.com/pubs/145347/BodyPartRecognition.pdf

• Introduction to Robotics By John J. Craig.

Thank YouAYUSH VARSHNEY

RITESH GAUTAM