Convex Optimisation for Inverse Kinematics

Tarun Yenamandra1,2, Florian Bernard1,2, Jiayi Wang1,2, Franziska Mueller1,2, and Christian Theobalt1,2

1MPI Informatics, 2Saarland Informatics Campus

Abstract

We consider the problem of inverse kinematics (IK),where one wants to find the parameters of a given kinematicskeleton that best explain a set of observed 3D joint loca-tions. The kinematic skeleton has a tree structure, whereeach node is a joint that has an associated geometric trans-formation that is propagated to all its child nodes. The IKproblem has various applications in vision and graphics,for example for tracking or reconstructing articulated ob-jects, such as human hands or bodies. Most commonly, theIK problem is tackled using local optimisation methods. Amajor downside of these approaches is that, due to the non-convex nature of the problem, such methods are prone toconverge to unwanted local optima and therefore require agood initialisation. In this paper we propose a convex op-timisation approach for the IK problem based on semidef-inite programming, which admits a polynomial-time algo-rithm that globally solves (a relaxation of) the IK problem.Experimentally, we demonstrate that the proposed methodsignificantly outperforms local optimisation methods usingdifferent real-world skeletons.

1. Introduction

The inverse kinematics (IK) problem plays an importantrole in robotics, computer games, graphics, and vision, asit is a fundamental building block for animating, control-ling, tracking and reconstructing articulated objects, suchas robotic arms or human bodies. The IK problem refersto the task of recovering parameters of a kinematic skele-ton (e.g. joint angles), given a set of observed locations for(some of) the joints. The kinematic skeleton is representedas a tree, where each node is a joint that has translationaland/or rotational degrees-of-freedom. In forward kinemat-ics, one computes the positions of the joints given the trans-lation and rotation parameters. Here, the transformation foreach joint is defined relative to its parent joint, so that achain of transformations is applied along the path from theroot node to a leaf node (end-effector). Inverting this pro-cess, i.e. given some of the joint positions, one wants torecover the kinematic parameters that lead to this joint po-

Ground Truth Local SDP-IKFigure 1. Our SDP-IK method results in a better fit compared tolocal optimisation. Left: Ground truth pose, where the observedpoints are shown in red. Centre: Local IK methods can get stuckin an unwanted local minimum. Right: Our convex SDP-IK for-mulation can reliably reach a good optimum that is much closer tothe ground truth.

sition configuration.

The IK problem is considered to be difficult due to sev-eral reasons. On the one hand, depending on the kinematicskeleton and the observed joint positions, the inverse kine-matics can be ill-posed so that there may be multiple config-urations of kinematic parameters that lead to the same jointpositions. On the other hand, depending on the given jointlocations, a solution that produces an exact fit to the obser-vations might not exist. Lastly, the resulting optimisationproblem is non-convex, which makes it generally difficultto find a globally optimal solution. Hence, approximationsare oftentimes used in practice, which in turn require aninitialisation that is sufficiently close to a global optimum.In computer vision, one of the most dominant applicationsof the IK problem is tracking and reconstructing articulatedobjects based on a temporal sequence of data (e.g. depthimages, or RGB images). A common approach for tack-ling tracking applications is to initialise the kinematic para-meters for the next frame using the tracked result from thecurrent frame, and then solve the IK problem using local

arX

iv:1

910.

1101

6v1

[cs

.LG

] 2

4 O

ct 2

019

optimisation methods. However, this is not possible for thevery first frame of a sequence. Hence, in many works theauthors assume that a good initialisation is available for thefirst frame, e.g. based on an initial calibration with a neutralpose, as done in [33]. In contrast, our proposed method isentirely initialisation-free, and is therefore well-suited forhandling such cases. The main contribution of this workis a polynomial-time solution of the IK problem, coinedSDP-IK, that finds a global optimum of a (convex relax-ation of) the IK problem based on semidefinite program-ming.

2. Related WorkIn this section we first address local optimisation meth-

ods for the inverse kinematics problem, followed by globalmethods. Subsequently, we summarise the most relevantworks that consider semidefinite programming relaxationsof related optimisation problems.

Local IK approaches: Local optimisation methodsseek to iteratively find a solution of the IK problem basedon a given initial estimate. One class of such meth-ods use the first-order Taylor approximation of the prob-lem and attempt to solve a linear system characterisedby the Jacobian matrix. Alternatives are the Moore-Penrose pseudoinverse method, the Jacobian transposemethod (equivalent to gradient descent for least-squares er-ror), the Levenberg-Marquardt method (equivalent to gra-dient descent for damped least-squares error), and othervariants [6, 10, 14, 20]. Second-order methods also ex-ist, but such approaches require the computation of theHessian of the forward kinematics function, which incurhigher computational cost. Quasi-Netwon methods, such asthe Broyden-Fletcher-Goldfarb-Shanno (BFGS) algorithm,have been used to provide faster approximations for solvingthe IK problem [23].

Instead of trying to approximate and invert the forwardkinematics function, heuristic methods employ simple rulesto be iteratively followed and can often reach the IK so-lution. Cyclic coordinate descent (CCD) and its variantsare heuristic methods that seek to minimise joint errors bychanging one kinematic parameter at a time [27, 38]. For-ward and backward reaching IK (FABRIK) [4] provides amethod that can provably converge to the correct solution,when feasible, for a single unconstrained kinematic chain[3]. Evolutionary algorithms such as particle swarm opti-misation and genetic algorithms, are heuristics inspired byevolution and are used to solve the IK problem in [34, 35].

Due to their speed and ease-of-use, local optimisationmethods are widely used in applications such as charac-ter animation [20], motion re-targeting [19], model-basedtracking [33, 36], post-processing on pose estimation [26,28], and data visualisation [27]. However, despite their pop-ularity, local IK approaches require a good initialisation for

complex, real-world skeletons, as otherwise they are proneto converge to unwanted local optima.

Global IK methods: Global IK methods aim to avoidthis tendency getting stuck in local optima, and instead seekto obtain a global solution. One way of of achieving initiali-sation independence is based on training a machine learningmodel to solve the IK problem. In [11], the authors learnan inverse kinematics function which maps from joint lo-cations to kinematic parameters using a structured learningmethod.

An alternative to learning approaches are global optimi-sation methods. In [16], the authors tackle the problem offorce-closure grasp synthesis based on sequential semidefi-nite programming, where rotation constraints are modelledin terms of bilinear matrix inequalities involving quater-nions and rotation matrices. An approach for a feasibil-ity formulation of the IK problem based on mixed-integerprogramming (MIP) is presented in [15]. The main ideahere is to discretise all non-convex constraints based on bi-nary variables. The resulting problem is then solved with abranch and bound algorithm, which is known to have expo-nential worst-case time complexity.

Contrary to the discussed works, which address fea-sibility versions of the IK problem, and/or do not ad-mit polynomial-time algorithms, we propose a principledpolynomial-time approach for the IK problem. Moreover,we consider a least-squares version of the IK problem,which is most relevant for the majority of applications invision and graphics, such as for the tracking or the recon-struction of articulated objects.

Semidefinite programming relaxations: Semidefiniteprogramming (SDP) relaxations are a popular way for tack-ling non-convex optimisation problems. Such methods havebeen successfully used for a range of different problemsin vision and beyond, e.g. for graph matching [32, 40] ormulti-graph matching [7, 21], the rigid registration of point-clouds [22, 25], the segmentation of images [39], or for per-mutation synchronisation [13]. However, generally such ap-proaches are computationally expensive, since many of theSDP relaxations are based on a lifting of the variables, sothat the size of the optimisation problem increases quadrat-ically when moving from the original non-convex problemto the convex relaxation [21, 32, 40].

One scenario where SDP relaxations particularly standout is in problems involving 3D rotation matrices. On theone hand, lifting a matrix variable of size 3×3 merely re-sults in a relatively small variable of size 1+(3·3)2=82, sothat such problems can be solved efficiently. On the otherhand, some relaxations that involve a single rotation ma-trix have been observed to be tight in practice, i.e. evenwhen solving a relaxation of a non-convex problem, theso-found solution is a global minimiser of the originalnon-convex problem. This has for example been empiri-

Figure 2. Illustration of a kinematic skeleton of a human hand (left)and a human body (right). The root joint for both skeletons admits6 DOF (global translation and rotation), whereas all other jointsadmit between 0 and 3 rotational DOF.

cally demonstrated in [9] for the registration of 3D objects(with known correspondence). Other approaches that con-sider SDP approaches for problems involving rotations havebeen demonstrated for SLAM [30], pose-graph optimisa-tion [12], or rotation averaging [5, 17, 37].

In our work we consider an SDP relaxation for the in-verse kinematics problem, which involves a composition ofseveral rotations that are propagated through the kinematicchain.

2.1. Notation

Here, we briefly outline the used notation. By Id we de-note the d×d identity matrix, by 0d ∈ Rd we denote thevector of all zeros, and the operator vec(X) vectorises agiven input matrix X by concatenating all the columns ofX . For an integer n we use the notation [n] := {1, . . . , n}.For a matrix X we write X:,i to denote the vector that isformed by the i-th column of X , and analogously Xi,: todenote the row vector that is formed by the i-th row of X .Moreover, for a 3D vector x ∈ R3, or a matrix T ∈ R3×3,we use x = [xT 1]T ∈ R4 and T ∈ R4×4 to denote theirrespective representation in homogeneous coordinates. ForX being a matrix, the notation X�0 means that X is sym-metric positive semidefinite.

3. Inverse Kinematics

In this section we describe our approach for tackling theinverse kinematics problem. To this end, we first define theforward kinematics model, which is followed by the precisestatement of the problem.

3.1. Forward Kinematics Model

We assume that we are given a tree that defines thekinematic skeleton, see Fig. 2. The root of the tree has6 degrees-of-freedom (DOF), of which 3 account for theglobal rotation, and 3 account for the global translation.Each non-root joint has between 0 and 3 rotational DOF,where each (non-zero) DOF accounts for a rotation of agiven angle around a given axis. Joints without DOF areused to model the end-effectors, e.g. the fingertips in thehuman hand, cf. Fig. 2.

One-DOF representation: W.l.o.g., for convenience,we redefine this generic kinematic skeleton in such a waythat each (non-end-effector) joint only has a single DOFthat allows for a rotation around a given axis. To this end,for each joint of the original skeleton that has more than 1DOF, we simply introduce one or two additional auxiliaryjoints (placed at the same position) that account for the addi-tional DOF. We emphasise that while this redefinition of thekinematic skeleton does not change its kinematic behaviour,such a representation is more convenient for defining ourSDP relaxation, as we will become apparent in Sec. 4.2.

Kinematic skeleton: Let J ∈ N be the total numberof joints, where each joint now has at most 1 DOF (due tothe redefinition of the skeleton). The global translation ofthe root is denoted by t ∈ R3. For each subsequent jointtransformation, let aj ∈ R3 for j ∈ [J ] denote the (unit-length) rotation axis of the j-th joint, and let vj ∈ R3 be the“bone-vector” of the j-th joint, i.e. the offset of joint j inthe coordinate system of its parent (cf. Fig. 2). For θj ∈ Rbeing the parameter of the j-th joint, by T (aj , θj) ∈ R4×4

we denote the transformation from the coordinate system ofjoint j to the coordinate system of its parent, represented inhomogeneous coordinates. To be more specific, we have

T (aj , θj) =

[K(aj , θj) vj

0T3 1

], (1)

where the rotation matrixK(a, θ) is obtained by Rodrigues’rotation formula as

K(a, θ) = I3+sin(θ)[a]×+(1− cos(θ))[a]2× . (2)

Here, [·]× is the skew-symmetric operator that gener-ates a 3×3 matrix from a 3D vector (i.e. for x, y ∈ R3

we have [x]×y = x × y). For joints with 0 DOF (i.e. forend-effectors), we define K(a, θ) to be the identity matrixI3.

Forward model: For Θ := [tT , θ1, . . . , θJ ]T ∈ R3+J

being the parameter vector that stacks all joint parameters,the forward model for computing the position xj(Θ) of thej-th joint is given by

xj(Θ) = t+

∏i∈α1

j

T (ai, θi)

03 , (3)

where by αnm we denote the path from the m-th joint tothe n-th joint (from children to root) in the kinematic skele-ton, and 03 = [0 0 0 1]T is the zero vector represented inhomogeneous coordinates. For brevity, we will refer to allelements of Θ as angles, even if they represent translations.

Joint angle constraints: In addition, for each joint thereis an interval Ij that defines the range of valid values forθj so that it must hold that θj ∈ Ij . For notational con-venience, we define I := R3 × I1 × . . . × IJ and writeΘ ∈ I.

3.2. Problem Statement

We are interested in the problem of finding the param-eters Θ such that the forward kinematics model best ex-plains a given set of 3D joint position observations. LetJ ⊂ [J ] denote the subset of joints for which the 3D po-sitions yj ∈ R3, j ∈ J , are known. The IK problem cannow be phrased as a (constrained) nonlinear least-squaresproblem that reads

minΘ∈I

∑j∈J‖yj − xj(Θ)‖2 =: f(Θ) (4)

⇔ minΘ∈I

∑j∈J‖yj − (t+

∏i∈α1

j

T (ai, θi)

03)‖2 .

Depending on the set J , there may be multiple parametervectors Θ that all lead to the same configuration of joint po-sitions xj(Θ) for all j ∈ J . As such, for general IK prob-lems of the form (4), the solution may not be unique sincethere can be multiple global optima. Most commonly, suchproblems are solved based on local optimisation methods,e.g. by modelling the hard constraints Θ ∈ I as penaltyin the objective function and then using a gradient descentprocedure for locally optimising the objective function. Amajor downside of using such iterative approaches is thatone requires a good initialisation for Θ, so that the optimi-sation does not result in an unwanted local optimum. Wewill tackle the problem of finding a good initialisation forProblem (4) based on a convex relaxation, as we will de-scribe next.

4. Convex Relaxation for Inverse KinematicsIn order to achieve a convex relaxation of the IK

problem, we will first redefine the problem as a non-convex quadratically constrained quadratic programme(QCQP) [1]. Subsequently, we will introduce our convexrelaxation based on semidefinite programming.

4.1. Inverse Kinematics as QCQP

Rather than phrasing the IK problem in terms of the pa-rameter vector Θ, we will directly optimise for rotation ma-trices and the global translation.

Global and relative rotations: Let Rj ∈ SO(3) ⊂R3×3 denote the global rotation of the j-th joint (i.e. rel-ative to the root), and let Rrel

j ∈ SO(3) ⊂ R3×3 denotethe rotation of the j-th joint relative to its immediate parent,where we use the notation π(j) ∈ [J ] to indicate the parentof joint j ∈ [J ]. For all joints we have the relation

Rj = Rπ(j)Rrelj , (5)

where we define Rπ(1) = I3. Forward kinematics: Theposition pj ∈ R3 of the j-th (non-root) joint is defined re-cursively as

pj = pπ(j) +Rπ(j)vj , (6)

where pπ(j) ∈ R3 is the 3D position of its parent, and wedefine pπ(1) := t.

Joint angle constraints: For the j-th joint the rotationrelative to its parent is constrained to be within the intervalIj , see Sec. 3.1. In our reformulation we impose a similarconstraint directly on the rotation matrix. In order to do so,we express Rrel

j in terms of a canonical rotation Rcanj , where

in our case we choose (w.l.o.g.) a rotation around the x-axis,so that we have the general structure

Rcanj =

1 0 00 cos(θj) − sin(θj)0 sin(θj) cos(θj)

=:

1 0 00 cj −sj0 sj cj

.(7)

As such, we can write Rrelj = STj R

canj Sj , for Sj ∈ SO(3)

being a suitably chosen matrix that is determined a priori(i.e. before optimisation). In order to impose the joint anglelimits we enforce that cj and sj lie within the unit circle,i.e. we impose the convex constraint

c2j + s2j ≤ 1 . (8)

In addition, for Ij = [θ−j , θ+j ], we consider the line passing

through (cos(θ−j ), sin(θ−j )) and (cos(θ+j ), sin(θ

+j )), and

enforce that the elements cj and sj ofRcanj are within a half-

space defined by this line. This results in a linear inequalityconstraint in sj and cj . We use Rcan

j ∈ Lj to refer to bothof these (convex) joint angle constraints.

Rotation constraints: The set of (proper) rotationsSO(3) can be defined with quadratic constraints as

SO(3) = {X ∈ R3×3 : XTX = I3, X:,1×X:,2 = X:,3} ,

where the cross-product is used to implement the right-handrule in order to ensure that the determinant is 1.

QCQP-IK: With the above elaborations we can now for-

mulate the IK problem as the QCQP

mint,{pj ,Rj ,Rrel

j ,Rcanj }

∑j∈J‖yj − pj‖2 (9)

s.t. pj = pπ(j) +Rπ(j)vj , pπ(1) = t ,

Rj = Rπ(j)Rrelj , Rπ(1) = I3 ,

Rrelj = STj R

canj Sj ,

Rcanj ∈ Lj ,

(Rj , Rrelj , R

canj ) ∈ SO(3)3 ,

where the constraints are applied for all j ∈ [J ].

4.2. Inverse Kinematics as SDP

Before we introduce our semidefinite programming re-laxation of the IK problem, we briefly summarise the mainidea of semidefinite programming relaxations for generalQCQPs.

4.2.1 Semidefinite Relaxations of Generic QCQPs

A generic QCQP can be written in canonical form as

minx∈Rn

xTA0x (10)

s.t. xTAix ≤ bi ,

where Ai ∈ Rn×n are given symmetric matrices (that arepossibly indefinite). Note that by using a homogeneouscoordinate representation, this form also allows for linearterms in the objective as well as for linear constraints. Com-monly, such non-convex QCQPs are solved by means oflifting, where an additional lifted variable Y of size n×n isintroduced. Based on the property that for a given matrixA it holds that xTAx = tr(xTAx) = tr(AxxT ), we canrewrite Problem (10) as

minx∈Rn,Y ∈Rn×n

tr(A0Y ) (11)

s.t. tr(AiY ) ≤ bi , xxT = Y .

It is well-known that the constraint xxT = Y is equivalentto [

1 xT

x Y

]� 0, rank(

[1 xT

x Y

]) = 1 .

Since the left part is a convex cone constraint, to obtain aconvex relaxation the rank constraint (that accounts for thenon-convexity) is dropped, which leads to the semidefiniteprogramming problem

minx∈Rn,Y ∈Rn×n

tr(A0Y ) (12)

s.t. tr(AiY ) ≤ bi[1 xT

x Y

]� 0 .

4.2.2 Semidefinite Relaxation for IK

In order to obtain our semidefinite programming relaxationfor the inverse kinematics problem, we systematically ap-ply the elaborations in Sec. 4.2.1 to Problem (9). In thefollowing we will elaborate on this. Matrix multiplicationconstraints: For three orthogonal matrices X,Y, Z of sizen × n, the matrix constraint X = Y Z can equivalently bewritten as [

In XXT In

]=

[YZT

] [YZT

]T. (13)

Similarly as above, by dropping rank constraints, we obtaina convex relaxation of this constraint as

(X,Y, Z) ∈M := {(X,Y, Z) :

In Y T ZY In XZT XT In

�0} .Rotation constraints: As indicated in Sec. 4.1, theSO(3) constraint can be represented with quadratic equal-ity constraints. A semidefinite relaxation of the constraintR ∈ SO(3) is achieved by working with the vectorisedR, i.e. vec(R) ∈ R9, and introducing a lifted variable Rof size 9×9, as done in Sec. 4.2.1. The interested readeris referred to [9, 31], where further details about the lift-ing and the constraints can be found. We use the notation(R,R) ∈ SO(3), where SO(3) is a convex set, to indi-cate that (R,R) is a pair of variables that satisfy the liftedrotation constraints.

Our convex relaxation: We now state the convex relax-ation of the inverse kinematics problem, which reads

mint,{pj ,Rj ,R

relj ,R

canj }

{Rj ,Rrelj ,R

canj }

∑j∈J‖yj − pj‖2 (14)

s.t. pj = pπ(j) +Rπ(j)vj , pπ(1) = t ,

(Rj , Rπ(j), Rrelj ) ∈M, Rπ(1) = I3 ,

Rrelj = STj R

canj Sj ,

Rcanj ∈ Lj ,

(Rj ,Rj) ∈ SO(3) .

(Rrelj ,R

relj ) ∈ SO(3) .

(Rcanj ,Rcan

j ) ∈ SO(3) .

Practical considerations: We solve our convex relax-ation using the general purpose modelling tool Yalmip [24]that interfaces the MOSEK solver [2]. In order to get atighter convex relaxation it helps to introduce convex re-laxations of redundant constraints, e.g. for orthogonalityone would use convex relaxations for both the constraintsXTX = I and XXT = I, see [9]. Once we have found asolution to Problem (14), we first project the obtained ma-trices onto the set SO(3) by means of Singular Value De-composition, and subsequently extract the joint angles Θ

directly from the projected matrices. The so-obtained jointangles Θ are not necessarily an optimum of the original IKobjective (4). Hence we use the found Θ as initialisationfor a trust region method as implemented in Manopt [8].We refer to our overall approach as SDP-IK.

5. ExperimentsIn this section we experimentally demonstrate the ben-

efits of our proposed approach based on two real-worldkinematic skeletons. To this end, in Sec. 5.1 we first com-pare different local optimisation methods. Subsequently, inSecs. 5.2 and 5.3, we compare the best-performing local op-timisation method to our convex relaxation approach, wherethe effectiveness of the proposed SDP-IK will become ap-parent. Since local IK methods are sensitive to initialisa-tion, for every pose we uniformly sample multiple randominitialisations (5 for the experiments in Sec. 5.1, and 20 forthe experiments in Sec. 5.2 and 5.3) within the joint limitconstraints, and then run local optimisation for each of thesampled initialisations. Note that we show the results ob-tained by all random initialisations in the comparisons. Wemeasure the quality of the results based on the square-rootof the normalised f in (4), which we denote as

cost(Θ) =

√1

|J |f(Θ) . (15)

In our experiments, we use two real-world skeletons (handand human body, see Fig. 2) in 100 different natural posessampled from captured motion data sequences [18, 29]. Themotion data is represented as a sequence of angle vectors Θthat animate the hand and body kinematic skeletons withpredefined bone vectors {vj}. We obtain the “observedjoint positions” for each pose by computing the forwardkinematics using the angle vector Θ. For the evaluationwe consider two different cases:

1. The observed joint positions are noise-free, i.e. thereexist kinematic parameters that yield an exact fit withan objective value of 0 (cf. Sec. 5.2).

2. The observed joint positions have added noise, i.e. theexistence of kinematic parameters for an exact fit is notguaranteed (cf. Sec. 5.3). This is the common case forpractical applications.

5.1. Local Optimisation Methods

To choose a representative local optimisation method asbaseline, we compare four methods on the task of fittingthe hand skeleton to observed joint locations when all jointsare observed (noise-free). For tackling the IK problem withlocal optimisation methods it is common practice to convertit to an unconstrained optimisation problem, where the joint

GD TR CG L-BFGS

0.0e0

5.0e3

1.0e4

Cost +

Penalty

GD TR CG L-BFGS0

10

20

30

Tim

e [s]

Figure 3. We show the performance of different local optimisationmethods: gradient descent (GD), trust region method (TR), conju-gate gradient method (CG), and LBFGS.

angle constraints are modelled as penalties. As such, weobtain the differentiable optimisation problem

minΘ∈R3+J

∑j∈J‖yj −xj(Θ)‖2+λ

3+J∑i=1

dist(θi, Ii)2 , (16)

where dist(θi, Ii) measures the distance of θi from the inter-val of plausible angles Ii (as done in [28]), and λ is the hy-perparameter that trades off joint limit violations with jointposition errors. For our experiments we fix λ = 100.

In Fig. 3 we show results of gradient descent (GD),the trust regions method (TR), conjugate gradient (CG),and limited memory Broyden-Fletcher-Goldfarb-Shanno(LBFGS) algorithms as implemented in the Manopt Mat-lab toolbox [8]. For all methods we used default parametersand allowed a maximum time budget of 30s. Each methodis tested with the same 8 poses with 5 random initialisa-tions, so that a total of 40 IK problems is solved. Whilethe first three methods yield solutions with similar quality,LBFGS performs significantly worse. Additionally, gradi-ent descent and conjugate gradient are significantly slowercompared to the other two methods. Hence we decide touse the trust region method (TR) as representative local IKmethod for the remaining experiments due to its good trade-off between speed and accuracy.

5.2. Fitting to Noise-Free Observations

In Fig. 4 we show quantitative results when solvingthe IK problem for noise-free observations. Our SDP-IKmethod almost always achieves an exact fit, i.e. the objec-tive function value reaches 0, and thereby consistently out-performs the local optimisation approach. This occurs bothwhen all joints are observed (All), and for the more ambigu-ous case where only the end-effectors and root are observed(End+Spine or End+Wrist), also cf. Fig. 2. Note that thelocal IK method suffers more from outliers due to bad ini-tialisations, while our method can consistently find betterminima.

5.3. Fitting to Noisy Observations

In many situations a given inverse kinematics problemcan potentially be infeasible, i.e. kinematic parameters that

SDP-IK Local

0.0

5.0

10.0

Co

st

Body: All

SDP-IK Local

0.0

5.0

10.0

15.0

Co

st

Body: End+Spine

SDP-IK Local

0.0

2.0

4.0

Co

st

Hand: All

SDP-IK Local

0.0

5.0

10.0C

ost

Hand: End+Wrist

Figure 4. Box plot for fitting to noise-free observed joint locationsof 100 poses for different skeletons (top: body, bottom: hand).Our SDP-IK method yields a consistently lower error in terms ofcost (15), both in the case where all joints are given (top), as wellas in the case when only the end-effectors are given (bottom).

exactly explain given observed joint locations may not ex-ist. In practice, this is the case when the joint position ob-servations are noisy. For example, neural networks havebeen employed successfully for joint location prediction forhands or human bodies. However, these networks mostlyoutput a set of independent joint locations without any con-straints to comply with the underlying kinematic structure.Hence, some approaches subsequently fit a kinematic skele-ton to the predictions to optimise for plausible joint angles,usually relying on local IK methods like gradient descent.The aim then is to find kinematic parameters Θ and hence akinematically consistent and plausible set of joint positionsthat achieve a minimum distance to the observed locations,as e.g. done in [26, 28].

In this experiment we mimic such a setting by fitting akinematic skeleton to joint locations that exhibit noise, sothat an exact fit may not be possible. To this end, we usethe same poses as for the noise-free fitting in Sec. 5.2, andadd noise to all observed joint locations. For each joint, the3D noise offset n = r · d ∈ R3 is obtained by uniformlysampling a unit length direction d ∈ R3, and a magnituder ∈ R from [0, rmax], where rmax is 10 mm for the handskeleton, and 100 mm for the body skeleton.

For infeasible cases, the minimum value of the IK costfunction in (15) fluctuates randomly with the sampled noise.The resulting box plot would merely capture how mucheach noisy sample violates the kinematic constraints insteadof the quality of the solution. Hence, instead of reportingthe raw value of the IK cost, we provide normalised costvalues. To this end, for each pose we subtract the valueof the smallest cost that is obtained by local optimisationamong all the 20 random initialisations.

SDP-IK Local

0.0

2.0

4.0

6.0

No

rma

lise

d C

ost

Body: All

SDP-IK Local

0.0

1.0

2.0

3.0

No

rma

lise

d C

ost

Body: End+Spine

SDP-IK Local

0.0

0.5

1.0

1.5

No

rma

lise

d C

ost

Hand: All

SDP-IK Local

0.0

2.0

4.0

6.0

No

rma

lise

d C

ost

Hand: End+Wrist

Figure 5. Box plot for fitting to noisy observed joint locations of100 poses for different skeletons (top: body, bottom: hand). OurSDP-IK method yields lower or similar error in terms of the nor-malised cost, both in the case where all joints are given (left), aswell as in the case when only the end-effectors are given (right).

In Fig. 5 we show that our method consistently achievessimilar, or better performance than a local optimisation forthe IK problem. Qualitative results for the local optimisa-tion method, the projected solution of the semidefinite pro-gramming approach in (14), and our SDP-IK are shown inFig. 6.

6. Discussion & Limitations

Currently, solving the semidefinite programming prob-lem (14) takes about 17s on average on an Intel i5 CPUwith 8GB of RAM, both for the hand as well as the hu-man body skeleton. We emphasise that our current imple-mentation is rather prototypical and is written so that it canwork with generic kinematic skeletons. Moreover, we didnot conduct any optimisations to improve the computationalefficiency—we expect that runtime improvements by oneorder of magnitude or even more could be achieved using aproblem-specific efficient implementation.

Currently, our model does not support translational DOFin non-root joints. In the future, we plan to also lookinto this case, as translational DOF can account for smallchanges in the bone length, which in turn can deal with thevariance of bone lengths in real-world data.

7. Conclusion

We have presented a convex optimisation approach forthe inverse kinematics problem based on a semidefinite pro-gramming relaxation. A major benefit of this approach isthat we can find the global optimum of a (relaxation of the)IK problem, in contrast to local optimisation methods thatheavily rely on good initialisations. Our experiments con-

Body Skeleton Hand Skeleton

Target

Local

SDP

Proj

ecte

dSD

P-IK

(Our

s)

Figure 6. Qualitative results for fitting kinematic skeletons to known noisy points of human body and hand skeletons in different poses(red spheres: noisy points, blue spheres: fitted points). Top row: observed noisy joint positions. Second row: inverse kinematics solvedvia local optimisation using a random initialisation (erroneous joints are circled). Third row: solution of the semidefinite programmingapproach in (14) after projection. Bottom row: Our proposed SDP-IK method.

firm this advantage and also demonstrate that our proposedSDP-IK approach is a useful method for tackling inversekinematics problems as they appear in computer vision andcomputer graphic problems.

Acknowledgements

This work was funded by the ERC Consolidator Grant4DRepLy. We thank Marc Habermann for making the hu-man body skeleton and motion sequences available to us.

References

[1] K. M. Anstreicher. Semidefinite programming versusthe reformulation-linearization technique for nonconvex

quadratically constrained quadratic programming. Journalof Global Optimization, 43(2-3):471–484, 2009. 4

[2] M. ApS. The MOSEK optimization toolbox for MATLABmanual. Version 9.0., 2019. 5

[3] A. Aristidou, Y. Chrysanthou, and J. Lasenby. Extendingfabrik with model constraints. Computer Animation and Vir-tual Worlds, 27:35–57, 02 2016. 2

[4] A. Aristidou and J. Lasenby. FABRIK: A fast, iterativesolver for the inverse kinematics problem. Graph. Models,73(5):243–260, Sept. 2011. 2

[5] A. S. Bandeira, M. Charikar, A. Singer, and A. Zhu.Multireference alignment using semidefinite programming.ITCS, 2014. 3

[6] C. Bensalah, J. Gonzalez-Quijano, N. Hendrich, and M. Ab-derrahim. Anthropomorphic robotics hand inverse kinemat-ics using estimated svd in an extended sdls approach. In

2013 16th International Conference on Advanced Robotics(ICAR), pages 1–7, Nov 2013. 2

[7] F. Bernard, C. Theobalt, and M. Moeller. DS*: TighterLifting-Free Convex Relaxations for Quadratic MatchingProblems. In CVPR, 2018. 2

[8] N. Boumal, B. Mishra, P.-A. Absil, and R. Sepulchre.Manopt, a Matlab toolbox for optimization on manifolds.Journal of Machine Learning Research, 15:1455–1459,2014. 6

[9] J. Briales and J. Gonzalez-Jimenez. Convex Global 3D Reg-istration with Lagrangian Duality. CVPR, 2017. 2, 5

[10] S. Buss. Introduction to inverse kinematics with jacobiantranspose, pseudoinverse and damped least squares methods.IEEE Transactions in Robotics and Automation, 17, 05 2004.2

[11] B. Bcsi, D. Nguyen-Tuong, L. Csat, B. Schlkopf, and J. Pe-ters. Learning inverse kinematics with structured prediction.In 2011 IEEE/RSJ International Conference on IntelligentRobots and Systems, pages 698–703, Sep. 2011. 2

[12] L. Carlone and G. C. Calafiore. Convex Relaxations for PoseGraph Optimization With Outliers. IEEE Robotics and Au-tomation Letters, 2018. 3

[13] Y. Chen, L. J. Guibas, and Q.-X. Huang. Near-Optimal JointObject Matching via Convex Relaxation. In InternationalConference on Machine Learning (ICML), 2014. 2

[14] A. Colom and C. Torras. Redundant inverse kinematics:Experimental comparative review and two enhancements.In 2012 IEEE/RSJ International Conference on IntelligentRobots and Systems, pages 5333–5340, Oct 2012. 2

[15] H. Dai, G. Izatt, and R. Tedrake. Global inverse kine-matics via mixed-integer convex optimization. In Inter-national Symposium on Robotics Research, Puerto Varas,Chile, pages 1–16, 2017. 2

[16] H. Dai, A. Majumdar, and R. Tedrake. Synthesis and op-timization of force closure grasps via sequential semidefi-nite programming. In Robotics Research, pages 285–305.Springer, 2018. 2

[17] A. P. Eriksson, C. Olsson, F. Kahl, and T.-J. Chin. RotationAveraging and Strong Duality. CVPR, 2018. 3

[18] M. Habermann, W. Xu, M. Zollhoefer, G. Pons-Moll, andC. Theobalt. Livecap: Real-time human performance capturefrom monocular video, 2019. 6

[19] C. Hecker, B. Raabe, R. W. Enslow, J. DeWeese, J. Maynard,and K. van Prooijen. Real-time motion retargeting to highlyvaried user-created morphologies. In ACM SIGGRAPH 2008Papers, SIGGRAPH ’08, pages 27:1–27:11, New York, NY,USA, 2008. ACM. 2

[20] B. Kenwright. Real-time character inverse kinematics usingthe gauss-seidel iterative approximation method. In 2012 4thInternational Conference on Creative Content Technologies(CONTENT 2012), pages 63–68, 2012. 2

[21] I. Kezurer, S. Z. Kovalsky, R. Basri, and Y. Lipman. TightRelaxation of Quadratic Matching. Comput. Graph. Forum,2015. 2

[22] Y. Khoo and A. Kapoor. Non-Iterative Rigid 2D/3D Point-Set Registration Using Semidefinite Programming. IEEETrans. Image Processing, 2016. 2

[23] Kwan Wu Chin, B. R. von Konsky, and A. Marriott. Closed-form and generalized inverse kinematics solutions for theanalysis of human motion. In Proceedings of the 19thAnnual International Conference of the IEEE Engineeringin Medicine and Biology Society. ’Magnificent Milestonesand Emerging Opportunities in Medical Engineering’ (Cat.No.97CH36136), volume 5, pages 1911–1914 vol.5, Oct1997. 2

[24] J. Lofberg. Yalmip: A toolbox for modeling and optimiza-tion in matlab. In IEEE international conference on roboticsand automation, pages 284–289, 2004. 5

[25] H. Maron, N. Dym, I. Kezurer, S. Kovalsky, and Y. Lip-man. Point registration via efficient convex relaxation. ACMTransactions on Graphics (TOG), 35(4):73, 2016. 2

[26] D. Mehta, S. Sridhar, O. Sotnychenko, H. Rhodin,M. Shafiei, H.-P. Seidel, W. Xu, D. Casas, and C. Theobalt.Vnect: Real-time 3d human pose estimation with a singlergb camera. volume 36, 2017. 2, 7

[27] D. Merrick and T. Dwyer. Skeletal animation for the ex-ploration of graphs. In Proceedings of the 2004 AustralasianSymposium on Information Visualisation - Volume 35, APVis’04, pages 61–70, Darlinghurst, Australia, Australia, 2004.Australian Computer Society, Inc. 2

[28] F. Mueller, F. Bernard, O. Sotnychenko, D. Mehta, S. Srid-har, D. Casas, and C. Theobalt. GANerated Hands for Real-Time 3D Hand Tracking from Monocular RGB. In Proceed-ings of Computer Vision and Pattern Recognition (CVPR),June 2018. 2, 6, 7

[29] F. Mueller, D. Mehta, O. Sotnychenko, S. Sridhar, D. Casas,and C. Theobalt. Real-time hand tracking under occlusionfrom an egocentric rgb-d sensor. In Proceedings of Interna-tional Conference on Computer Vision (ICCV), 2017. 6

[30] D. M. Rosen, C. DuHadway, and J. J. Leonard. A convexrelaxation for approximate global optimization in simultane-ous localization and mapping. ICRA, 2015. 3

[31] J. Saunderson, P. A. Parrilo, and A. S. Willsky. SemidefiniteDescriptions of the Convex Hull of Rotation Matrices. SIAMJournal on Optimization, 2015. 5

[32] C. Schellewald and C. Schnorr. Probabilistic subgraphmatching based on convex relaxation. In EMMCVPR, 2005.2

[33] S. Sridhar, H. Rhodin, H.-P. Seidel, A. Oulasvirta, andC. Theobalt. Real-time hand tracking using a sum ofanisotropic gaussians model. In Proceedings of the Inter-national Conference on 3D Vision (3DV), Dec. 2014. 2

[34] S. Starke, N. Hendrich, S. Magg, and J. Zhang. An effi-cient hybridization of genetic algorithms and particle swarmoptimization for inverse kinematics. In 2016 IEEE Inter-national Conference on Robotics and Biomimetics (ROBIO),pages 1782–1789, 2016. 2

[35] S. Starke, N. Hendrich, and J. Zhang. A memetic evolution-ary algorithm for real-time articulated kinematic motion. InIEEE Congress on Evolutionary Computation (CEC), pages2473–2479, 2017. 2

[36] A. Tkach, M. Pauly, and A. Tagliasacchi. Sphere-meshes forreal-time hand modeling and tracking. ACM Trans. Graph.,35(6):222:1–222:11, Nov. 2016. 2

[37] L. Wang and A. Singer. Exact and stable recovery of rota-tions for robust synchronization. Information and Inference,2013. 3

[38] L. . T. Wang and C. C. Chen. A combined optimizationmethod for solving the inverse kinematics problems of me-chanical manipulators. IEEE Transactions on Robotics andAutomation, 7(4):489–499, Aug 1991. 2

[39] P. Wang, C. Shen, and A. van den Hengel. A FastSemidefinite Approach to Solving Binary Quadratic Prob-lems. CVPR, 2013. 2

[40] Q. Zhao, S. E. Karisch, F. Rendl, and H. Wolkowicz.Semidefinite programming relaxations for the quadratic as-signment problem. Journal of Combinatorial Optimization,2(1):71–109, 1998. 2