Robotics

11/02/2020

QUT Verified Signature

P F

{wfp = (xfp, yfp)T | f = 1, ..., F, p = 1, ..., P} .

W =

⎡⎢⎢⎣w11 . . . w1P

wF1 . . . wFP

⎤⎥⎥⎦ .

W 2F × P

M S

W = MS .

W

2 × 3 Rf

2F × 3 P

WM

S

W

Rf 2× 3

M

S 2F × 3 3 × P

W

W = UDV T ,

D U V T

U

3× 3 D V T

W ′ = U ′D′V ′T U ′D′V ′T

‖W −W ′‖2F s.t. rank(W ′) = 3 .

M = U ′√D′ S =√D′V ′T

S

Rf

G

MS = M(GG−1)S = (MG)(G−1S) = MS .

G MG

iTf GGT if = 1

jTf GGT jf = 1

iTf GGT jf = 0 .

Q = GGT

Q

MQ Q−1S

P F

{wfp = (xfp, yfp)T | f = 1, ..., F, p = 1, ..., P}

W =

⎡⎢⎢⎣w11 . . . w1P

wF1 . . . wFP

⎤⎥⎥⎦ .

W 2F × P

M S W = MS

W

2F × 3F

R1...Rf 2× 3

3F × P

P

WM

S

W 10P × 10s × 30fps × 2(x, y) = 6, 000

3 × 10

900 × 10

K

B1...BK

cf = [cf1, cf2, ..., cfK ]

Sf =

K∑d=1

cfdBd ,

Sf 3×P f Bd 3×P

K cfd

B1, ..., BK c1, ..., cK

Wf = Rf (

K∑d=1

cfdBd) .

K

W = M(C ⊗ I3)B = ΠB ,

I3 3×3 B Π = M(C⊗ I3)

W Π

R ∈ 2×3

WΠ

R ∈ 2×3

W K

K

K

Π ∈ 2F×3K B ∈ 3K×P

3K × 3K Q

Q Π

Ri

Q

Mf = [cf1Rf ...cfKRf ] Π

Q

MS = MQQ−1S G = QQT

Qk Q

k Mk = MQk Mk, k = 1, ...,K

M Mk

kth cfk

M M = MQ

MfGkMTf = cfkcfkRfR

Tf ,

Gk = QkQTk 3K × 3K

Qk Mf

ith

MfGkMTf = c2fkI2 ,

I2 2×2 Gk

cfk

M2f−1GkMT2f−1 − M2fGkM

T2f = 0

M2f−1GkMT2f = 0 .

Gk

QHkQT Hk

Gk

W = ΠGG−1B Π = M(C ⊗ I3)

Π

Qk ∈ 3K×3K Qk = GkGTk Π2f−1:2fQkΠ

T2f−1:2f = c2fkI2

cfk Qk

Π2f−1QkΠT2f−1 = Π2fQkΠ

T2f

Π2f−1QkΠT2f = 0 .

Qk

2K2−K

A

W

vec(.) qk = vec(Qk)

vec(AXBT ) = (B ⊗A)vec(X)

[Π2f−1 ⊗ Π2f−1 − Π2f ⊗ Π2f

Π2f−1 ⊗ Π2f

]qk = Afqk = 0 ,

Π2f−1, Π2f 2f − 1th 2fth Π

(f = 1, ..., F )

Aqk = 0 ,

A = [AT1 , A

T2 , ..., A

TF ]

T qk (3K)(3K+1)/2

2F ≥ (3K)(3K +

1)/2 qk

2K2 −K

A

Qk

2K2 −K

A

{Avec(Qk) = 0} ∩ {Qk � 0} ∩ {rank(Qk) = 3} .

Qk

trace(Qk) s.t.

Qk � 0 ,

Avec(Qk) = 0 .

Qk Gk ∈ 3K×3

Gk

K

K

K(LK

)L

2×3

S F × 3P S

S K K

B S = C B

C ∈ F×L K Π = M(C ⊗I3) Π

2× 3 M 2× 3

W L

K

L

3× 2

B,Π‖W T −BΠ‖2F s.t. ‖Πi‖0,3 ≤ K, i = 1 : N/2 ,

‖Πi‖0,3 3 × 2 Π

B

Π

S

2 × 3

3K × 3K

3× 3 3L× 3L

G

Π B

G

643

G(V, E)V

E

α

α

R3

R3

(s, t, u) = o + s + t + u ,

o ,

0<s<1, 0<t<1 0<u<1 i,j,k

i,j,k = o +i

l+

j

m+

k

n,

l m n

l+1,m+1 n+1 , ,

(s, t, u) =

l∑i=0

m∑j=0

n∑k=0

Bi,l(s)Bj,m(t)Bk,n(u) i,j,k,

Bθ,n(x)

n

i,j,k i, j, k

n

Bθ,n(x) =

(n

θ

)xθ(1− x)n−θ,

(nθ

)i,j,k

= ,

∈ RN×3 ∈ R

N×M

∈ RM×3 N

M

t s

u

l,m, n=3( , , ) i,j,k

l,m, n=3

S TS

T

S( S , S) T ( T , T )

ST S ′( ′, S)

Δ

1

2‖ − ( A ⊗ 3)( +ΦΔ )‖22 +

γ

2‖ΦΔ ‖22.

∈ R3P

vec([ T ]A) [ T ]A ∈ R3×P

P A A⊗

∈ R3M vec( �) M

Δ ∈ R3M

Φ ∈ R3M×3M

x−Φ

L2

γ L2

Δ

x−, y−z− Φ

vec(·)

sdist

sdist =1

| ′|∑∈ ′

f( ,ST ; θ) +1

| T |∑∈ T

f( ,S ′; θ),

f( ,S; θ) =

⎧⎨⎩1 dist( ,S) > θ

0

θ

sIoU =V ′ ∩ VTV ′ ∪ VT

,

V ′ VT

G sdist < θdist sIoU > θIoU

θdist θIoU

G

Ω Sc( c, c)

i ∈ Ω

S( , )

= αc c +∑i∈Ω

αiic, = c,

α

G∈ R

2×P P

S

L

∈ R2×3 ∈ R

2×1

s

Δ , ,s,

1

2‖ L −

(( L ⊗ s )( +ΦΔ ) +

)‖22

+γ

2‖ΦΔ ‖22, � = 2.

L

LΔ

Φ

L2 γ

Δ , , ,

1

2‖ L −

(( L ⊗ )( +ΦΔ ) +

)‖22

+γ

2‖ΦΔ ‖22, � = s2 2, = ,

s

Lρ( , ,Δ , ,Λ) =1

2‖ L −

(( L ⊗ )( +ΦΔ ) +

)‖22

+γ

2‖ΦΔ ‖22 + 〈Λ, − 〉F +

ρ

2‖ − ‖2F ,

Λ ρ

〈., .〉F

k = Lρ( , k−1,Δ k−1, k−1,Λk−1),

� = s2 2;

k = Lρ(k, ,Δ k−1, k−1,Λk−1);

Δ k =Δ

Lρ(k, k,Δ , k−1,Λk−1);

k = Lρ(k, k,Δ k, ,Λk−1);

Λk = Λk−1 + ρ( k − k).

Lρ( , k−1,Δ k−1, k−1,Λk−1) =[(σ1 + σ2)/2

(σ1 + σ2)/2

]�,

− Λρ = [ σ1

σ2 ]� σ

− Λρ

Lρ(k, ,Δ k−1, k−1,Λk−1) =

(( L − ) � +Λ+ ρ

)( � + ρ )+,

= unvec( +ΦΔ ) L ∈ R3×|L| |L|

ΔLρ(

k, k,Δ , k−1,Λk−1) =

(Φ�( L ⊗ )�( L ⊗ )Φ+ γΦ

)+(Φ�( L ⊗ )�

(L − ( L ⊗ ) +

)).

Lρ(k, k,Δ k, ,Λk−1) =

∑l∈L( l − l)

|L| .

G

S α Sc( c, c)

Ω

S( , )

= αc c+∑

i∈Ω αiic c

ic

unvec(·)

Sc Sic i

i α

s, ,α

1

2

∑l∈L

‖ l −(s (αc[ c]l +

∑i∈Ω

αi[ic]l) +

)‖22

+ μ

N∑l=1

C(s (αc[ c]l +

∑i∈Ω

αi[ic]l) +

)+

γ

2

∑i∈Ω

α2i ,

l l l = 1, . . . , P

N Cμ γ

L2

∇αi

αi

∇αi =∑l∈L

(l −

(s (αc[ c]l +

∑i∈Ω

αi[ic]l) +

))�s [ i

c]l

+ μN∑l=1

∇C�s [ ic]l + γαi,

∇C

e[ξ]×

[·]× ∇ξ

ξ

∇ξ =∑l∈L

(l −

(s (αc[ c]l +

∑i∈Ω

αi[ic]l) +

))�

(s

∂[ξ]×ξj

(αc[ c]l +∑i∈Ω

αi[ic]l)

)

+ μ

N∑l=1

∇C�(s

∂[ξ]×ξj

(αc[ c]l +∑i∈Ω

αi[ic]l)

).

∇

∇ =∑l∈L

(l −

(s (αc[ c]l +

∑i∈Ω

αi[ic]p) +

))+ μ

N∑l=1

∇C.

s α

eRP

epose e3D

l,m, n = 3 4×4×4

sdist

sIoU sdist < θdist

sIoU > θIoU θdist θIoU

S Tsdist T

S ′ 1e−4 θdist = 1e−3

S ′

S ′

TS ′

T S ′

VT V ′ 0.1694 θIoU = 0.25

θdist θIoU

1283

eRP epose

e3D

eRP epose e3D eRP epose e3D eRP epose e3D eRP epose e3D

eRP epose e3D

S T T VT

S′ S′ V ′

ST

VT TS ′ V ′

S ′

T sdist S ′

sIoU

FFD

FFD

c Δ αG cG

Δα

c Δ

α G

G

Ω Sc( c, c)

i ∈ Ω

S( , )

= αc c +∑i∈Ω

αiic, = c,

α

G

= Φ( +Δ ),

∈ RN×3 ∈ R

N×M

∈ RM×3 N M

Δ

Φ ∈ R3M×3M

Δ

α

c Δ α

Δ α

Δ

α

{8, 16, 32} {5, 3, 3} {3, 3, 3}

{16, 8, 1} {3, 3, 5} {3, 3, 3}220× 220

2202 → 722 → 242 → 82 → 242 → 722 → 2202

∈ R2,048

8 × 3 × 3

Δ α

f : → {Δ ,α}

κ ∈ RMN M N

α

κ ∈ R126

(32×3) α

α

Gα α

256 × 192

70 30

sdist sdist

dist3D =1

| |∑∈

dist( ,S) + 1

| |∑∈

dist( , S),

S S

IoU =V ∩ VV ∪ V

,

V V

MSE ∼ t Acc( ) Prec( ) Rec( ) ∼ t MSE ∼ t dist3D IoU

Acc Prec Rec t

GG

Δα

G

G

α

α

43

Δ ∈ R32×3

α ∈ R30

κ ∈ R126

c κ

c

Δ

= Φ( + Δ )

α

= αc c +∑

i∈Ω αiic

dist3D

dist3D

IDX

FFD LC

IDXGT

dist3D IoU dist3D IoU

IDX FFD LC

IDX + LCGT

IDX + FFDGT + LCGT IDX + FFD + LCGT IDX

1e−3 1e−5

dist3D IoU dist3D IoU dist3D IoU dist3D IoU

IDX FFD LCFFD LC

dist3D IoU dist3D IoU dist3D IoU dist3D IoU

IDXFFD LC IDX

IDXGT

dist3D IoU dist3D IoU dist3D IoU dist3D IoU dist3D IoU

2.62 −3 . −2.16 −5 . −3.59 −5 . −

. −2.58 −5 . −

1.34 −3 . −

IDX FFDLC

FFDLC IDX

FFD LCGT

FFD LCGT

LC FFDGT

1e−3

Δ α

Δ TΔ

γ

c ∈ Rnv×3

∈ Znf×3, 0 ≤ Fij < nv i = [p, q, r]

p q r c = { , }c

c

λc

A B

λc(A,B) =∑a∈A b∈B

‖a− b‖2 +∑b∈B

a∈A‖b− a‖2.

λem

λem =φ:A→B

∑a∈A

‖a− φ(a)‖,

φ

IoU

(A,B) =|A ∩B||A ∪B| .

,

l,m n l+1,m+1, n+1 , ,

(s, t, u) =l∑

i=0

m∑j=0

n∑k=0

Bil(s)Bjm(t)Bkn(u) ijk,

B·N (x)

N

ijk i, j, k

= ,

∈ RM×3 ∈ R

N×3 M

N ∈ RN×M

Δ (t)

T c(t) 0 ≤ t < T

(t) = (t) (t)

Δ˜ (qt)

˜(qt) = (t)(

(t) +Δ˜ (qt)).

γ(qt)

λ0 =∑q, t

f(γ(qt)

)λc

(s(q), s(qt)

),

f

γ(qt) = (1− εγ) γ(qt)0 +

1

Tεγ ,

γ(qt)0

t εγ 0 < εγ � 1

t∗ =t

γ(qt),

c(q) = {˜(qt∗), (t∗)}.

Δ˜

f

f(γ) = γ)

f

f(γ) = − (1− γ)

λe =∑t

γ(t)(γ(t)

),

γ(t) t

λ0

λ′e = λ0 + κeλe.

κe = e−b/b0κe0,

κe0 b b0

λr =∑q,t

γ(qt)|Δ˜ (qt)|2,

| · |2

λ′r = λ0 + κrλr,

κr κr0

εe

α = 0.25

192 × 192

1 × 1

512

4 l = m = n = 3

16, 384

1, 024

192×256×3

30◦ 45◦

192× 256× 36× 8× 256

1× 1 6× 8× 643, 072512

(t) 192 + 1

3 × 43 = 192 Δ˜ (qt)

γ(qt)

10−3 β1 = 0.9 β2 = 0.999 ε = 10−8

100, 000 b0 = 10, 000

εe = 0.02

323

εγ f(γ) κe0 κr0γ

− (1− γ)γγ

z ≥ 0 3.2

323

5

13

1000× (λc λem 1− )

λc λem

5

13

λc λem

λIoU = 0.33

5

13

1000 × λc

Template0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

Norm

alizedfrequency

b

w

e

r

Template0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

Norm

alizedfrequency

b

w

e

r

10−1 100

X

0.0

0.2

0.4

0.6

0.8

1.0

λc<X

b

w

e

r

10−2 10−1 100

X

0.0

0.2

0.4

0.6

0.8

1.0

λc<X

b

w

e

r

10−1

100

X

0.0

0.2

0.4

0.6

0.8

1.0

λc<X

T=1

T=2

T=4

T=8

T=16

T=30

0.0 0.2 0.4 0.6 0.8 1.0

X

0.0

0.2

0.4

0.6

0.8

1.0

IoU>X

T=1

T=2

T=4

T=8

T=16

T=30

T > 1T = 1

T = 1

T ∈ {2, 4, 8, 16}N

±

Γ(t)

φ(x, t) : Ω×R �→ R

φ

Γ(t) = {x : φ(x, t) = 0} ,

φ(x, t)

Γ

φ

Γ

φ

∂Γ

∂t= vn,

Γ v

n

Γ

∂Γ

∂t= −∂E(Γ)

∂Γ= vn.

φ

Γ

∀tφ(Γ(t), t) = 0.

∂φ(Γ(t), t)

∂t= 0,

∂φ(Γ(t), t)

∂t=

∂φ

∂Γ

∂Γ

∂t+

∂φ

∂t= ∇φ

∂Γ

∂t+

∂φ

∂t,

∇φ = ∂φ∂Γ

∇φ‖∇φ‖

∂φ

∂t= −∇(φ)

∂Γ

∂t= −∇(φ)vn = −∇(φ)v(−1)

∇φ

‖∇φ‖ = v‖∇φ‖.

Γ v

φ v‖∇φ‖

φ

∂φ

∂t= v‖∇φ‖.

φ v

Γ

Γ S ⊂R3

xi ∈ Sni xi Γ S

ni

ni ∼ ±ni S m

X = {xi}mi=1 N = {ni}mi=1 dX (x)

X

d(x,X ) =y∈X

‖x− y‖2.

EX (Γ) =

(∫Γd(s,X )pds

)1/p

, 1 ≤ p ≤ ∞,

Γ ds

Lp Γ

XN

Γ

SLp Γ

N

EN (Γ) =

(∫Γ(1− |N(s) · nΓ(s)|)p ds

)1/p

, 1 ≤ p ≤ ∞,

N(s) = ni xi s

Γ

nΓ(s) =∇φ(s)

‖∇φ(s)‖ ,

EN (Γ)

EN (Γ) =

(∫Γ

(1−

∣∣∣∣N(s) · ∇φ(s)

‖∇φ(s)‖

∣∣∣∣)p

ds

)1/p

.

Γ

Earea =

∫Γds,

Evol =

∫Γds,

Γ ds

Γ

f(x) = ±y∈Γ

‖x− y‖,

f(x) > 0 x Γ

|∇f | = 1

Γ

Esdf (φ) =

∫(‖∇φ(x)‖ − 1)2dx.

n Ij

Sj = {X j ,N j}θ

I φ(I; θ)

θ

L(θ) =∑j∈D

EX j (Γ(Ij ; θ)) + α1

∑j∈D

EN j (Γ(Ij ; θ))

+ α2

∑j∈D

Esdf (φ(Ij ; θ)) + α3

∑j∈D

Earea(Γ(Ij ; θ))

+ α4

∑j∈D

Evol(Γ(Ij ; θ)).

Γ φ

I Γ(I; θ) = {x : φ(I; θ) = 0} D = {1, ..., n} α1 − α4

δ H

∑j∈D

EX j (Γ(Ij ; θ)) =

=∑j∈D

(∫R3

δ(φ(x, Ij ; θ))d(x,X j)pdx

)1/p

,

∑j∈D

EN j (Γ(Ij ; θ)) =∑j∈D

(∫R3

δ(φ(x, Ij ; θ))

(1−

∣∣∣N j(x) · ∇φ(x; Ij , θ)

‖∇φ(x, Ij ; θ)‖

∣∣∣)pdx

)1/p

,

∑j∈D

Esdf (φ(Ij ; θ)) =

∑j∈D

∫R3

(‖∇φ(x, Ij ; θ)‖ − 1)2dx,

∑j∈D

Earea(Γ(θ, Ij)) =

∑j∈D

∫R3

δ(φ(x, Ij ; θ)) dx,

∑j∈D

Evol(Γ(θ, Ij)) =

∑j∈D

∫R3

H(φ(x, Ij ; θ)) dx.

Ω

C1 C2

δ H

δε(x) =

{12ε

(1 + (πxε )

), |x| ≤ ε,

0, |x| > ε,

Hε(x) =

⎧⎪⎨⎪⎩

12

(1 + x

ε +1π (πxε )

), |x| ≤ ε,

1, x > ε,

0, x > −ε,

H ′ε(x) = δε(x)

Lε φ

φj(x) = φ(x, Ij ; θ)) dj(x)p = d(x,X j)p

Lε(θ) =∑j∈D

(∑x∈Ω

δε(φj(x))dj(x)p

)1/p

+ α1

∑j∈D

(∑x∈Ω

δε(φj(x))

(1−

∣∣∣N j(x) · ∇φj(x)

‖∇φj(x)‖

∣∣∣)p)1/p

+ α2

∑j∈D

∑x∈Ω

(‖∇φj(x)‖ − 1)2 + α3

∑j∈D

∑x∈Ω

δε(φj(x))

+ α4

∑j∈D

∑x∈Ω

Hε(φj(x)).

Lε φ Ω

∂Lε

∂φ=

∑j∈D

1

p

(∑x∈Ω

δε(φj(x))dj(x)p

) 1−ppδ′ε(φ

j(x))dj(x)p

+α1

p

∑j∈D

(∑x∈Ω

δε(φj(x))

(1−

∣∣∣N j(x) · ∇φj(x)

‖∇φj(x)‖

∣∣∣)p) 1−p

p

(δ′ε(φ

j(x))(1−

∣∣∣N j(x) · ∇φj(x)

‖∇φj(x)‖

∣∣∣)p+

δε(φj(x))

∂

∂φ

(1−

∣∣∣N j(x) · ∇φj(x)

‖∇φj(x)‖

∣∣∣)p)

+ α2

∑j∈D

∑x∈Ω

(‖∇φj(x)‖ − 1) ∇ ·(

∇φj(x)

||∇φj(x)||

)

+ α3

∑j∈D

δ′ε(φj(x)) + α4

∑j∈D

δε(φj(x)).

Lε∂Lε

∂φφ

Γ φ

500 2000

20

80/20

p = 2 ε = 0.15 α1 = 0.8 α2 = 1

α3 = α4 = 0.1

E(p) = − 1

N

N∑n=1

[pn pn + (1− pn) (1− pn)],

p N

p

10−6

A B

(A,B) =|A ∩B||A ∪B| .

0 1

P1 P2

d (P1,P2) =1

|P1|∑x∈P1

y∈P2

||x− y||+ 1

|P2|∑y∈P2

x∈P1

||y − x||.

203 303

φ Δ φ Δ

203 303

Δ

203 303

φ Δ φ Δ

203

303 Δ

Δ

O(NM) N M

203 303

2563

203 203 303 303

203 203

303 303

203 203 303 303

203 203

303 303

203 203 303 303

203 203

303 303

203 203 303 303

203 203

303 303

203 203 303 303

203 203

303 303