planning, optimisation and classification of 3d

Planning, optimisation and classification of 3D trajectories for robotic steerable needles in keyhole neurosurgery with a deductive reasoning approach28th April 2021

Candidate: Valentina Corbetta, 919294

Supervisor: Prof. Elena De Momi

Co-supervisor: Prof. Francesco Calimeri, Dott. Ing. Alice Segato

Politecnico di Milano

School of Industrial and Information Engineering

Department of Electronics, Information and Bioengineering (DEIB)

Master of Science in Biomedical Engineering

A.Y. 2019-2020

Presentation outline

Introduction

State of the Art

Statement of Purpose

Materials

Path planning

Optimisation and classification

Results

Conclusions and Future Work

2

Valentina Corbetta

Valentina Corbetta

Keyhole neurosurgery

3

Robotic tools Keyhole neurosurgery

Allows access to the brain througha tiny hole in the skull

Advantages:● Lower infection rates● Lower complications● Overall better post-operative

outcomes

Introduction

4

Steerable needlesIntroduction Valentina Corbetta

Rigid needles

Straight trajectories

4

Valentina Corbetta

Steerable needlesIntroduction

Rigid needles

Steerable needles

Curvilinear trajectories

Straight trajectories

4

Valentina Corbetta

Rigid needles

Steerable needles

Curvilinear trajectories

Advantages:● Active control of needle

trajectory● Compensate for target

movement● Access to deep structures

Drawbacks:● Require complex path planning

Solutions:● development of an automatic

path planner

Straight trajectoriesSteerable needlesIntroduction

5

Path planning problemState of the Art Valentina Corbetta

Uobst

Ufree

qI

qG

q1

path P(p0, p1, …, pn-1)

possible actions

Definition of a collision-free path in Ufree from an Entry Point (EP) to a Target Point (TP).

In neurosurgery:

● Clearance from safety regions: corticospinal tracts, vessels, ventricles, thalamus

● Respect of kinematic constraints of the needle:○ outer diameter (OD)○ maximum curvature (Kmax)

6

Path planning methodsState of the Art Valentina Corbetta

Graph-based

● Only minimise length● Do not take into account

expert’s knowledge● Do not take into account

the kinematic constraints

Leibrandt et al., Likhachev et al.

6

Valentina Corbetta

Path planning methodsState of the Art

Graph-based




Sampling-based


the kinematic constraints● Do not take into account

expert’s knowledge

Leibrandt et al., Likhachev et al. Segato et al., Patil et al.

6

Valentina Corbetta


Graph-based




Sampling-based




Learning-based

● Require large datasets for training

● Computationally intensive

Segato et al., Patil et al.Leibrandt et al., Likhachev et al. Tan et al., Chi et al.

6

Valentina Corbetta

Graph-based




Sampling-based




Learning-based

● Require large datasets for training

● Computationally intensive

Reasoning-based

● Explicitly represent domain knowledge

● Take into account kinematic constraints

● Take into account expert’s knowledge

● Take into account many parameters for optimisation


Segato et al., Patil et al.Leibrandt et al., Likhachev et al. Tan et al., Chi et al.

7

Statement of Purpose Valentina Corbetta

The aim of this thesis is to try and solve the path planning problem for steerable needles in neurosurgery with a reasoning-based approach. The end goal is to develop a tool that can assist the neurosurgeon in the pre-operative phase, taking into account the kinematic constraints of the needle and leveraging the clinician’s expertise, to find the optimal path to the target structure.

8

Materials Valentina Corbetta

● Declarative programming paradigm born in the field of logic programming and non-monotonic reasoning

● A problem is modeled by a collection of rules; the solution of the encoded model is called answer set

● Rules are in the form

a0|...|ah :- b1,...,bn not bn+1,...,bm

atoms

head body

● A fact is a rule with a single element in the head and no body; it represents a certainly true information

● A constraint is a rule with empty head; they can be hard (symbol :- and must be satisfied) or weak (symbol :~ and should be satisfied)

Answer Set Programming (ASP)

8

Materials Valentina Corbetta

r1: color(X,red) | color(X,blue) | color(X, lightblue):- node(X).r2: :- arc(X,Y), color(X,C), color(Y,C).

Why ASP

9

Path planning Valentina Corbetta

Path planning with ASP

Environment

Dataset

Resize

Voxel Grid

EP and TS

selection

Path planner

arrival_at(T):-needle_at(T,X,Y,Z),finish(X,Y,Z).

arrival :- arrival_at(T).

Shortest Path

2

1

3

4

5

Set of rules and encoded problem in ASP

Segato, A., Corbetta, V., Calimeri, F., & De Momi, E. (2020, October). Inductiveand Deductive Reasoning for Robotic Steerable Needle in Neurosurgery.

In 2020 IEEE International Conference on Intelligent Robots and Systems.

10

Optimisation and classification Valentina Corbetta

System architecture

Surgeon

Dataset

Kinematic

constraints

Parameters and

risk weights

EPs and TS

Manual

trajectories

System

Dataset 3D

visualisation

Hard constraints

Soft constraints

Best trajectory

Search space

definition

ASP trajectory

optimisation

Parameters

extraction

ASP trajectory

classification

12

3

4

5

6

7

8

9

10

10

Valentina Corbetta

Surgeon

Kinematic

constraints

Parameters and

risk weights

EPs and TS

Manual

trajectories

System

Hard constraints

Soft constraints

Search space

definition

ASP trajectory

optimisation

Parameters

extraction

ASP trajectory

classification

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

Optimisation and classificationSystem architecture

10

Valentina Corbetta

Surgeon

EPs and TS

Manual

trajectories

System

Search space

definition

ASP trajectory

optimisation

Parameters

extraction

ASP trajectory

classification

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

Kinematic

constraints

Parameters and

risk weights

Hard constraints

Soft constraints

3

4


10

Valentina Corbetta

Surgeon

EPs and TS

Manual

trajectories

System

Search space

definition

ASP trajectory

optimisation

Parameters

extraction

ASP trajectory

classification

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

Kinematic

constraints

Parameters and

risk weights

Hard constraints

Soft constraints

3

4

Outer diameter

0.0014 mm-1

2.5 mm

Kinematic constraints

Rule weight CED

6

9

6

Wd_min

Wd_tot

Wc_max


Maximum curvature

10

Valentina Corbetta

Surgeon

Kinematic

constraints

Parameters and

risk weights

System

Hard constraints

Soft constraints

ASP trajectory

optimisation

Parameters

extraction

ASP trajectory

classification

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

5

6

Search space

definition

7

EPs and TS

Manual

trajectories

5

6


10

Valentina Corbetta

Surgeon

Kinematic

constraints

Parameters and

risk weights

System

Hard constraints

Soft constraints

ASP trajectory

optimisation

Parameters

extraction

ASP trajectory

classification

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

EPs and TS

Manual

trajectories

5

6

Search space

definition

7


Valentina Corbetta

Surgeon

Kinematic

constraints

Parameters and

risk weights

System

Hard constraints

Soft constraints

ASP trajectory

optimisation

Parameters

extraction

ASP trajectory

classification

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

EPs and TS

Manual

trajectories

5

6

Search space

definition

7● Set of concentric circles

with 0 ≤ r ≤ 5 mm

● tj points sampled on each

circle

● Range of radius and n.° of

circles specified by user


10

10

Valentina Corbetta

Surgeon

Kinematic

constraints

Parameters and

risk weights

System

Hard constraints

Soft constraints Parameters

extraction

ASP trajectory

classification

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

EPs and TS

Manual

trajectories

5

6

Search space

definition

7

ASP trajectory

optimisation

8


10

Valentina Corbetta

Surgeon

Kinematic

constraints

Parameters and

risk weights

System

Hard constraints

Soft constraints Parameters

extraction

ASP trajectory

classification

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

EPs and TS

Manual

trajectories

5

6

Search space

definition

7

ASP trajectory

optimisation

8

● Search space encoded as

ASP problem

Example:

node(P1).arc(X,Y).


10

Valentina Corbetta

Surgeon

Kinematic

constraints

Parameters and

risk weights

System

Hard constraints

Soft constraints

ASP trajectory

optimisation

ASP trajectory

classification

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

EPs and TS

Manual

trajectories

5

6

Search space

definition

7

Parameters

extraction

9


9

10

Valentina Corbetta

Surgeon

Kinematic

constraints

Parameters and

risk weights

System

Hard constraints

Soft constraints

ASP trajectory

optimisation

ASP trajectory

classification

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

EPs and TS

Manual

trajectories

5

6

Search space

definition

7

Parameters

extraction

9

Minimum distance from obstacles

Average distance from obstacles

Total length of trajectory

Maximum curvature

99


10

Valentina Corbetta

Surgeon

Kinematic

constraints

Parameters and

risk weights

System

Hard constraints

Soft constraints

ASP trajectory

optimisation

Parameters

extraction

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

EPs and TS

Manual

trajectories

5

6

Search space

definition

7

ASP trajectory

classification

10


10

10

Valentina Corbetta

Surgeon

Kinematic

constraints

Parameters and

risk weights

System

Hard constraints

Soft constraints

ASP trajectory

optimisation

Parameters

extraction

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

EPs and TS

Manual

trajectories

5

6

Search space

definition

7

ASP trajectory

classification

10

Hard constraints

● cmax must be lower than Kmax

● dmin must be higher than r = OD/2

Example:

:- choose(X), radius(r), distObst(X,dmin), dmin<r.

10


10

Valentina Corbetta

Surgeon

Kinematic

constraints

Parameters and

risk weights

System

Hard constraints

Soft constraints

ASP trajectory

optimisation

Parameters

extraction

3

4

5

6

7

8

9

10

DatasetDataset 3D

visualisation

12

12

EPs and TS

Manual

trajectories

5

6

Search space

definition

7

ASP trajectory

classification

10

Soft constraints

● Minimisation of dtot

● Maximisation of dmin

● Minimisation of cmax

Example:

#maximize{dmin@wd_min,X:

choose(X), disObst(X,dmin)}.

10


10

Valentina Corbetta

Surgeon

Dataset

Kinematic

constraints

Parameters and

risk weights

EPs and TS

Manual

trajectories

System

Dataset 3D

visualisation

Hard constraints

Soft constraints

Search space

definition

ASP trajectory

optimisation

Parameters

extraction

ASP trajectory

classification

12

3

4

5

6

7

8

9

10


10

11

Valentina Corbetta

Experimental setup

● Modalities: manual vs

deductive reasoning

● 5 experiments for each

modality, 10 trajectories for

each experiment

Input:

● EP, TS

● OD, Kmax

● wd_min, wd_tot, wc_max

EXP1 EXPK...

Manual EXPK

ManualTK...ManualT1

Manual ...

Manual EXP1

T1 ... Tj

Visual

ASP EXPK

ASPTK...ASPT1

ASP ...

ASP EXP1

T1 ... Tj

Classification

Optimisation and classification

Manual trajectories designed

by expert neurosurgeon from

the Oncology and Emato-

oncology department of

Università degli Studi di Milano

12

Results Valentina Corbetta

Segato, A., Corbetta, V., Zangari, J., Calimeri, F., & De Momi, E. (2021, June). Optimized 3D path planner for steerable catheters with deductive reasoning. In 2020 IEEE International Conference on Robotics and Automation.

p-value

dtot 0.043

cmax 0.002

dmin 4.3551e-05

davg 0.0031

statistically significant,

p<0.05

13

Conclusions and future work Valentina Corbetta

The proposed methodology implements a tool that can assist the neurosurgeon in the pre-operative phase for path-planning of a steerable needle

Obstacle clearanceCustomisable

preferences

Respect of

kinematic constraints

Explicit representation

of domain knowledge

Future work

● Application to Deep Brain Stimulation

● Application to other fields other than neurosurgery

● Integration with other path planning methods

Thanks for your attention

Questions?

Valentina Corbetta

Results: ASP vs A*● Comparison between A*

and ASP

● 10 experiments with a

couple of randomly

selected EP and TS, within a

maximum distance of 10

voxels

Input: EP, TS

EXP1 EXPK...

ASP

ASP

EXPK

...

ASP EXP1

A* EXPk

A* ...

A* EXP1

ASPTK...ASPT1A*TK...A*T1

Average n.° of steps

Computational time [s]

ASP 7.4 2043.01

A* 7.4 0.002

Path planning

15

16

Valentina Corbetta

Integration of Unity and ASPOptimisation and classification

C#

Simulator

Parameters extraction

d_min d_tot c_maxd_avg

Write Clingo files

Invoke Python script

Python

Define search space

Invoke Clyngor

Clyngor

Run Clingo

Output solutions

planning, optimisation and classification of 3d

Documents