topos toolkit for pure data -...
TRANSCRIPT
![Page 1: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/1.jpg)
!
T O P O S T O O L K I T F O R P U R E D ATA
L U I Z N A V E D A S TA T E U N I V E R S I T Y O F M I N A S G E R A I S ( B R A Z I L )
I C M C | S M C 2 0 1 4 - A T H E N S G R E E C E
I VA N I S A N TA N A F E D E R A L U N I V E R S I T Y O F B A H I A ( B R A Z I L )
exploring the spatial features of dance gestures for interactive musical applications
S E P T / 2 0 1 4
![Page 2: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/2.jpg)
C O N T E X T
< 1 0 0 U S D < 3 0 0 U S D > 5 0 0 0 U S D
M O T I O N C A P T U R E S Y S T E M S
M O T I O N C A P T U R E
D ATA
accelerometers infrared 3D image
infrared 3D marker
![Page 3: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/3.jpg)
P R O B L E M S
• How to extract realtime high-level information from motion capture data of music and dance performances?
• Imposed Cartesian coordinates ➜ meaningful feature descriptors for dance and music?
• Complex full body datasets ➜ simple and accessible motion descriptors
• “Ad-hoc” programming routines ➜ fast prototyping and mapping
![Page 4: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/4.jpg)
Becking curves1928
Truslit1938
Dahl2000
R E P R E S E N TAT I O N O F M U S I C A N D D A N C E G E S T U R E
![Page 5: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/5.jpg)
R E P R E S E N TAT I O N O F M U S I C A N D D A N C E G E S T U R E
Becking curves1928
Truslit1938
Dahl2000
![Page 6: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/6.jpg)
C O G N I T I O N O F S PA C E A N D G E S T U R E
Paillard (1991)
Morphokinetic: Deals with shape or form regardless of spatial location
Topokinetic: Deals with spatial location and spatial
intentionality regardless of shape
![Page 7: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/7.jpg)
C O G N I T I O N O F S PA C E A N D G E S T U R E
Paillard (1991)Morphokinetic: Deals with shape or form regardless of spatial location
Topokinetic: Deals with spatial location and spatial
intentionality regardless of shape
Leman & Naveda (2010)
Naveda & Leman (2010)
![Page 8: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/8.jpg)
Topokinetic: Deals with spatial location and spatial
intentionality regardless of shape
T G A T O P O L O G I C A L G E S T U R E A N A LY S I S
Naveda & Leman (2010)
![Page 9: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/9.jpg)
W T G A W I N D O W E D T G A A N A LY S I S
• Topologies are defined by previous and next movement trajectories
X Y
Z
+0.5 sec-0.5 sec
Intentionality Upcoming gesture
Short-Term Memory Preceding gesture
+0.5 sec-0.5 sec
Naveda & Santana (2010)
![Page 10: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/10.jpg)
T O P O S A L G O R I T H M
gemhead Second inlet: x y z floats
pack f f f
First inlet: gemlist
6
topos ID XYZ 100 10
Clock
new sample
Code:Projection onto a point cloud (ID)
2 1
...
3
5
6
4point cloud
topos.skeleton 1
topos.quality
topos.rec topos.rec.slave
topos.geo.2 ID xyz1 xyz2
topos.geo.4 ID xyz1 xyz2 xyz3 xyz4
topos.rel.points ID xyz1 xyz2 0.3
topos.rel.pointRegion ID xyz1 IDtopos 0.3
topos.features ID
topos ID xyz1 100 20
topos.rel.regions ID IDtopos1 IDtopos2 0.3
Transformation
Feature extraction
Description ofrelationships
Tools
xyztrajectories
[topos]Transformations
Topos-Featureextraction
Geom-Featureextraction
Relationshipfeatures
Symbolic link
3D data Sampling period
number of points
![Page 11: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/11.jpg)
“ T O P O S ” F E AT U R E S
![Page 12: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/12.jpg)
T O P O S L I B R A R Y
D ATA F L O W A N D M A N A G E M E N T V I S U A L I Z AT I O N F E AT U R E
D E S C R I P T I O N R E L AT I O N S H I P S
synapse
osceleton
skeleton
Point clouds
Points (2,4)
Point clouds
points and point clouds
points
![Page 13: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/13.jpg)
T O P O S L I B R A R YP L AY / R E C
V I S U A L I Z AT I O N
A N A LY S I S
F E AT U R E R E T R I E VA L
topos.skeleton 1
topos.quality
topos.rec topos.rec.slave
topos.geo.2 ID xyz1 xyz2
topos.geo.4 ID xyz1 xyz2 xyz3 xyz4
topos.rel.points ID xyz1 xyz2 0.3
topos.rel.pointRegion ID xyz1 IDtopos 0.3
topos.features ID
topos ID xyz1 100 20
topos.rel.regions ID IDtopos1 IDtopos2 0.3
Transformation
Feature extraction
Description ofrelationships
Tools
xyztrajectories
[topos]Transformations
Topos-Featureextraction
Geom-Featureextraction
Relationshipfeatures
• I N P U T S : K I N E C T ( O S C E L E T O N , S Y N A P S E ) , M O C A P T O O L B O X * O R R E C O R D E D D ATA
• 1 1 A B S T R A C T I O N S / O B J E C T S • E N T I R E LY W R I T T E N I N P U R E
D ATA
![Page 14: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/14.jpg)
S Y N T H E S I S
gemheadfreeze $1
route velocity distanceFrom
topos rhand1 1/r_hand/pos 30 50
topos.rel.pointRegion hands l_hand/pos rhand1 0.3
list-math / 3t b f
random 24
pack f f
+ 60
dac~
mtl/additive~
d(n)
d
n points
Distances from lhand to thepoint cloud of rhand are usedas magnitudes of oscilatorsin addirive synthesis
![Page 15: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/15.jpg)
C A S E : D A R M S TA D 5 8 ' ( 2 0 1 3 - 1 4 )
B R A Z I L
M E X I C O S PA I N
A U D I O , V I D E O , C O N T R O L
Receiving 2 skeletons Sending 1 skeleton
![Page 16: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/16.jpg)
C A S E : M I R A D A S D O C A O S I I ( 2 0 1 4 )
kinect blob detection (pix_freenect)
Topos Library
![Page 17: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/17.jpg)
F I N A L C O N S I D E R AT I O N S
• Contributions
• Minimal set of tools for high-level descriptors
• Provide access to a number of features and gestural relationships
• Problems
• Contextual features X strict synchronisation
• Dance traditions X expertise in controlling sound
![Page 18: TOPOS TOOLKIT FOR PURE DATA - luiznaveda.weebly.comluiznaveda.weebly.com/uploads/3/1/2/5/31253421/smc-icmc-2014we… · DATA FLOW AND MANAGEMENT VISUALIZATION FEATURE DESCRIPTION](https://reader035.vdocuments.site/reader035/viewer/2022071219/6055ab6e16a190675154c5c5/html5/thumbnails/18.jpg)
Library, tutorials and documentation www.naveda.info
Thanks!