step: a scripting language for embodied agents

1

STEP: a Scripting LanguageSTEP: a Scripting Languagefor Embodied Agentsfor Embodied Agents

Zhisheng Huang, Anton Eliens,and Cees Visser

연세대학교 컴퓨터과학과로봇 공학 특강

2004 2 학기

10410898 유 지 오

2

AgendaAgenda• Introduction• Principles• The Scripting Language STEP

– Reference Systems– Primitive Actions and Composite Operators– Dynamic Logic– High-Level Interaction Operators

• Examples– Walk– Run– Look at ball– Touch

• Related Work• Conclusions

3

IntroductionIntroduction• Interaction multi-modal communicative acts

– Verbal– Non-verbal

Gestures, postures, facial expressions

– Specifying communicative acts is not easy

• Scripting language STEP (Scripting Technology for Embodied Persona)– Focus on specification and modeling of gestures and

postures for 3D web agents– Specification of communicative acts can be separated from

the programs Advantage!!– For H-anim-based humanoids in DLP (based on WWW)– Prolog-like syntax

4

PrinciplesPrinciples• Design principles For External-oriented communicative acts

• Principle 1: Convenience– Hide geometric difficulties– The authors who have limited knowledge of C.G. can use it in

a natural way– Example

Turn a left arm forward slowlyturn(Agent, left_arm, front, slow)

More specifty…Turn(Agent, left_arm,rotation(1,0,0,1.57), 3)

– In STEPUses some natural-language-like terms for references

5

Principle cont’dPrinciple cont’d• Principle 2: Compositional Semantics

– Specification of composite actions based on existing components

– Sequence action seq, parallel action par, repeat action repeat

– Examples Turn arms forward slowly

par([turn(Agent, left_arm, front, slow), turn(Agent, right_arm, front, slow)])

– In STEP A set of built-in action operators

• Principle 3: Redefinability– Scripting actions can be defined in terms of other action explicitly

– Example Run then kick

script(run_then_kick(Agent), Action) :- Action = seq([run(Agent), kick(Agent)])

– In STEP Rule-based specification system

Principles

6

Principle cont’d 2Principle cont’d 2• Principle 4: Parameterization

– Adapted to be other actions– Can be specified in terms of how they cause changes over

time to each individual degree of freedom– Example

When define the action “run” with respect to a degree of freedom “tempo”

– In STEPProlog-like syntax

• Principle 5: Interaction– Should be able to interact with the world, including objects

and other agents– In STEP

Powerful meta-language

Principles

7

Reference SystemsReference Systems• Direction reference

– Natural-language-like direction reference scheme

The Scripting Language STEP

Direction reference for humanoid Combination of the directions for left arm

x

y

z

8

Reference Systems cont’dReference Systems cont’d• Body reference

– According to the H-anim standard– All body joints are contained in a hierarchical structure

The Scripting Language STEP

Typical joints for humanoid

Elbow joint in different situations

9

Reference Systems cont’d 2Reference Systems cont’d 2• Time reference

– Same time reference system as VRML– Flexible time reference system based on..

Beat interval for body movementTempo number of beats per minute

The Scripting Language STEP

10

Primitive Actions and Composite OperatorsPrimitive Actions and Composite Operators

• turn(Agent, BodyPart, Direction, Duration)– Direction: natural-language-like direction or rotation value– Duration: speed name or explicit time specification

• move(Agent, BodyPart, Direction, Duration)– Direction: natural-language-like direction or increment value

• turn_body(Agent, Direction, Duration)

• move_body(Agent, Direction, Duration)

• Sequence operator: seq([Action1, …, Actionn])

• Parallel operator: par([Action1, …, Actionn])

• Non-deterministic choice operator: choice([Action1, …, Actionn])

• Repeat operator: repeat(Action, T)

The Scripting Language STEP

11

STEP and Dynamic LogicSTEP and Dynamic Logic• STEP is based on dynamic logic

– Allows for arbitrary abstractions using the primitive and composition operators

• A state the properties at a paticular moment• An action a relation between two states• Two sub-languages in dynamic logic

– Sub-language for actions: a– Sub-language for states: Ψ, Φ

The Scripting Language STEP

][a

12

High-Level Interaction OperatorsHigh-Level Interaction Operators• Scripting actions can directly interact with

– Internal states of embodied agents– External states of worlds

• Lower case greek letters Φ, Ψ, Χ denote formulas in the meta language– test(Φ) : if Φ holds then skip, otherwise fail– do(Φ) : make the state Φ true– If_then_else(Φ,action1,action2)– until(action, Φ) : perform action until Φ holds

• All temporal relation can be defined in STEP – 13 possible temporal relations between two actions

Before, meets, overlaps, starts, during, finishes, equals and their inverse relations

– Exampleoverlap(A1,A2)=par([A1,seq([duration(A1,T1),do(random(R)),

do(N is T1*R),wait(N),A2])])

The Scripting Language STEP

13

WalkWalk

Examples

script(walk_pose(Agent), Action):- Action=seq( [par( [ turn(Agent,r_shoulder,back_down2,fast), turn(Agent,r_hip,front_down2,fast), turn(Agent,l_shoulder,front_down2,fast), turn(Agent,l_hip,back_down2,fast)] ), par( [turn(Agent,l_shoulder,back_down2,fast), turn(Agent,l_hip,front_down2,fast), turn(Agent,r_shoulder,front_down2,fast), turn(Agent,r_hip,back_down2,fast)] )] ).

script(walk_forward_step(Agent), Action):- Action=par( [walk_pose(Agent), move(Agent,front,fast)] ).

script(walk_forward_step07(Agent), Action):- Action=par( [walk_pose(Agent), move(Agent,increment(0.0,0.0,0.7),fast)] ).

script(walk_forward_step0(Agent,StepLength), Action):- Action=par( [walk_pose(Agent), move(Agent,increment(0.0,0.0, StepLength),fast)] ).

script(walk_forward(Agent,StepLength,N), Action):- Action=repeat(walk_forward_step0(Agent,StepLength),N).

The walk action

Walking posture + moving action

Walking forward N steps

14

RunRun• Based on the action “walk”

Examples

script(basic_run_pose(Agent), Action):- Action=par( [turn(Agent,r_elbow,front,fast), turn(Agent,l_elbow,front,fast), turn(Agent,l_hip,front_down2,fast), turn(Agent,r_hip,front_down2,fast), turn(Agent,l_knee,back_down,fast), turn(Agent,r_knee,back_down,fast)] ). Start pose (b)

script(run_pose(Agent,N), Action):- Action=seq([basic_run_pose(Agent), repeat(walk_pose(Agent),N)] ).

Run action (a)

script(basic_run_pose_elbow(Agent,Elbow_Angle), Action):- Action=par( [turn(Agent,r_elbow,rotation(1,0,0,Elbow_Angle),fast), turn(Agent,l_elbow,rotation(1,0,0,Elbow_Angle),fast), turn(Agent,l_hip,front_down2,fast), turn(Agent,r_hip,front_down2,fast), turn(Agent,l_knee,back_down,fast), turn(Agent,r_knee,back_down,fast)] ).

script(run(Agent,StepLength,N), Action):- Action=seq([basic_run_pose(Agent), walk_forward(Agent,StepLength,N)] ).

Run action with specifying Steps

15

Look at BallLook at Ball• Interaction with Virtual Worlds• Calculate angle

Examples

||||cos

21

21

vv

vv

script(turn_to_direction(Object,ScrVector,DestVector), Action):- Action=seq( do(vector_cross_product(SrcVector,DestVector,vector(X,Y,Z),R)), do(setRotation(Object,X,Y,Z,R))] ).

script(look_at_position(Agent,X1,_Y1,Z1), Action):- Action=seq( [do(getPosition(Agent,X,_Y,Z)), do(Xdif is X1-X), do(Zdif is Z1-Z), turn_to_direction(Agent,vector(0.0,0.0,1.0), vector(Xdif,0.0,Zdif))] ).

script(look_at_ball(Agent,Ball), Action):- Action=seq( [do(getPosition(Ball,X1,Y1,Z1)), look_at_position(Agent,X1,Y1,Z1)] ).

16

TouchTouch• An inverse kinematics problem

Examples

)2

(222

uf

dfucos arc

)

2(

222

ud

fducos arc

script(getABvalue(Agent,position(X0,Y0,Z0),Hand,A,B), Action):- Action=seq( [ getDvalue(Agent,position(X0,Y0,Z0),Hand,D), get_upperarm_length(Agent,L1), get_forearm_length(Agent,L2), do(D1 is L1+L2), if_then_else(sign(D1-D)>sign(0.001-D), seq( [do(cosine_law(L1,L2,D,A)), do(cosine_law(L1,D,L2,B))] ), seq( [do(A is 1.57*(1+sign(D-0.001))), do(B is 0.0)]))]).α=π and β=0

script(getVvalue(Agent,position(X0,Y0,Z0),Hand,V), Action):- Action=seq( [ get_shoulder_center(Agent,Hand,position(X2,Y2,Z2)), do(direction_vector(position(X2,Y2,Z2),position(X0,Y0,Z0),V))] ).

17

Touch cont’dTouch cont’d

Examples

script(touch(Agent,position(X0,Y0,Z0),l), Action):- Action=seq( [ getABvalue(Agent,position(X0,Y0,Z0),l,A,B), do(R1 is 3.14-A), getVvalue(Agent,position(X0,Y0,Z0),l,D), get_arm_vector(Agent,l,V0), do(vector_cross_product(V0,V,vector(X3,Y3,Z3),c)), do(R2 is C-B), par( [turn (Agent,l_shoulder,rotation(X3,Y3,Z3,R2),fast), turn (Agent,l_elbow,rotation(X3,Y3,Z3,R2),fast), turn (Agent,l_wrist,rotation(X3,Y3,Z3,-0.5),fast)])]).

script(touch_absolutePosition(Agent, postion(X1,Y1,Z1),Hand), Action):- Action=seq( [do(getPosition(Agent,X,Y,Z)), do(getRotation(Agent,X2,Y2,Z2,R)), do(X3 is X1-X), do(Y3 is Y1-Y), do(Z3 is Z1-Z), do(R1 is –R), do(position_rotation(position(X3,Y3,Z3), rotation(X2,Y2,Z2,R1), position(X4,Y4,Z4))), touch(Agent,position(X4,Y4,Z4), Hand)]).

18

Related WorkRelated Work• Perlin and Goldberg’s Improv system

– A system for scripting interactive actors in virtual worlds– Compare with STEP…

STEP is based on the H-anim specificationSTEP is design to separate the scripting language from the

agent architecture

• Badler et al ‘s PAR– Similar to

Supports the specification of complex actions for agents

– Different..PAR uses a special syntax and has many built-in notions

19

Related Work con’dRelated Work con’d• Funge’s CML (Cognitive Modeling Language)

– CML is not VRML/X3D and H-anim based– CML is based on first-order predicate logic

vs. STEP is based on dynamic logic

• Prendinger’s MPML, SCREAM– Higher level concepts such as affect and social context

20

ConclusionsConclusions• Proposed the scripting language STEP for embodied agents,

in particular for their communicative acts, like gestures and postures

• Can be extended for other communicative acts like facial expressions or speech