commonsense physical reasoning: boxes and pitchers

Download Commonsense Physical Reasoning: Boxes and Pitchers

Post on 12-Jan-2016




0 download

Embed Size (px)


Commonsense Physical Reasoning: Boxes and Pitchers. Ernest Davis New York University June 6, 2007. Commonsense inferences. You can carry a collection of objects U at location A to location B by bringing an open box OB to A, loading the objects one by one into OB, and carrying OB to B. - PowerPoint PPT Presentation


  • Commonsense Physical Reasoning: Boxes and PitchersErnest DavisNew York UniversityJune 6, 2007

  • Commonsense inferencesYou can carry a collection of objects U at location A to location B by bringing an open box OB to A, loading the objects one by one into OB, and carrying OB to B.You can transfer liquid from container OS to container OD by pouring it: i.e. lifting OS over OD and tilting it

  • Part of Common Sense Reasoning EnterpriseRepresent knowledge of commonsense domains and automate commonsense reasoningPotential applications to natural language interpretation, automated planning, vision, expert systems, automated tutoring, etc.

  • Commonsense and ScienceHow experiment manipulation and perception at the human level relate to the underlying scientific theory.

  • Understanding variantsWhat would happen if:There were no test tubes?The test tubes were right side up?The test tubes were slanted?The test tubes were initially full of air?One of the electrodes was outside the tube?Both electrodes were in the same tube?

  • Variants Will fail if:The cargo objects dont fit in the boxThe box is tilted too farUnless the box has a lidThe box has a hole in the bottomUnless the hole is too small for the objectsSome other object falls into the boxSome agent lifts a cargo object out

  • More variantsYou can put one box inside another.You can make a pitcher overflow by dropping pebbles into it.You can ladle liquid out of a pail with a spoon.

  • Classic AI Representation(define (domain Box) (:action load (?o ?b ?l) :precondition (and (at ?o ?l) (at ?b ?l) (box ?b)) :effect (and (in ?o ?b) (not (at ?o ?l)))) (:action move (?b ?l1 ?l2) :precondition (at ?o ?l1) :effect (and (at ?o ?l2) (not (at ?o ?l1)))))

  • Scientific ComputingGiven precise boundary conditions --- object shapes, material properties, initial state, motion of manipulated objectsOutput precise prediction.Rigid solid objects: fine-grained time. Model all contact forces and impacts.Liquids: fine-grained time and space. Model forces on every piece of liquid.

  • Commonsense ReasoningAware of geometryDoes not require exact spatial informationInconvenient /impossible to perceive or measure.Information from inference or natural languageSystem under design / reasoning genericallySupports alternative directions of inferencefuture to past, behavior to shape, etc.Robust under variation of shape and motionContrast Newtonian analysis.

  • Logic-Based Analysis (Hayes)Develop Ontology (microworld)First-order languageFormal first-order theory (plus 2 default rules)Problem specificationModel qualitative reasoning as logical inference

  • ConstraintsQualitative reasoning: Shapes, motions, number of cargo objectsNo derivativesNo velocities or accelerationsNo forces or energy Reasoning over extended time and spaceKeep nonmonotonicity to a minimumClosed world assumptions in problem spec.

  • OutlineRelated workPre-formal analysis:Box microworldLiquids microworldRepresentationFuture work and Conclusion

  • Related WorkPat Hayes: Nave Physics Manifesto, Ontology for Liquids AI Solid Objects: Forbus, Faltings, Nielssen, Joskowicz, Shrobe & Davis, Gelsey.AI Liquids: Gardin & Meltzer, Collins, KimSensorless Robotics: Mason, Erdmann, LaValleSci. Computing, Graphics, Robotics

  • Pre-Formal AnalysisModelling assumptions:Objects are rigid. Liquids are incompressible.Motion is continuous. Large friction. Inelastic collisions. No viscosity, surface tension dry water.Objects in contact with liquids move slowly.Manipulation is motion of grasped object

  • Box/CupRegion R is boxed in state S iff the interior of R is connected;no solid object is in the interior of R;every boundary point of R below its top is a boundary point of some solid object

  • Boxed regions

  • Plan 1boxTransport(U:object[set], OB:box, L1, L2: location)sequence(while (some object O is not in OB) load(O into OB); carry(OB from L1 to L2) )

  • ConditionsIn s1, uCargo and oBox are stably placed on oTable1uCargo and oBox are inside a region isolated from other objects.uCargo fits inside oBox.The path from oTable1 to oTable2 is isolated from other objects.oBox is kept fairly upright while being carried.No other agent manipulates any of the objects.

  • What can go wrong?Some object is screwed into the table.Loading one object knocks another off the table.The configuration in which the objects fit in the box is unattainable. (Note: Cant assume that the objects stay where they were originally placed)The box falls over while being loaded.

  • What else can go wrong?Loading one object causes another to fly out of the box.

  • What else can go wrong?

    An object can come out of the box while it is being carried:Box is tilted too far.Rolling object in bowl sloshes out.Large accelerations push object out of box.

  • FixesEach cargo object is separately on the table, with a clear path to the box.Lots of excess room in box: |uCargo| H/2DW/2DL/2D Each object loaded to fairly low point.Vertical projection of inside of box inside convex hull of points of support on table.

  • Default rulesBy default, no object in a heap moves upward w.r.t. all the supports of the heap.If:O is supported by OS.Objects in set U are in heaps supported by O.U remains over convex hull of points of support of points of O by OSthen O does not fall over.

  • Liquid MicroworldKinematic constraints: Liquids and solids do not interpenetrate, continuity, rigidity, incompressibility.

  • 3 scenariosCupped region: If throughout a history a cupped region has greater volume than the liquid inside, then there is no outflow.If the volume of a cupped region falls below the liquid inside, then it overflows on top.Liquid flows down when it can, flowing around obstacles.

  • OverflowDuring overflow, the liquid rises a finite distance over the spout and goes out a distance beyond the spout. Falling down rule is suspended.

  • OntologyReal-valued timeEuclidean space. Geometric entities as needed.Objects. Sets of objects.Chunks of liquidsAgents.StatesHistoriesFluents (Boolean and non-Boolean)Pseudo-objects (inside of a box; center of mass)Plans

  • Open Box openBoxShape(RB,RI:region)

    openBoxShape(RB,RI) bounded(RB) ^ bounded(RI) ^ ec(RB,RI) ^connected(interior(RI)) ^[P Pboundary(RI) ^ height(P) < top(RI) Pboundary(RB)]

  • Cupped regionholds(S, cuppedRegion(R)) holds(S, openBoxShape#(solidSpace,R)).

  • Liquid in cupped regiondynamic(H) ^ slowObjectsInContact(Q,H) ^ continuous(Q,H) ^ throughout(H,singleCup#(Q) ^# #fullOfLiquid#(Q)) noOutflow(Q,H).

    (Note use of region-valued fluent Q: greatinsight of Hayes.)

  • Default against flying out of boxupwardMotion(O,US,H) OSUS,QC [source(QC)=OS ^ coordSys(QC)^ sometime(H,zAxis*(QC) =# z)] value(end(H),zCoor*(cMass(O),QC)) > value(start(H),zCoor*(cMass(O),QC)).

  • Default (cntd)anomUp(O,H) UH,US OUH ^ holds(start(H),heap(UH,US)) ^ throughout(H,isolated(UH,US,)) ^ upwardMotion(O,US,H).

    noAnomUp(H) O,H1 subHistory(H1,H) anomUp(O,H1).

    dynamic(H) : noAnomUp(H) / noAnomUp(H).

  • Feasibility of manipulation (simplified)kinematic(H) ^ holds(start(H),grasps(A,O)) H1,H2 prefix(H1,H) ^ duration(H1) 0 ^sameMotionOn(H2,H1,{O}) ^ start(H2)=start(H) ^ throughout(H2,grasping(A,O)) ^ dynamic(H2).

  • Box plan specificationPlan1 =seq(while(unloaded , seq(loadBox(a1,unloaded, qinBox,manip1), waitUntil(stable(uCargo {oBox, oTab1}))) carryBox(a1,oBox,qIn,uCargo, oTab2,manip2))

  • Variants I can handleBoxes inside boxesBoxes composed of multiple partsBoxes with holes (milk crate)Boxes with lidsLadling liquid out of a bowlRaising water level by adding pebbles

  • Close variants I cant (yet) handleUnloading a boxCarrying a heap of objects on a trayPouring liquid onto a horizontal surface.

  • Future workMore physical domainsProbabilistic inference (e.g. stability of a heap)Tractable rule systemEstablish soundness w.r.t. a plausible modelWhat qualitative inferences are important?

  • David Waltz (1995):It was widely believed that logic could successfully model images and scenes, even though the baroque improbability of that effort should have long been clear to everyone who read Pat Hayes Nave Physics Manifesto.

  • ConclusionKnowledge-based analysis using a logical language allows us to characterize a broader range of qualitative physical reasoning than any other known technique.