steps toward developing an intelligent robotics course kutztown university
DESCRIPTION
Steps Toward Developing an Intelligent Robotics Course Kutztown University. PACISE 2011 April 9, 2011 Oskars J. Rieksts Jeffrey W. Minton. Acknowledgments – Funding. CS Department & LAS College TYCO Electronics Foundation Kutztown University Foundation - PowerPoint PPT PresentationTRANSCRIPT
Steps Toward Developing an Steps Toward Developing an Intelligent Robotics CourseIntelligent Robotics Course
Kutztown UniversityKutztown University
PACISE 2011PACISE 2011April 9, 2011April 9, 2011
Oskars J. RiekstsOskars J. Rieksts
Jeffrey W. MintonJeffrey W. Minton
2011 Kutztown University 2
Acknowledgments – FundingAcknowledgments – FundingCS Department & LAS CollegeCS Department & LAS CollegeTYCO Electronics FoundationTYCO Electronics FoundationKutztown University FoundationKutztown University FoundationKU Faculty Professional KU Faculty Professional
Development CommitteeDevelopment CommitteeKU Undergraduate Research KU Undergraduate Research
CommitteeCommitteePASSHE Faculty Professional PASSHE Faculty Professional
Development CouncilDevelopment Council
2011 Kutztown University 3
MotivationsMotivationsRodney BrooksRodney BrooksDARPA Grand ChallengeDARPA Grand ChallengeREU @ AuburnREU @ AuburnHardware-software synthesisHardware-software synthesis
““Hardened” hardwareHardened” hardwareCentral role of softwareCentral role of software
2011 Kutztown University 4
MotivationsMotivationsApplied A.I.Applied A.I.Applied cognitionApplied cognitionMobilityMobilityAssistive roboticsAssistive robotics
Robotic wheelchairsRobotic wheelchairs
2011 Kutztown University 5
OutlineOutlineEarly effortsEarly efforts
Hacking Roomba & other venturesHacking Roomba & other venturesResearch Experience for Research Experience for
UndergraduatesUndergraduatesCreate and MindstormsCreate and MindstormsCaptain KURKCaptain KURKTrinity firefighting contestTrinity firefighting contestMyro and PythonMyro and PythonReading-Berks Science FairReading-Berks Science Fair
2011 Kutztown University 6
Coming into FocusComing into FocusEmerging goalsEmerging goals
IntelligenceIntelligenceVisionVisionCommuncationCommuncationPhilosophical issuesPhilosophical issues
2011 Kutztown University 7
Point of convergencePoint of convergenceCourse in intelligent roboticsCourse in intelligent robotics
ControlControlVisionVisionCommuncationCommuncationInvestigate cognitive issuesInvestigate cognitive issues
2011 Kutztown University 8
Course design objectivesCourse design objectivesRobot isRobot is
SituatedSituatedEmbodiedEmbodied
Shared environmentShared environmentHumanHumanMachineMachine
Shared communicationShared communicationFramework for investigationFramework for investigation
2011 Kutztown University 9
Robot is situatedRobot is situated
Operates within an environmentOperates within an environmentEmbedded in the worldEmbedded in the worldChief knowledge source:Chief knowledge source:Data stream drawn from Data stream drawn from
environmentenvironment““The world is its own best The world is its own best
representation” – Brooksrepresentation” – Brooks
2011 Kutztown University 10
Robot is embodiedRobot is embodiedEntity 1Entity 1stst, agent 2nd, agent 2nd
Self reliantSelf reliantSecretary of State modelSecretary of State model
Self-seated conceptual frameworkSelf-seated conceptual frameworkWorld concepts grounded in sensor suiteWorld concepts grounded in sensor suiteBehavior set emanates from actuator Behavior set emanates from actuator
suitesuiteDecision apparatus grounded in Decision apparatus grounded in
sensor/actuator suitessensor/actuator suites
2011 Kutztown University 11
Robot is embodiedRobot is embodiedEames: Eames: Design is a plan for arranging Design is a plan for arranging
elements to accomplish a particular elements to accomplish a particular purposepurpose
Elephants don’t play chess (Brooks)Elephants don’t play chess (Brooks)““Mind” is to fit the bodyMind” is to fit the body
Design on need to basisDesign on need to basisKnowKnowThinkThinkDoDo
2011 Kutztown University 12
Shared environmentShared environmentShared conceptual Shared conceptual frameworkframework
Robot’s Robot’s Derivative of human’sDerivative of human’sSimplified in structureSimplified in structure
Shared factbaseShared factbaseRobot’s Robot’s
Subset of human’sSubset of human’sSimplified ontologySimplified ontology
Vision basedVision based
2011 Kutztown University 13
Common communication frameworkCommon communication frameworkGrounded inGrounded in
SharedSharedEnvironmentEnvironmentConceptual frameworkConceptual frameworkOntologyOntology
Vision as main sensorVision as main sensor
Intersecting language constructsIntersecting language constructs
2011 Kutztown University 14
Projected course topics/activitiesProjected course topics/activitiesSpecialized robot control softwareSpecialized robot control softwareRobot control architectureRobot control architectureBasics of image processingBasics of image processingSpecialized image processing Specialized image processing
softwaresoftwareBasics of communication theoryBasics of communication theoryKey issues of cognitive roboticsKey issues of cognitive robotics
2011 Kutztown University 15
Slow, steady progressSlow, steady progress
Browning:Browning:Reach should exceed graspReach should exceed grasp
Rapid prototypingRapid prototypingIterative developmentIterative development
2011 Kutztown University 16
Structure of the classStructure of the classFour teamsFour teamsRolesRoles
Team leaderTeam leaderDesignerDesignerCoderCoderDocument guruDocument guruHistorianHistorianTest designerTest designerTest administratorTest administratorHardware specialistHardware specialist
2011 Kutztown University 17
PACT demonstrationPACT demonstrationPennsylvania Association of Pennsylvania Association of
Council of TrusteesCouncil of TrusteesVery early in learning curveVery early in learning curveNavigation within a “corral”Navigation within a “corral”Well receivedWell received
Gateway navigationGateway navigationBull fighting robotBull fighting robot
PR for robotics @ PASSHEPR for robotics @ PASSHE
2011 Kutztown University 18
Jeff – gateway navigationJeff – gateway navigation
2011 Kutztown University 19
Investigate cognitive issuesInvestigate cognitive issuesSight and touchSight and touch
George StrattonGeorge StrattonSpatial harmony of sight & touchSpatial harmony of sight & touch
SpaceSpaceBenjamin KuipersBenjamin KuipersSemantic spatial hierarchySemantic spatial hierarchy
MetricalMetricalTopologicalTopologicalHybridHybrid
2011 Kutztown University 20
Investigate cognitive issuesInvestigate cognitive issuesLanguage and meaningLanguage and meaning
Stevan HarnadStevan HarnadSymbol grounding problemSymbol grounding problem
Language and shared spaceLanguage and shared spaceAmichai KronfeldAmichai KronfeldShared referent problemShared referent problem
2011Kutztown University
21
Investigate cognitive issuesInvestigate cognitive issuesEmbodied cognitionEmbodied cognition
Randall BeerRandall BeerSituated, embodied, dynamicalSituated, embodied, dynamical
Principle of interactivismPrinciple of interactivismMark BickhardMark BickhardEmergence of representational Emergence of representational
contentcontent
2011 Kutztown University 22
Illustration – Shared Referent ProblemIllustration – Shared Referent ProblemFrom KronfeldFrom Kronfeld
Steve & wife Bev are at a partySteve & wife Bev are at a partySteve: Bob & his wife are certainly Steve: Bob & his wife are certainly
enjoying the partyenjoying the partyBev: Sally is not his wife!Bev: Sally is not his wife!
NoteNoteSteve & Bev share a referent, SallySteve & Bev share a referent, SallyDespite misidentification no Despite misidentification no
communication problems arise!communication problems arise!
2011 Kutztown University 23
The KLO TriadThe KLO TriadFor communication involving For communication involving
referentsreferentsFor each communicant – this triadFor each communicant – this triad
Object (the referent)Object (the referent)KB – representation of objectKB – representation of objectLanguage – reference to objectLanguage – reference to object
For successful communicationFor successful communicationReferents must matchReferents must matchFor HMC (Human-machine For HMC (Human-machine
communication)communication)Human & machine KBs do Human & machine KBs do notnot match match
2011 Kutztown University 24
Establishing co-refrenceEstablishing co-refrencePer Frege’s distinctionPer Frege’s distinctionExtensional approachExtensional approach
Pointing or equivalentPointing or equivalentIntensional approachIntensional approach
Language aloneLanguage aloneHMC goalHMC goal
Minimize extensionalMinimize extensionalMaximize intensionalMaximize intensional
2011 Kutztown University 25
The Trinity RobotThe Trinity Robot
2011 Kutztown University 26
The Trinity RobotThe Trinity RobotVEX robotics kit for chasisVEX robotics kit for chasisVEX included motors for locomotionVEX included motors for locomotionSonar rangefindersSonar rangefindersWeb-camWeb-camArduino to control motors and sensorsArduino to control motors and sensorsNetbook to control Arduino and Netbook to control Arduino and
process imagesprocess images
2011 Kutztown University 27
The Trinity RobotThe Trinity RobotInaccurate motor controlInaccurate motor controlSonar signals bounce inside Sonar signals bounce inside
cornerscornersProvide inaccurate measurementsProvide inaccurate measurements
Viewing angle of web-cam too Viewing angle of web-cam too smallsmall
2011 Kutztown University 28
BrobiBrobi
2011 Kutztown University 29
Brobi – hardwareBrobi – hardwareCreate by iRobotCreate by iRobotPlatform built onto Create cargo Platform built onto Create cargo
bay to accommodate equipmentbay to accommodate equipmentWeb-cam with increased viewing Web-cam with increased viewing
angleangleIR rangefinderIR rangefinder
IR light does not bounce like soundIR light does not bounce like soundArduinoArduino
2011 Kutztown University 30
Brobi – softwareBrobi – softwareOpenCV and PythonOpenCV and PythonConsultation – John SpletzerConsultation – John Spletzer
LehighLehighLittle Ben in Urban ChallengeLittle Ben in Urban Challenge
MATLABMATLABImage processingImage processingCreate APICreate APIA programming languageA programming language
2011 Kutztown University 31
Image processingImage processing““A picture is worth a thousand A picture is worth a thousand
words”words” need to extract discrete objects need to extract discrete objects
from images to identify themfrom images to identify themK-Means clusteringK-Means clustering
2011 Kutztown University 32
Goal: identify green ballGoal: identify green ball
2011 Kutztown University 33
K-Means ClusteringK-Means ClusteringCluster sets of dataCluster sets of data
Into user-defined number of Into user-defined number of segmentssegments
Number of segments referred to as KNumber of segments referred to as KClusters defined by MEAN of all Clusters defined by MEAN of all
values in clustervalues in cluster
K-Means AlgorithmK-Means Algorithm
K-Means ExampleK-Means Example
K-Means using k = 6
K-Means using k = 5
2011 Kutztown University 36
Natural Language ProcessingNatural Language Processing
““Go to the green ball”Go to the green ball”The meanings behind words must be The meanings behind words must be
inferredinferred““Go,” conceptually can represent Go,” conceptually can represent
many thingsmany thingsTake a turn in a gameTake a turn in a gameThe Chinese strategy gameThe Chinese strategy gameTravel to a locationTravel to a location
Determine the concept being referred Determine the concept being referred toto
Conceptual parsingConceptual parsing
2011 Kutztown University 37
Conceptual ParsingConceptual Parsing
Words are mapped to conceptsWords are mapped to concepts
ConceptsConceptsRules define set of related conceptsRules define set of related conceptsOne concept may have many One concept may have many
separate rule setsseparate rule sets
Concept TreeConcept Tree
2011 Kutztown University 39
Assessment – PlatformAssessment – PlatformBest hardware platform to dateBest hardware platform to dateOpens up many avenues of course Opens up many avenues of course
developmentdevelopmentSoftware for robot control systemsSoftware for robot control systemsRobot control architecturesRobot control architecturesImage processingImage processingCommunicationCommunicationCognitive robotics issuesCognitive robotics issues
2011 Kutztown University 40
Assessment – MATLABAssessment – MATLABCan do all 3 thingsCan do all 3 things
Sensor/actuator interface with Sensor/actuator interface with Lehigh APILehigh API
Image acquisition & processingImage acquisition & processingRobot control system programmingRobot control system programming
Good IDEGood IDEGood documentationGood documentation
17 pdf files; 64 mB17 pdf files; 64 mBOnline documentationOnline documentation
2011 Kutztown University 41
Assessment – MATLABAssessment – MATLABGood IDEGood IDEStrong user communityStrong user community
BlogsBlogsDiscussion boardsDiscussion boardsGood code segmentsGood code segments
Good programming languageGood programming languageMatrix optimizedMatrix optimizedAdvance features, e.g., lambda, Advance features, e.g., lambda,
applyapply
2011 Kutztown University 42
Assessment – CreateAssessment – CreateUsefulUsefulStableStableUbiquitousUbiquitousNegativesNegatives
Battery short lastingBattery short lastingDoes not travel in straight lineDoes not travel in straight line
2011 Kutztown University 43
Assessment – AccessibilityAssessment – AccessibilityCost not prohibitiveCost not prohibitive
Create – $130Create – $130$220 with battery and charger$220 with battery and charger
Netbook ~ $330Netbook ~ $330Arduino ~ $30Arduino ~ $30MATLABMATLAB
$900 for 10 seat license$900 for 10 seat license$130 for student version with image $130 for student version with image
acquisitionacquisition
2011 Kutztown University 44
Future directionsFuture directionsSoftware repositorySoftware repositoryLearningLearning
BayesianBayesianGenetic algorithmsGenetic algorithmsNeural networks, etc.Neural networks, etc.
Control architecuresControl architecuresBehavior-basedBehavior-basedHybridHybridBlackboard?Blackboard?
2011 Kutztown University 45
Future directionsFuture directionsExperiment with other sensorsExperiment with other sensors
Touch {whiskers}Touch {whiskers}OdorOdorSound {whistle, a la Sound of Sound {whistle, a la Sound of
Music}Music}HeatHeatOdorOdorLight/brightnessLight/brightness
Wireless interfaceWireless interface““Watson, come here. I need you”Watson, come here. I need you”
2011 Kutztown University 46
Future of robotics at KUFuture of robotics at KUStrong student interestStrong student interestRejuvenated courseRejuvenated course
New syllabusNew syllabusApprovedApproved400 level – a mixed blessing400 level – a mixed blessingNot yet scheduledNot yet scheduledIn rotation?In rotation?PoliticsPolitics
2011 Kutztown University 47
Further InformationFurther InformationJeff’s graduate thesisJeff’s graduate thesis
Do You See What I'm Saying: Do You See What I'm Saying: Relating Language and Vision to Relating Language and Vision to Create Interaction Between Humans Create Interaction Between Humans and Robotsand Robots
Delves further into concepts Delves further into concepts discussed herediscussed here
https://github.com/jeffminton/Thesis
2011 Kutztown University 48
Questions?Questions?
2011 Kutztown University 49
The EndThe End
2011 Kutztown University 50
Extra slidesExtra slides
2011 Kutztown University 51
TemplateTemplateFont 44Font 44
font 40font 40Font 44Font 44
font 40font 40Font 44Font 44
font 40font 40Font 44Font 44
font 40font 40
2011 Kutztown University 52
AssessmentAssessmentSensors can be addedSensors can be addedOffers new areas of explorationOffers new areas of exploration
Heat sensorsHeat sensorsSound sensorsSound sensorsOdor sensorsOdor sensors
2011 Kutztown University 53
Unsuccessful segmentationUnsuccessful segmentation
2011 Kutztown University 54
Successful segmentationSuccessful segmentation
2011 Kutztown University 55
TemplateTemplateFont 44Font 44
font 40font 40Font 44Font 44
font 40font 40Font 44Font 44
font 40font 40Font 44Font 44
font 40font 40