UNIVERSTI TEKNIKAL MALAYSIA MELAKA

HUMAN ROBOT INTERACTION (HRI) USING SIMPLE,

INTERACTIVE ROBOTS

This report is submitted in accordance with requirement of the Universiti Teknikal

Malaysia Melaka (UTeM) for the Bachelor Degree of Manufacturing Engineering

(Robotics and Automation) (Hons.)

by

AHMAD FARIS BIN AZMIN

B051210190

930420-03-6001

FACULTY OF MANUFACTURING ENGINEERING

2016

DECLARATION

I hereby, declared this report entitled “Human-Robot Interaction (HRI) using Simple,

Interactive Robot” is the results of my own research except as cited in references.

Signature : …………………………………………

Author’s Name : AHMAD FARIS BIN AZMIN

Date : 24 JUN 2016

APPROVAL

This report is submitted to the Faculty of Manufacturing Engineering of UTeM as a

partial fulfillment of the requirements for the degree of Bachelor of Manufacturing

Engineering (Robotics and Automation) with Honours. The member of the

supervisory committee is as follow:

…………………………………………………………….

(Project Supervisor)

(DR. SYAMIMI BINTI SHAMSUDDIN)

ABSTRACT

Most of the robots used in human-robot interaction (HRI) studies are very expensive

due to their advance complexities and function. This has become a limitation in this

research area. Thus, this project is motivated to investigate how simple, interactive

robots such as My Keepon and RoboBuilder robot can be feasible alternative for HRI

studies. Simple robots can be the key platform in HRI studies since they are cheap

and are re-programmable for many purposes. In this project, HRI interaction

programs were developed using My Keepon, a simple toy like-creature and also

RoboBuilder, a simple humanoid robot. These robots will give initial exposure to the

children about robotic technologies. The feedback on My Keepon and RoboBuilder

features that are attractive to the children during HRI are analyzed using Kansei

Engineering method. Kansei Engineering is used to analyze whether the children are

attracted towards simple robot or not. Four children consisting of both girls and boys

from age 4 and 5 years old were selected from a kindergarten for the HRI

observation. The children interacted with My Keepon through static interaction and

active interaction for RoboBuilder. Video records of the interaction are analyzed

using Kansei Engineering by giving scores on eight different emotions. An interview

session after the experiment showed how the children emotionally perceive My

Keepon and RoboBuilder as a robotic play companion. From the results, the children

showed positive reactions when interacting with both robots. It is suggested that a

simple robot can be a play companion for the children in a kindergarten.

i

ABSTRAK

Kebanyakan robot yang dicipta untuk interaksi manusia-robot (HRI) adalah sangat

mahal disebabkan oleh kerumitan untuk menciptanya dan juga fungsi robot tersebut.

Ini telah menjadi satu limitasi bagi pengkaji di dalam bidang ini. Projek ini adalah

didorong oleh keadaan ini, bagaimana robot yang mudah dan interaktif seperti robot

My Keepon dan RoboBuilder boleh digunakan sebagai alternatif untuk kajian HRI.

Robot seperti ini boleh menjadi platform utama dalam kajian HRI dan mereka

senang diprogramkan semula untuk pelbagai tujuan. Projek ini membangunkan

program-program interaksi berasaskan HRI untuk My Keepon, robot yang seakan-

akan sebuah permainan dan juga RoboBuilder, sebuah robot yang berbentuk seperti

badan manusia. Robot ini akan memberi pendedahan awal kepada kanak-kanak

tentang teknologi robot. Maklum balas kepada ciri-ciri My Keepon dan RoboBuilder

yang menarik kepada kanak-kanak semasa interaksi HRI dianalisis menggunakan

kaedah Kansei Engineering. Kansei Engineering digunakan untuk menganalisisa

sama ada kanak-kanak tertarik ke arah robot mudah atau tidak. Empat kanak-kanak

yang terdiri daripada dua kanak-kanak perempuan dan dua kanak-kanak lelaki yang

berumur 4 dan 5 tahun akan dipilih daripada sebuah tadika untuk interaksi HRI.

Kanak-kanak berinteraksi dengan My Keepon melalui interaksi statik manakala

interaksi aktif untuk RoboBuilder. Video ini akan dianalisa menggunakan kaedah

Kansei Engineering dengan memberi skor kepada lapan emosi yang berbeza. Sesi

temuduga selepas eksperimen menunjukkan bagaimana kanak-kanak melihat My

Keepon dan RoboBuilder sebagai teman sepermainan. Dari keputusan itu, kanak-

kanak ini menunjukkan reaksi positif apabila berinteraksi dengan kedua-dua buah

robot. Ciri-ciri yang mudah pada kedua-dua robot berjaya menarik perhatian kanak-

kanak tersebut. Daripada ini, dapat dicadangkan bawaha robot mudah boleh menjadi

teman sepermainan kanak-kanak di sesebuah tadika.

ii

ACKNOWLEDGEMENT

Bismillahirrahmanirrahim,

In the first place, Alhamdulillah praises to the Almighty God, Allah SWT, with His

willing has given me the opportunity to successfully complete this Final Year Project

(FYP).

Secondly, I would like to express my sincere gratitude to my project supervisor, Dr

Syamimi Binti Shamsuddin for the continuous support throughout this project. Her

patience, motivation, enthusiasm and professionally giving advice had help me a lot

in making this project possible and running smoothly as per planning schedule.

Not to mention all those lecturers and staffs in Faculty of Manufacturing

Engineering, UTeM who helped me in many ways in this project especially to those

who are under the Department of Robotics and Automation. Their tireless

cooperation, insightful comments and inspiration had given me the ideas to run this

project successfully.

Thanks also to my friends who have been with me and contributed well in my project

by constantly supporting me, giving ideas, and always stay beside me when I am

having hard time during this project.

Last but not least, I would like to thank all my family especially to my parents for

giving me support spiritually and for their endless support and prayer to see me

completely finish this project.

iii

TABLE OF CONTENTS

Abstract i

Abstrak ii

Acknowledgement iii

Table of Contents vi

List of Tables vii

List of Figures viii

List of Abbreviations x

CHAPTER 1 INTRODUCTION

1.1 Background 1

1.2 Problem Statement 3

1.3 Project Objectives 4

1.4 Scope and Limitation 4

CHAPTER 2 LITERATURE REVIEW

2.1 What is a robot? 6

2.1.1 Types of Robot 8

2.2 Human-Robot Interaction (HRI) 10

2.2.1 Types of Robot Used in HRI 12

2.3 Kansei Engineering 15

2.4 My Keepon Robot 17

2.5 Review of Past Studies 20

2.6 Summary 28

iv

CHAPTER 3 METHODOLOGY

3.1 Flowchart of Project Methodology 29

3.2 Stage 1: Preliminary Study 31

3.3 Stage 2 and 3: Programming & Modification 32

3.3.1 Program Flow for My Keepon 32

3.3.2 Part A: My Keepon Hardware and Software 32

3.3.2.1 My Keepon Structure and Function 34

3.3.3 Part B: Designing circuit and Program 36

3.3.3.1 Arduino Shield 36

3.3.3.2 Developing the Program 37

3.3.4 Part C: Commisioning of My Keepon System 40

3.3.4.1 Hardware Testing and Analysis 41

3.3.5 Program Flow for RoboBuilder 41

3.3.5.1 Software Overview for RoboBuilder 43

3.3.6 Part F: Commissioning of RoboBuilder System 46

3.3.7 Part G: Debugging Process 46

3.3.8 Hardware Testing and Analysis 46

3.3.8.1 Uploading Programming 46

3.4 Stage 4: Experimental Setup for HRI 49

3.4.1 Experimental Procedures 51

3.4.2 Results & Documentations 53

CHAPTER 4 RESULTS & DOCUMENTATION

4.1 Designing Programming 55

4.2.1 Flowchart for the First Session Interaction (My Keepon) 55

4.2.2 Flowchart for the Second Session Interaction (RoboBuilder) 57

v

4.2 Results 58

4.2.1 Survey Results 59

4.2.1 Kansei Engineering Results 61

4.3 Discussion 66

4.3.1 Survey Clustering (My Keepon) 66

4.3.2 Survey Clustering (RoboBuilder) 68

4.3.3 Kansei Engineering Analysis (My Keepon) 70

4.3.4 Kansei Engineering Analysis (RoboBuilder) 72

4.3.5 Comparison on Kansei Engineering Results 74

4.4 Summary 75

CHAPTER 5 CONCLUSIONS & RECOMMENDATION

5.1 Conclusion 76

5.2 Recommendation for Future Works 77

5.3 Sustainability 77

REFERENCES 79

APPENDICES

A QUESTIONNAIRE SURVEY (MY KEEPON)

B QUESTIONNAIRE SURVEY (ROBOBUILDER)

C KANSEI EVALUATION SHEET

D KANSEI EVALUATION SHEET

E SUBJECT INFO SHEET FOR PARENT

F ETHICS APPROVAL LETTER

G GANTT CHART FOR PSM I

H GANTT CHART FOR PSM II

vi

LIST OF TABLES

2.1 Type of HRI used based on remote and proximate interaction 14

2.2 Comparison on previous research 21

3.1 Functions & Descriptions of MotionBuilder 44

4.1 Survey result for My Keepon 60

4.2 Survey result for RoboBuilder 60

4.3 Results on Kansei Engineering Method Analysis for My Keepon (Boys) 62

4.4 Results on Kansei Engineering Method Analysis for My Keepon (Girls) 63

4.5 Results on Kansei Engineering Method Analysis for RoboBuilder (Boys) 64

4.6 Results on Kansei Engineering Method Analysis for RoboBuilder (Girls) 65

vii

LIST OF FIGURES

1.1 NAO robot 2

1.2 Keepon Pro robot 3

2.1 Unimate, the first industrial robot builds 7

2.2 Classification of type of robot 8

2.3 U-Bots sorting objects 11

2.4 An example of HRI 12

2.5 The Micto Surge System 13

2.6 A furniture-assembling robot in IKEA 14

2.7 The Process of Kansei 15

2.8 Kansei Gateways 16

2.9 Keepon’s deformable body 17

2.10 Keepon’s kinematic mechanism 18

2.11 Keepon’s trajectory movement 18

2.12 Internal and external structures of Keepon 19

2.13 An example of kid playing with Keepon 20

3.1 Flowchart of the project methodology 30

3.2 Concept of Literature Review 31

3.3. Sub-Processs for Stage 2 33

3.4 Keepon trajectory movement 34

3.5 Keepon;s four degree of freedom 34

3.6 Keepon controller board 35

3.7 Connection between the systems 35

3.8 Arduino Shield 37

3.9 Systematic approach to programming 38

3.10 User Interface of ViKeepon 40

3.11 Programming process flow for RoboBuilder 42

3.12 MotionBuilder software 43

3.13 RoboBuilder Robot Configuration 45

viii

3.15 Debuggging Process Flowchart (RoboBuilder) 47

3.16 RoboBuilder Download Tool 48

3.17 Process flow for the experiment 49

3.18 Experimental plan layout for My Keepon 52

3.19 Experimental plan layout for RoboBuilder 53

4.1 The process flow for first session 56

4.2 The process flow for second session 57

4.3 My Keepon interaction session 58

4.4 RoboBuilder interaction session 59

4.5 Video evaluation session using Kansei Engineering 61 4.5 Understanding Rating (My Keepon) 66

4.6 Experience Rating (My Keepon) 67

4.7 Enjoyment Rating (My Keepon) 67

4.8 Understanding Rating (RoboBuilder) 68

4.9 Experience Rating (RoboBuilder) 69

4.10 Enjoyment Rating (RoboBuilder) 69

4.11 Kansei Graph (My Keepon) (+ve) 70

4.12 Kansei Graph (My Keepon) (-ve) 71

4.13 Kansei Graph (RoboBuilder) (+ve) 72

4.14 Kansei Graph (RoboBuilder) (-ve) 73

4.15 Radar graphs (+ve) 74

4.16 Radar graphs (-ve) 75

ix

LIST OF ABBREVIATIONS

HRI - Human-Robot Interaction

HCI - Human-Computer Interaction

KE - Kansei Engineering

UART - Universal Asynchronous Receiver/Transmitter

FKP - Fakulti Kejuruteraan Pembuatan

UTeM - Universiti Teknikal Malaysia Melaka

DOF - Degree of Freedom

DOE - Design of Experiment

ECE - Early Childhood Education

S.Pos - Start Position

D.Pos - Display Position

x

CHAPTER 1 INTRODUCTION

Chapter 1 will give a brief explanation about this project, starting with the

background of the project title, “Human-robot Interaction (HRI) with a Simple,

Interactive Robot”. This chapter will also discuss about the problem statement,

objectives, scope and limitation, and also the project planning..

1.1 Background

According to Fong et al. (2001) robot can be generally considered as a machine or

device in which it can performs tasks according to what the human command. Some

robots require some degree of guidance, which may be done using a remote control,

robot manipulator or with a computer interface that can be program and re-program

by human operator. This re-programmability characteristic can helps to distinguish

between robots and other automatic machine.

With the advancing technologies and rapid progress of digital computer that has

brought about the era of intelligent robots, they have evolved so much and are

capable of mimicking humans that they seem to have a mind of their own. In this 21st

century, robots are increasingly being developed for real world application areas,

such as robots as diseases treatment, rehabilitation, eldercare, or as household

appliance. Since intelligent robots can accomplish more complex and varied tasks,

the role of human control over the robot also dramatically increased.

1

Human-robot interaction (HRI) is an emerging field aimed at improving the

interaction between human beings and robots in various activities in which operate in

changing real-world environments. It is suggested that robots as partners can help us

accomplish more meaningful work and achieve better results (Fong et al., 2001)

Nowadays, HRI have been widely used in many fields to help us to achieve a better

life in the future. However, in this country that we live, HRI is still very unfamiliar

because not so many people have taken the step do the study and experimenting the

HRI due to some difficulties.

In order to do research in HRI field, the robot must be design in the simplest way

possible so that the human may feel comfortable and ease when interacting with the

robot. Robots such as Asimo and NAO (Figure 1.1) are designed based on human-

like structure or also can be called as humanoid robot while robot like Keepon Pro

(Figure 1.2) has a toy-like structure for the purpose of interacting with children

especially those with autistic disorder (Kozima. et al., 2005). There is several more

robots structure that is designed for the purpose of HRI study with each robot shape

suits different application

Figure 1.1: Humanoid robot NAO

(Source: https://www.ald.softbankrobotics.com/en/cool-robots/nao)

2

Figure 1.2: My Keepon robot

(Source: http://www.mykeepon.com)

1.2 Problem Statement

There are three problems statement that need to be highlighted in relation to this

project.

a) Most of the robots platforms used for research and study for HRI are very

expensive. Successful robots such as Asimo, NAO, Keepon Pro have a very high

price tag due to their complexity with high degrees of freedom. On the contrary,

low cost and simpler robots like Furby, Pleo or Ono are not officially modifiable.

Thus, modifying cheap commercial robots can be a key to develop low-cost

programmable platforms for HRI studies.

b) Advancements in technology are happening every single day. Kids around the

world are being exposed to the robotic technologies. Simple interactive HRI robot

can help to expose the children about the ever advancing robotic technologies

c) Robot developed for child-robot interaction is still at an early stage. To develop

robot which is suitable for children, experiment is done to analyze whether the

children are more attracted to complex or simple robot. By using the Kansei

Engineering method in this experiment, it can study and analyze the features of

the robot that managed to attract the attention of the children thus improving the

HRI robot for future studies.

3

1.3 Project Objectives

The ultimate goal of this project is to program My Keepon and RoboBuilder robot to

interact with children. The specific objectives that need to be achieved are;

a) To develop HRI programs for My Keepon and RoboBuilder to interact with the

children at a kindergarten

b) To analyze the emotional feedback of the children towards the robot through

interview session and Kansei Engineering Method

1.4 Scope and Limitation

The purpose of this project is to develop Human-Robot Interaction using a simple

interactive robot. The HRI involves both software and hardware to integrate the robot

to interact with humans.

This project involves theoretically and practically study on the development of My

Keepon and RoboBuilder robot to interact with children. The scope and limitation of

this project shall be indicated as follow.

a) This project concentrates on the programming of My Keepon robot using Arduino

Uno as the interface. Microsoft C# and Arduino will be used as the computer

languages to do the programming.

b) For RoboBuilder, MotionBuilder software which is provided by the manufacturer

will be used to program the robot.

c) 4 kindergarten children (age 4 – 5 years old) will be the participants for this study

since children are very sincere in giving opinion and expression.

d) The children that participate must be able to at least understand English since

there will be survey and questionnaire question at the end of the experiment.

e) The analysis of Kansei Engineering method will be focusing based on the

response from the children and their facial reaction towards the features of the

robot that they may find interesting during the interaction.

4

f) The children will do static interaction with My Keepon and spontaneous

interaction with RoboBuilder.

5

CHAPTER 2 LITERATURE REVIEW

This chapter presents a literature summary at the beginning of a research which aims

to review the critical points of the research. Literature review has been conducted in

order to obtain the information on the current technology available and the

methodology being used by other researchers on the same topic or field all around

the world. This chapter provides the key points related to human-robot interaction

(HRI) and Kansei Engineering method as a tool to evaluate HRI.

2.1 What is a robot?

Girme et al. (2007) proposed that a robot is an electro-mechanical device that can

accomplish either one of autonomous or semi-autonomous or pre-programmed tasks.

According to Robot Institute of America (1979), a robot is defined as a

reprogrammable, multifunctional manipulator designed to move material, parts,

tools, or specialized devices through various programmed motions for the

performance of a variety of tasks. (Bekey, 1999) stated in his thesis that robots are

generally machines that can sense, think and act which also frequently called as

intelligent agents.

However, as strange as it might seem, there is really no standard definition for a

robot. Joseph Engelberger, a pioneer in industrial robotics once said that even he

cannot define what actually is a robot but he know it is a robot when he see one

(Harris, 2007).

The word ‘robot’ was first cited by the Czech playwright and novelist Karel Capek

(1890 – 1938) from the Czech word, robota, meaning “forced labor”. The use of the

6

word Robot was introduced into his play R.U.R (Rossum’s Universal Robot) in the

early 1921. In the play, robot featured as fictional humanoid devices as the servants

for humans in which the robots eventually overthrow their human creators.

In the year 1954, George Devol invented the first digital and programmable robot

known as Unimate, the first kind of robot manipulators. Unimate (Figure 2.2) were

proposed by Devol in the United States; a company started by Devol and

Engelberger produced the first commercial versions of these machines in 1962.

Waurzyniak and Patrick (2006) said that the invention of Unimate has ultimately laid

the foundation in technology of modern industrial robotics. Industrial robots had

rapidly grown as an important role in the manufacturing industries since then.

Figure 2.1: Unimate, the first industrial robot build

(Source: http://www.computerhistory.org/)

Since robots are agents in the world they are also subject to the laws, rules, or

principles which are designed to ensure a friendly robot behavior environment is

created. Asimov has proposed in his book back on year 1950, ‘I, Robot’ the three

laws in robotics, also known as Asimov’s Laws. He then latter added a ‘zeroth law’.

\

7

• Law Zero : A robot may not injure humanity, or, through inaction, allow

humanity to come to harm.

• Law One : A robot may not injure a human being, or, through inaction, allow a

human being to come to harm, unless this would violate a higher

order law.

• Law Two : A robot must obey orders given it by human beings, except where

such orders would conflict with a higher order law.

• Law Three : A robot must protect its own existence as long as such protection

does not conflict with a higher order law.

2.1.1 Types of Robot

In the recent years there have been drastic improvements of robot whereby robots

do a lot of different tasks in many fields and the robot becomes more complex as

greater number of jobs are entrusted to them. In order to identify the robots, we

divide them by their application. A robot needs to be above all functional and

designed with qualities that suit its primary tasks. It depends on the task at hand

whether the robot is big, small, and able to move or nailed to the ground. Each and

every task means different qualities, form and functions; a robot needs to be

designed with the task in mind. Figure 2.2 further shows the classification of type

of robots.

Figure 2.2: Classification of type of robots

Type of robots

Mobile robot Stationary robot

Autonomous robot

Remote Control robot

8

Mobile robots are robots that are able to move freely according to the design and

programs setup. Usually they perform task such as loading and unloading of load in

factory, search areas, used in war, delivering heavy parts in industrial factories. For

an instant, AGV robots are used in industry to autonomously delivering parts

between several stations in a factory. Mobile robots are utilized for task where

individual cannot access without risking their lives.

Stationary robot is a robot that is completely contra to the mobile robot. Most

stationery robots are big and heavy, and created to perform repeating tasks without

ever for them need to move. Particularly dull, repeating and heavy tasks are

suitable for this kind of robots. Stationery robot is used for dull and repeating task

because robot will never be exhausted or tired, it can continuously perform its duty

day and night without ever complaining. Once the task is done, the robot can be re-

programmed easily to perform another task.

Autonomous robots are intelligent machine that capable to works on their own or

independently in which they can rely on the brains instead of human operator.

Autonomous robots run programs that give them the opportunity to decide the

action that they want to perform depending on the surroundings or environments.

Autonomous robots can learn or gain new knowledge by adjusting themselves to

the new environments or by adjusting new methods of accomplishing task instead

of doing it the same way.

In case where a robot must to perform more complicated yet undetermined tasks, an

autonomous robot is not the right decision to practice. Complicated tasks are still

best performed by human beings with genuine intellectual brainpower to make the

critical decision. Generally, a robot can only make decisions based on what the

human program them, whereas human can make decision based on emotional and

environment factors. Thus, in some situation, a human decision is needed whereby

a robot capability is used to perform task. In this way, a human can guide a robot

by remote control to do task which cannot be done by the human. A person can

perform difficult and usually dangerous tasks without being at the spot where the

tasks are performed.

9

For the purpose of studying HRI, robots should be designed to interact with human in

the simplest way possible. Robots used in HRI field are usually autonomous and

have living-like creatures so that the human will feel more comfortable when

interacting with them. The robot can be in the form of animal, human or other things

that it may look like it is alive. The design of the robot is different based on the

application and environment.

2.2 Human Robot Interaction (HRI)

From the earliest starting point of naturally enlivened robots, researchers have been

interested by the likelihood of robots interfacing with one another. Universally, the

world is moving forward into a situation where there is a significant desire within the

society to see useful robots ubiquitous in our day-to-day lives. In the modern era of

21st century, robots are not developed only to do heavy job in the industrial field,

instead they are also created to help humans in various activities. Humans seem to

have a specific interest about understanding and mimicking the nature in general,

and, specifically, human beings (Dautenhanh, 2007). The nature of interactivity and

social behavior in robot and humans are the main obstacles in the research field of

HRI.

Human-robot interaction research are closely affiliated with human-computer

interaction (HCI), as most of the modern robotic system employ hardware and

software components are used in other common computing system. Nonetheless, a

clear comparison can be made between them. Human-computer interaction basically

deals with user interface technologies such as the input devices; keyboard, mouse,

etc, in which the input is done manually by human operators. Whereby, HRI deals

with direct interaction between the human and robot itself, the robot interact with the

world and physically with the human operators.

As the field of simulated life developed, scientists started applying standards, for

example, indirect communication between individuals via modifications made to the

shared environment to accomplish "aggregate" or "swarm" robot conduct. Grass’e

10

clarify how social insect societies can collectively produce complex behavior

patterns and physical structures, regardless of the possibility that every individual

seems to work alone (Bonabeau, 1999). Deneubourg and his collaborators pioneered

the first experiments in simulated and physical “ant-like robots” (Deneubourg et al.,

2000) in the early 1990’s. Since then, numerous researchers have developed robot

collectives (Kube, Bonabeau, 2000) and have used robots as models for studying

social insect behavior.

Figure 2.3: U-Bots sorting objects (Melhuish, Holland and Hoddell, 1998)

However, the research shown above only as far uses the methods of self-organization

and behavior inspired by social insect societies. Such social orders are unknown,

homogeneous gatherings in which individuals don't make any difference. This kind

of "social conduct" has ended up being an attractive model for robotics technology,

especially due to the abilities of the simple robot to perform difficult task (e.g.,

soccer, basketball playing). (Dautenhahn, Billard, 1999) proposed that social robots a

part of the heterogeneous group; a society consisting of both robots and humans.

They are able to recognize each other and engage in social interaction, interpret the

environment based on their experience, and explicitly communicate with and learn

from each other.

HRI robot represents an interaction technology, which breaks the boundaries of

interaction between human and robot. Behaviours and appearances of robot have

drastically changed since early 1990s, and with the never ending technologies, they

still and will continue to change. Charles Rich in 2008 said we need to understand

how to program human-like robots if we want them to move freely among us.

11