robotics and neuroscience · robotics, machine learning, and brain-based theories.-predictive...

Proceedings of the 4th Congress on Robotics

and Neuroscience Co

ngre

ss P

roce

edin

gs

November 15-17, 2018VALPARAÍSO, CHILE

Cristobal J. Nettle Miguel A. Solis (Eds.)

Cristóbal J. Nettle,Miguel A. Solis (Eds.)

Proceedings of the4th Congress on Roboticsand NeuroscienceNOVEMBER 15-17, 2018

Valparaíso, ChileCongress Proceedings

Volume Editors

Cristóbal J. Nettle, M.Sc. on ElectronicsTreasurer and Researcher at Centro de Innovación y Robótica, [email protected]

Miguel A. Solis, Dr.-Eng. on InformaticsGeneral Secretary and Researcher at Centro de Innovación y Robótica, ChileAcademic at Universidad Católica del Norte, [email protected]

ISBN: 978-956-09282-0-7.

Copyright ©2019 for the individual papers by the papers’ authors.Copying permitted only for private and academic purposes. Thisvolume is published and copyrighted by its editors.

2

[email protected]

[email protected]

Contents

Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

F. Ollino, M.A. Solis, H. AllendeBatch Reinforcement Learning on a RoboCup Small Size League KeepawayStrategy Learning Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

F. TorresTrending Topics on Science, a Tensor Memory Hypothesis Approach . . . . . . . . . . . . . . 18

R. Alfaro, M. Aubel, P. Yañez, et al.AIS: Artificial Intelligent Soccer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

C. Angel, C.J. NettleA Theoretical Strategy for Enhancing Learning Through Metacognitive Practiceson a Constructivist Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

M.M. Vallejo-Jiménez, J.J. Martínez-Puerta, S. Bedoya-Agudelo et al.SENA Tecnoacademia Risaralda and Caldas as a Collaborative LearningScenario in Robotics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

O.A. Silva, P. Sigel, W. Eaton et al.CRABOT: A Six-legged Platform for Environmental Exploration andObject Manipulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

3

Page intentionally left blank.

Proceedings of the4th Congress on Robotics and Neuroscience

CRoNe2018Valparaíso, Chile. November 15-17, 2018

5

Preface

CRoNe, the Congress on Robotics and Neuroscience, is an encounterthat acts not only as a bridge but also as a fruitful land for collabora-tion and discussion concerning recent advances in the frontiers of ar-tificial intelligence, robotics and neuroscience, fostering the exchangeof ideas among different, and often fairly separated, scientific fields.The congress, part of the Latin American Robotics Week1, organized 1 Further details in

http://www.roboticsweekla.com.by Innovación y Robótica Estudiantil UTFSM 2, is a meeting point for2 a multidisciplinary group of both,undergraduate and graduate students,focused on R&D with emphasis onrobotics. More information inhttp://innovacionyrobotica.usm.cl.

people from engineering, human and biological sciences promotingthe development and understanding of complex intelligent systems.

At its 4th version, the Congress on Robotics and Neuroscience wasfocused on four different areas:- Development of meaning: with approaches from Developmental

Robotics, Machine Learning, and Brain-based theories.- Predictive coding for cognitive development: introducing analyses

from Neuroscience, Computational Neuroscience, and AI.- Multimodal cognition: under neuroscientific and psychological

scopes.- Experimental analyses and methodologies: presenting novel

methodologies for closed-loop brain training, brain functionalconnectivity analyses and Machine Learning applications.

On this opportunity, we had the pleasure to count with ten keynotespeakers who presented state-of-the-art results encouraging discus-sions in relation to one or more of these areas. Three workshops in-troducing scientific tools and methodologies completed the program,spreading knowledge and access to processing and developmentalenvironments on Interactive Reinforcement Learning and data analy-sis in Electroencephalogram signals.

The accepted works presented during the conference, collectedin this volume, tackles a wide variaty of problems comming fromareas as robotics, education or data analytics. Along the differentproposed analyses, while some approaches are based on a singledomain, discussions as the ones in Torres (2018) are crossed amongareas. Torres (2018) evaluates different strategies for finding trendingtopics on a set of scientific articles, through a discussion arround the

6

http://www.roboticsweekla.com

http://innovacionyrobotica.usm.cl

recent approaches aiming to describe the brain processes of memory.Following context characteristics, Ollino et al. (2018) propose a

single-agent approach to tackle a multi-agent problem using batchreinforcement learning for developing defensive strategies in aRoboCup SSL (Small Size League) robotic team. The proposed imple-mentation takes advantage of the framework of the team controller,which usually has a team-level decision-maker. The controller learnsand executes different responses detailing the action for each agent,from a pre-defined set of actions, following a single team goal to min-imize the score of an enemy team. In this same scenario, the alreadycompeting team shown in Alfaro et al. (2018), presents an in-depthdescription about design and implementation tasks for different areastackled in a real SSL team (with physical prototypes), with chal-lenges ranging from structural issues to field strategy problems. Inanother real world scenario, Silva et al. (2018) describes CRABOT, asix-legged robotic platform for autonomous field recognition and ob-ject manipulation. CRABOT was developed under a leg-arm hybriddesign: similar to a crab, it can stand on four of its legs leaving twocompletely free for object interaction, being able to deploy differenttools attached to each limb.

Regarding to educational approaches, work in Vallejo-Jiménezet al. (2018) describes some experiences from environments de-fined as technological academies for STEM (Science-Technology-Engineering-Math) learning scenarios, for primary and secondarylevel school students in courses such as Mathematics, Physics or ap-plied sciences such as Robotics or Virtual Technologies. In a moretheoretical approach, work in Angel and Nettle (2018) presents areview over the last decades of educative methodologies, identify-ing keys for the development of competences and the acquisitionof meaningful learning. The proposal from Angel and Nettle (2018)conclude with a procedural model for educational scenarios, whichincorporates metacognitive practices for achieving student’s aware-ness of what is learnt and how and when to apply it.

7

References

Alfaro R, Aubel M, Yañez P, Reyes P, Rodenas T, Hernández N, PintoF, Kim SH, Barrera T, Torres D, Vicencio I, Alvarez J, Osorio F,Castillo S. AIS: Artificial Intelligent Soccer. In: Nettle CJ, Solis MA,editors. Proceedings of the 4th Congress on Robotics and Neuroscience,2018 No. 2312 in CEUR Workshop Proceedings, Aachen; 2018. p.25–31. http://ceur-ws.org/Vol-2312/#CRoNe2018_article_3.

Angel C, Nettle CJ. A Theoretical Strategy for Enhancing LearningThrough Metacognitive Practices on a Constructivist Methodology.In: Nettle CJ, Solis MA, editors. Proceedings of the 4th Congress onRobotics and Neuroscience, 2018 No. 2312 in CEUR Workshop Pro-ceedings, Aachen; 2018. p. 32–38. http://ceur-ws.org/Vol-2312/

#CRoNe2018_article_4.

Nettle CJ, Solis MA, editors. Proceedings of the 4th Congress onRobotics and Neuroscience (CRoNe2018) No. 2312 in CEUR Work-shop Proceedings, Aachen; 2018, http://ceur-ws.org/Vol-2312/.

Ollino F, Solis MA, Allende H. Batch Reinforcement Learning ona RoboCup SSL keepaway strategy learning problem. In: NettleCJ, Solis MA, editors. Proceedings of the 4th Congress on Roboticsand Neuroscience, 2018 No. 2312 in CEUR Workshop Proceed-ings, Aachen; 2018. p. 11–17. http://ceur-ws.org/Vol-2312/

#CRoNe2018_article_1.

Silva OA, Sigel P, Eaton W, Osorio C, Valdivia E, Frois N, Vera F.CRABOT: A six-legged platform for environmental exploration andobject manipulation. In: Nettle CJ, Solis MA, editors. Proceedingsof the 4th Congress on Robotics and Neuroscience, 2018 No. 2312 inCEUR Workshop Proceedings, Aachen; 2018. p. 46–51. http:

//ceur-ws.org/Vol-2312/#CRoNe2018_article_6.

Torres F. Trending Topics on Science, a tensor memory hy-pothesis approach. In: Nettle CJ, Solis MA, editors. Proceed-ings of the 4th Congress on Robotics and Neuroscience, 2018 No.2312 in CEUR Workshop Proceedings, Aachen; 2018. p. 18–24.http://ceur-ws.org/Vol-2312/#CRoNe2018_article_2.

Vallejo-Jiménez MM, Martínez-Puerta JJ, Bedoya-Agudelo S,Salgado-Giraldo ND. SENA Tecnoacademia Risaralda and Caldasas a collaborative learning scenario in Robotics. In: Nettle CJ, SolisMA, editors. Proceedings of the 4th Congress on Robotics and Neuro-science, 2018 No. 2312 in CEUR Workshop Proceedings, Aachen;2018. p. 39–45. http://ceur-ws.org/Vol-2312/#CRoNe2018_

article_5.

8

http://ceur-ws.org/Vol-2312/#CRoNe2018_article_3



http://ceur-ws.org/Vol-2312/








OrganizationCRoNe 2018 was organized by Innovación y Robótica Estudiantil UTFSM in collaboration with Centro deInnovación y Robótica. This Congress was held at Universidad Técnica Federico Santa María in Valparaíso,Chile.

Program Committee Chairs

Cristóbal J. Nettle (Centro de Innovación y Robótica, Chile)Miguel A. Solis (Universidad Católica del Norte, Chile)

Program Committee

Juan Felipe Calderón (Universidad Andrés Bello)Evelyn Cordero (Pontificia Universidad Católica de Chile)Francisco Cruz (Universidad Central de Chile)Lorena Lobo (Universidad a Distancia de Madrid)Nicolás Navarro-Guerrero (Aarhus University)

9

Proceedings of the 4th Congress on Robotics and Neuroscience

Batch Reinforcement Learning on aRoboCup Small Size Leaguekeepaway strategy learning problemFranco Ollino1, Miguel A. Solis2,3*, Héctor Allende1

*For correspondence:[email protected] ()

1Universidad Técnica Federico Santa María; 2Universidad Católica del Norte; 3Centro deInnovación y Robótica

Abstract Robotic soccer provides an adversarial scenario where collaborative agents have toexecute actions by following a hand-coded or a learned strategy, which in the case of the Small SizeLeague, is given by a centralized decision maker. This work takes advantage of this centralizedapproach for modelling the keepaway strategy learning problem which is inherently multi-agent, asa single-agent problem, where now each robot forms part of the state of the model. One of theclassical reinforcement learning methods is compared with its batch version in terms of amount oftime for learning and concluding about updates efficiency based on experiences reusability.

IntroductionWhen we talk about Batch Reinforcement Learning (BRL), we refer to one of the current line ofresearch in the field of Reinforcement Learning (RL), also concerned about solving sequentialdecision problems modelled by a Markovian Decision Process (MDP). Given the nature of theseproblems, as the intuition may suggest, the scope of this type of learning has extended to areas likeRobotics applications (Kober et al., 2013).As in the classical approach, with online algorithms, we still focus on teaching an agent how

to behave under certain conditions based on punishments or rewards (reinforcement signals)depending on the results of applying a certain action (Sutton et al., 1998). Q-learning (Watkins andDayan, 1992) is one of the most popular online algorithms, whose updates are computed in anincremental manner.BRL approach aims to collect a bunch of experiences and then use them for updating action

influences instead of updating the action value function in an incremental way. In this batchframework, algorithms like Experience Replay (ER) or Fitted Q Iteration (FQI) (Ernst et al., 2005) canbe found.The Robot Soccer World Cup (RoboCup, (Kitano et al., 1997)) is an annual competition whose

main objective is far beyond than just playing a robotics soccer game, it presents a natural scenariowhere RL problems can be found, in addition to several multidisciplinary challenges on its differentleagues such as small size league, standard platform league, humanoid league and others. In thisproblem, a team of cooperative agents have to play a soccer match against another team composedof autonomous agents, noting that a possible objective for a given team could be to keep as faras possible the ball from its own goal area. In order to achieve this objective, many works can befound in the literature, from keepaway strategies using a multi-agent approach (Stone et al., 2005)to algorithms focused on training just the goalkeeper, as (Ahumada et al., 2013) or (Celiberto et al.,2007).

11


This work, like in (Ahumada et al., 2013) and (Celiberto et al., 2007) uses a grid for discretiz-ing the state space of the agent and therefore avoiding to deal with a continous state spacerepresentation where tabular methods become impractical (Baird and Klopf, 1993).Unlike the above references, most of the works found on literature (Pietro et al., 2002; Kalyanakr-

ishnan and Stone, 2007; Sawa and Watanabe, 2011; Stone et al., 2005) generates a state spacerepresentation based on angles and distances from the keeper (current learning robot that possessthe ball) to every robot on the confined space of interest. Since large (or continuous) state spacesrequire function approximation, (Stone et al., 2005) uses tile-coding for approximating Q-valueswhen implementing and comparing online RL algorithms (Q-learning vs Sarsa(�)).Getting closer to our case of interest, assumptions on (Kalyanakrishnan and Stone, 2007) allow

the agents to communicate with each other in order to share their experiences. They also compareBRL with online RL algorithms, stating that Fitted-Q Iteration and Experience Replay reach a closeperformance with each other, but they both outperforms the online learning algorithm used in(Stone et al., 2005).This document intends to compare Q-learning and its batch-version using Experience Replay

on a simulation of the RoboCup Small Size League, noting that this involves a centralized decisionproblem, given the setup of the league, reducing the learning problem to a single agent case whereeach robot plays a fundamental role on the state space representation.The remainder of this document is as follows: Section 2 makes a further description for BRL, pre-

senting the algorithms that will be used later. Section 3 makes a brief description of RoboCup SmallSize League, and explains how this setup can be used for introducing variations on the approachesfound on literature for learning a keepaway strategy, while Section 4 shows the implementation ofBRL algorithms on a simulated environment. Finally, Section 5 draw some final conclusions.Batch Reinforcement LearningReinforcement learning (Sutton et al., 1998) (RL) tackles the problem of an agent that learns whileinteracting with the environment, deciding which action a to execute on the current state s of itsenvironment, which transfers the agent to another state s′ receiving a reward (reinforcement signal)whose nature would provide a quantification of how desirable was that choice. This problem canbe formulated as an MDP (Sutton et al., 1998), composed by a tuple ( ,, ,) where

• : denotes the set of all possible states.• : is a set of all the actions the agent can execute.• : × × → [0, 1] is a state transition function, which gives the probability that when theagent is in state s and executes action a, the agent will be transferred to another state s′.

• : × → ℝ is a scalar (real-valued) reward function.• �: → denotes the mapping from states to action, describing the policy the agent shouldtake given a certain state.

As mentioned before, the task of the agent is to learn the sequence of actions (therefore the optimalpolicy, �∗ ) that leads to maximize the expected sum of all the rewards received in the long-term.This is tackled by maximizing the return Rt, i.e. the discounted sum of rewards that the agent willobtain from time t, given by

Rt =n−1∑k=0

krt+1+k, (1)where stands for the discount factor, with 0 ≤ < 1, and rt+1 stands for the expected (scalar)reward obtained for executing action at in state st. Then, two quantifications for the expected returnare defined, the value function and action value function, V � and Q� respectively.Value function is defined as the expected return when the agent is on state st at time t,

V �(s) = E�t{Rt|st = s}, (2)

12


while the action value function is defined as the expected return when the agent executes at onstate st at time t following policy �,Q�(s, a) = E�t{Rt|(st = s, at = a)}. (3)

Both functions are clearly related, asV �(s) = Ea|s{Q�(s, a)}. (4)

A representative method in model-free RL is Q-Learning (Watkins and Dayan, 1992), which makesan approximation of the optimal action-value function based on the optimal policy, by makingsuccessive updates for estimations of Q, this update would be given by

Q(st, at) ← (1 − �)Q(st, at) + �(rt+1 + max

aQ(st+1, a)

), (5)

where this approximation, Q, corresponds to the learned action-value function, and � stands forthe learning rate.

In order to understand the difference between the incremental update from online algorithmsand simultaneous update of batch algorithms, consider two consecutive transitions (s, a, r, s′),(s′, a, r, s′′) and the classical online Q-learning algorithm. Then, when Q(s′, a) is computed using theupdate rule on (5), this change will not be backpropagated to Q(s, a) nor any of the state-actionpairs preceding s′, being updated just when those states are visited again.

In the pure batch reinforcement learning approach, the agent does not interact with the envi-ronment while the learning phase is taking place. In growing batch reinforcement learning, whichmost of the modern batch algorithms are based on, the task of collecting transitions and learningfrom them are alternated for improving the exploration policy.Algorithm 1 describes the procedural form of a (growing) BRL approach independently of the

algorithm used for updating Q-values, as shown on (Kalyanakrishnan and Stone, 2007). Note thatwhen the number of forgotten experiences,m, is the same as the size of the size of the batch, i.e.m = |D|, experiences are forgotten so growing BRL is reduced to pure BRL, which is not the case ofthis proposal.ALGORITHM 1Batch reinforcement learning procedure1: Initialize Q(s, a) arbitrarily ∀s ∈ , a ∈ 2: Initialize batch of experiences D as an empty set3: repeat4: for each episode do5: for each step t on current episode do6: Identify current state st7: Choose a suitable action at in state st using policy derived from Q8: Observe rt+1 and st+1 when taking action at9: Add experience (st, at, rt+1, st+1) on batch D10: st ← st+111: end for12: end for13: Update Q values14: Forget m experiences from batch D15: until action value function convergence is reachedMoreover, (Kalyanakrishnan and Stone, 2007) states that is better (for their task) to use all

the experiences gathered so far. This means that if every batch consists on experiences from20 episodes, then the first updates of Q estimations will consists on experiences from those 20

13


Figure 1. RoboCup SSL scheme

episodes. Then the second time these updates are computed it will consists on experiences from40 episodes and so on, which represents an extremely memory consuming process.One of the basic BRL algorithms, Experience Replay (ER) aim to improve the speed of conver-

gence of the action value function by replaying observed transitions repeatedly just as if they werenew observations. Algorithm 2 shows the procedural form of this algorithm.ALGORITHM 2Experience Replay procedure1: for each training iteration do2: for each transition (si, ai, ri, si+1) on D do3: Update Q(si, ai) by using

Q(si, ai) ← (1 − �)Q(si, ai) + �(ri+1 + max

aQ(si+1, a)

)

4: end for5: end forIt is immediate to note that what this algorithm does, is to compute several times the updates

of Q-learning on collected transitions as an offline algorithm would do, thus speeding up thepropagation of Q values to preceeding states, but then the system is allowed to collect newtransitions for improving those previously computed estimates.Test domain: RoboCup SSLRoboCup presents a challenging domain where a team of robots have to play a soccer match againstanother robotic team, where the particular assumptions on the game varies across the differentleagues. This application focuses on the Small Size League (SSL), inspired by the development andresearch work made by Sysmic Robotics USM (previously known as AIS Soccer) (Rodenas et al.,2018), a group of students whose main objective is to compete on this annual event, and also teststate-of-the-art computational intelligence techniques on this particular setup.Figure 1 depicts the scheme of this league, where the current positions of each robot at both

teams is given as result of the image proccessing made by SSL-Vision, whose images are acquiredthrough video cameras provided by the organization comittee, located at the top of the soccer field.Then, both teams receive the exact same data to their own decision maker programs, which oncean action is chosen informs the actions to take for each robot of its team via a wireless channel.Although we tackle the problem of finding a keepaway strategy, several challenges arises at

the Small Size League in addition to the already mentioned problems like goalkeeper training on(Ahumada et al., 2013), learning the opponent strategy as on (Yasui et al., 2013),or learning tocontrol the dribbler (Riedmiller et al., 2008), noting that the work therein focuses on the MiddleSize League. This latter problem also applies to the Small Size League, being specially difficult to

14


Figure 2. problem setup on GrSim simulator, with 3 keepers and 2 takers

keep possesion of the ball with the dribbler while changing the orientation of the robot.Implementation and resultsModelling the learning problemAlthough we use a slightly different state space representation compared with (Stone et al., 2005),by using a centralized decision taking problem given that we have a global vision of the field unlikesome other RoboCup leagues. Then, we set the keepaway learning problem to be composed of 3keepers, robots in charge of keeping the ball as long as possible away from their goal area, and 2takers, which are robots from the opponent team and whose objective is to take the ball and shootto the (center of) goal area.As the offense strategy learning for allowing the takers to learn better strategies to effectively

score is out of the scope of this work, we fixed their policy in a manner that they are always chasingthe ball, and once they got it, just shoot to the goal area.Figure 2 shows the setup of this problem in the simulated environment where algorithms will

be tested, GrSim (Monajjemi et al., 2011), which has been very helpful for testing computationalintelligence methods before implementing them on the real robots.The state is composed by distances from every keeper to all the other robots, including takers

and other keepers as shown in Figure 2. Also the distance from the ball to every robot is considered,and the angle between a keeper and each taker (with respect to an imaginary horizontal line acrossthe soccer field). In other words, the state st at a given time t is composed by

• dist(Ki ,ball),• dist(Tj ,ball),• dist(Ki ,Kj ), i ≠ j,• dist(Ki ,Tj ),• angle(Ki ,Tj ),

where Ki stands for the i-th keeper and Tj for the j-th taker. Also, the reader should note thatalthough different state representations could work for a given problem, the angle is neccessary formodelling this problem. Even when assuming that all the robots are always facing the ball, since ifjust the distance d from the i-th keeper to the j-th taker is used, there would be theoretically infinitepoints around a circle of radius d and centered on the position of the keeper where the taker couldpossibly be.Then, the possible actions to execute by a keeper are• ℎold(): all keepers remains on their current positions without making any pass nor trying tointercept the ball.

• pass(Ki, Kj): the i-th keeper performs a pass to the j-th keeper, where obviously i ≠ j since itwould be equivalent to hold() action.

• intercept(K1, K2, ..., Kn): send keepers to intercept the ball whenever its respective binary argu-ment is set to 1.

15


Figure 3. Goals scored by enemy team

Figure 4. Time of possesion on the ball by keepers

In this case, since there are 3 keepers, intercept(0, 1, 0) would send 2nd keeper to interceptthe ball, while intercept(1, 0, 1) would send 1st and 3rd keepers to intercept the ball. Note thatintercept(0, 0, 0) is not allowed, since it would be equivalent to ℎold() action.Note that unlike the work in (Stone et al., 2005), since we have a global vision of the field and

thus focused on a centralized decision making problem, we learn a Q value function for the wholesystem and not one for each keeper. We have identificators for each keeper, so the 1st keeper willbe K1 always, and does not refer to the keeper who is closest to the ball.Since the final objective of the keepaway learning problem, is to learn to keep as long as possiblethe ball away from the goal area, then we will reward actions that privileges the ball possesion,and punish actions that leads to lose possesion and punish harder when it leads to a goal scoredagainst the team.Simulation resultsWhen implementing the algorithms described on Section 2, we used a growing BRL approach wherethe batch of experiences D contains transitions from 20 episodes, where each one lasts 2 minutesof gameplay (without considering reset time when a goal is scored and robots are re-locating). Then,after updating Q-values estimations all those transitions are discarded, so the size of the Batchalways have the data for 20 episodes when entering to the learning phase.According to rewards obtained through the learning episodes, whose values are set to 5 for

keeping possesion on the ball, -5 in case of losing possesion and -50 in case of the enemy teamscoring a goal. Then, according to these reinforcement values, Figure 4 shows the evolution of timepossesion on the ball.

It can be seen from Figure 3, where the line represents themean through 10 reproductions of thelearning task, that batch version of Q-learning using Experience Replay achieve better performancecompared with its classical online version on a smaller amount of time. However, it is expected thatafter several learning episodes more, batch version would learn faster but they both achieve thesame results at last.Despite the efficiency on the use of collected transitions of the learning agent, speed of con-

vergence for both algorithms is directly affected by the number of possible states obtained from

16


the chosen state space representation. Then, as the discretization grid becomes thinner, the statespace becomes larger and tabular methods become slower and even impractical for a continuousstate space representation, so function approximation methods are needed.ConclusionsAs expected because of data reusability of experiences gathered so far, Experience Replay learnfaster in terms of defending the goal area, and this is mainly due to its synchrony nature and a betteruse of collected experience on the interaction process between the agent and its environment forthis task. Obtained results shows the benefits of re-using data efficiently and in an inherently multi-agent problem tackled from a single agent learning task given the centralized setup of this league.Future work may include a more in-depth analysis including other update rules and strategies inBatch Reinforcement Learning methods, as well as field testing in other leagues, and consideringa continuous state space representation using function approximators such as artificial neuralnetworks or a fuzzy representation of states.ReferencesAhumada GA, Nettle CJ, Solis MA. Accelerating Q-Learning through Kalman Filter Estimations Applied in aRoboCup SSL Simulation. In: Robotics Symposium and Competition (LARS/LARC), 2013 Latin American IEEE; 2013.p. 112–117.Baird LC, Klopf AH. Reinforcement learning with high-dimensional, continuous actions. Wright Laboratory,Wright-Patterson Air Force Base, Tech Rep WL-TR-93-1147. 1993; .Celiberto LA, Ribeiro CH, Costa AH, Bianchi RA. Heuristic reinforcement learning applied to robocup simulationagents. In: Robot Soccer World Cup Springer; 2007. p. 220–227.Ernst D, Geurts P, Wehenkel L. Tree-based batch mode reinforcement learning. Journal of Machine LearningResearch. 2005; 6(Apr):503–556.Kalyanakrishnan S, Stone P. Batch reinforcement learning in a complex domain. In: Proceedings of the 6thinternational joint conference on Autonomous agents and multiagent systems ACM; 2007. p. 94.

Kitano H, Asada M, Kuniyoshi Y, Noda I, Osawa E. Robocup: The robot world cup initiative. In: Proceedings of thefirst international conference on Autonomous agents ACM; 1997. p. 340–347.

Kober J, Bagnell JA, Peters J. Reinforcement learning in robotics: A survey. The International Journal of RoboticsResearch. 2013; 32(11):1238–1274.Monajjemi V, Koochakzadeh A, Ghidary SS. grsim–robocup small size robot soccer simulator. In: Robot SoccerWorld Cup Springer; 2011. p. 450–460.

Pietro AD, While L, Barone L. Learning in RoboCup keepaway using evolutionary algorithms. In: Proceedings ofthe 4th Annual Conference on Genetic and Evolutionary ComputationMorgan Kaufmann Publishers Inc.; 2002. p.1065–1072.

Riedmiller M, Hafner R, Lange S, Lauer M. Learning to dribble on a real robot by success and failure. In: Roboticsand Automation, 2008. ICRA 2008. IEEE International Conference on IEEE; 2008. p. 2207–2208.

Rodenas T, Alfaro R, Reyes P, Pandolfa D, Pinto F, Aubel M, Yanes P, Barrera T, Kim SH, Castillo S. AIS TeamDescription Paper. . 2018; .Sawa T, Watanabe T. Learning of keepaway task for RoboCup soccer agent based on Fuzzy Q-Learning. In:Systems, Man, and Cybernetics (SMC), 2011 IEEE International Conference on IEEE; 2011. p. 250–256.

Stone P, Sutton RS, Kuhlmann G. Reinforcement learning for robocup soccer keepaway. Adaptive Behavior.2005; 13(3):165–188.Sutton RS, Barto AG, Bach F, et al. Reinforcement learning: An introduction. MIT press; 1998.Watkins CJ, Dayan P. Q-learning. Machine learning. 1992; 8(3-4):279–292.Yasui K, Kobayashi K, Murakami K, Naruse T. Analyzing and learning an opponent’s strategies in the RoboCupsmall size league. In: Robot Soccer World Cup Springer; 2013. p. 159–170.

17


Trending Topics on Science, a tensormemory hypothesis approachFelipe Torres1*

*For correspondence:[email protected] (FT) 1Universidad Técnica Federico Santa María

Abstract The current human knowledge is written. Documenting is the most used manner topreserve memories and to store fantastic stories. Thus, to distinguish the reality from fiction, thescientific writing cites previous works moreover than become form experimental setups. Booksand scientific papers are only a small part of the existent literature but are considered more thrustas information sources. It is useful to find more relations and to know where to focus the lookup ofa topic using the information about the authors and the keywords on the titles and abstracts. Thisis possible using relational databases or knowledge graphs, a semantic approach, but with thetensor memory hypothesis, that adds a temporal dimension, is possible to process the informationwith an episodic memory approach. If well, knowledge graphs are of extended use on questionanswering and chatbots, they need a previous relational schema generated automatically orby-hand and stored in an easy-to-query file format. I use JATS, a standard format that allowsintegrating scientific papers in semantic searches but is not spread on all scientific publishers, toextract the markup tags from PDF files, current year journal articles of one particular topic, andthen construct the tensors memory with their references to extract relations and predictions withstatistical relational learning techniques.

IntroductionMemory is defined as the ability to record information and after recall it. Writing is a humaninvention that facilitates this capacity in particular for declarative memories that are facts or eventsthat can be expressed with language and it could be of two types: semantic or episodic (Tresp et al.,2017).The memories and knowledge of humanity are stored on written documents, getting more

reliability if they include references to previous works from others authors. Scientific articles arethe model of well-structured presentation and storage of information, each one of them with anown title, explicit authorship, and references to information related to other documents or withinthe same document. But, what almost always is relevant for the consideration of reading them,the retrieval action, is their publishing year. Thus, their ordered structure makes possible to usethem as a representation of global human episodic knowledge and memories. Also, scientificpublication as a human activity could be modeled as a social network. From this kind of networksthe expression “trending topic” emerged to call the more frequent term or word used in a specifictemporal window and it is understood as the principal theme or main subject that is related to theinformation described in a piece of content.

In a mathematical and computational framework, semantic memories could be representedas knowledge graphs, where the entities are nodes and the links are relations between them.A relation between entities is then possible to define as a triple (s, p, o) or as a simple sentencesubject-predicate-objective. An episodic memory adds a time marker, thus a temporal preposi-tional phrase is added to the simple sentence: subject-predicate-objective-temporal_preposition

18


or a quad (s, p, o, t). This approach is widely used on semantic web technologies under the LinkedDatamethodology (Bizer et al., 2011).Thus, it is plausible to use complex networks analysis tools to search for the most relevant

relations between authors, paper titles or keywords. The scientific publication databases can easilycontain millions of authors, papers and their respective citations. A reduced number of relevantdocuments is expected from a specific topic query, and not thousands of results that search engineslike Google Scholar or publisher’s own engines could generate for a given chain of words. The fieldof science of science studies these relations and the former works were realized using knowledgegraphs, that are expressed as adjacency matrices. If the temporal dimension and various typesof relationships are considered, then its possible to form tensors of fourth order. A matrix Xof the network could be bipartite (X ∈ ℝn×m) if there are two types of nodes (authors-articles,authors-words, articles-words) or monopartite (X ∈ ℝn×n); unweighted ( xij ∈ {0, 1}) or weighted(xij ∈ ℝ), directed or undirected (XT = X) (Zeng et al., 2017).(Tresp and Ma, 2017) introduced the Tensor Memory Hypothesis, where a knowledge graph is

represented by a Tucker decomposition of the tensors. It is based on representational learning, i.e,a discrete entity e is associated with a vector of real numbers ae called latent variables. (Tresp andMa, 2017) also argue that representational learning might also be the basis for perception, planningand decision making. From a physiological point of view, there is evidence that the hippocampusplays a central role in the temporal organization of memories and supports the disambiguationof overlapping episodes (Eichenbaum, 2014a), then in the standard consolidation of memorytheory (SCT), the episodic memory is a neocortical representation that arises from hippocampalactivity while in the multiple trace theory (MTT) the episodic memory is only represented on thehippocampus and is used to form semantic memories on the neocortex. Also, there is evidence ofthe existence of “place cells” and “time cells”in the hippocampus and that these support associativenetworks that represent spatiotemporal relations between the entities of memories (Eichenbaum,2014b).Table 1. PCA variance for the number of latentcomponents.Latent Components PCA variance (%)

3 2.935 4.310 7.3215 10.0320 12.525 14.850 24.99100 41.88200 63.9

There are some previous works on trendingor hot topics in science: (Griffiths and Steyvers,2004) used Latent Dirichlet Allocation (LDA) toanalyze the abstracts from Proceedings on the Na-tional Academy of Sciences (PNAS) from 1991 to2001. (Wei et al., 2013) performed a statisticalanalysis to find if scientists follow hot topics ontheir investigations, they used published papersfrom the American Physical Society (APS) PhysicalReview journals beginning in 1976 and endingin 2009. (Kang and Lin, 2018) used non-smoothnon-negative matrix factorization (snNMF) to ex-tract themore prominent topics from a dataset ofkeywords from scientific articles related to "Ma-chine Learning" from 2014 to 2016 in arXiv.orgstat.ML, the similarity of this work with the Tensor

Memory Hypothesis belongs to the use of matrix decomposition to reduce the rank of the matrix.(Alshareef et al., 2018) indexes based on cosine similarity to estimate a score that represents theanticipation of a prospective relationship between authors. They used two subsets of the IEEE digitallibrary containing the keywords “database” and “multimedia”.ResultsThe quantity of latent components is not associated with a specific statistical measure of data.However, to have an approach, table 1 presents the correspondent percentage of variance if thesame number of PCA components were employed.

19


Table 2. Most probable words for the query with an entity type.Entity Type

Latent Components Authors Articles Words3 neuromodulation neuromodulation neuromodulation5 stimulus, presented stimulus, presented stimulus, technique10 presented presented presented15 sleep, memory sleep sleep20 stimulus, memory stimulus, cued stimulus, cued25 memory, sws memory, spatial, sws memory, sws50 sleep, stimulus sleep, stimulus sleep, stimulus100 assr, memory assr, memory assr, memory200 wireless, monitoring sleep, slow sleep, slow

Table 3. Most probable word with NMF decomposition.Entity Type

Latent Components Authors Articles Words3 slow, sleep, auditory stimulation, sleep sleep, memory5 spindles, auditory, sleep sleep sleep10 sleep, stimulation sleep, stimulation sleep, memory15 sleep, memory brain, consolidation sleep, memory20 sleep, memory oscillations, sleep sleep, memory25 sleep, stimulation activity, memory sleep, memory50 sleep, memory oscillations, humans sleep, memory100 sleep, role reactivation, slow-wave sleep, memory200 sleep, slow sleep, brain sleep, memory

The words with more relations in the complete tensor, before decomposition, are sleep,memory, stimulation, slow, brain, consolidation, auditory, spindles, reactivation, andactivity. Table 2 is populated using a selection strategy of most frequently word from queries ofthe type

wordi = argmaxo{P (s, o, t)}, (1)where s is each author, paper title or word in the database, o a word, t a year and, i is the index of aentity .The most probable words, from the same queries, using more latent components are more

than using a few latent variables. For example, there are 21 different words from query resultsusing 200 latent components. In the other hand for few latent components, the results of queriesare only the words shown in table 2.Table 3 is populated using the of NMF decomposition in the collapsed on time matrix, adding

the weights of each year. The more frequently words are selected from which are maximum foreach topic or k-row in the matrix H of the decompositions. The same processing using nsNMFdecomposition results with the words sleep and memory as the most probable for all the cases.The analysis of relationships between entities needs a metric of distance. Each entity is rep-

resented by latent vectors, then one metric selection could be the Euclidean distance but giventhis particular type of data, content from documents, the usual metric employed is the cosinesimilarity. However, the use of distances on the original data space demand high computationalcosts, the use of a reduced space alleviates the computational cost of calculating distances butrequires a previous high cost of space transformation. Figure 1 is an example of the Euclidean

20


distance and cosine similarity that was extracted from the R tensor of the RESCAL factorization. Thedifference between the years of sources and the years of only cited papers is most evident withless latent components. Moreover, the similarity is greater, then lesser Euclidean distance, betweenthe entities of the previous years.DiscussionThere are scientific papers meta-data databases or it is possible to extract article’s meta-data froma specific journal or publisher. But in practice, it is usual to have few references from a previoussearch and they are from different journals or publishers, then to extract the meta-data I usedthe JATS format 1, a semantic web standard format for scientific papers popularized by NationalCenter for Biotechnology Information (NCBI). A most popular format is the Resource DescriptionFramework (RDF) and various scientific publishers are adopting this one.The analysis of the statistical features of the tensor without any other process could give in-

formation of the most related entities, as the most cited author, most cited article or most usedword in each slice of time. However, employing a tensor decomposition technique allows theuse of a latent components space, where more information could be extracted given that therelationships are expressed in fewer variables, thus, clustering some properties of data. This workis an example of how from a small sample of documents with a known relationship betweenthem, the topic was already known, some words that are not the most frequent could be ex-tracted and provide a new perspective of the topics covered on the documents. The figure 1 isan example of extracted information that is not easy to visualize in the original space of the data.

0 10

years

0

5

10

years

3

0 10

years

0

5

10

25

0 10

years

0

5

10

200

0.6

0.7

0.8

0.9

1.0

� 0.4� 0.20.00.20.40.60.81.0

0.0

0.2

0.4

0.6

0.8

1.0

0 10

years

0

5

10

years

3

0 10

years

0

5

10

25

0 10

years

0

5

10

200

0.000.020.040.060.080.100.120.14

0.00.10.20.30.40.50.60.7

0.00.10.20.30.40.50.6

A

B

Figure 1. Distance metrics on the latent space. A.Cosine similarity between years with 3, 25, and 200latent components. B. Euclidean distance betweenyears with 3, 25, and 200 latent components.

The comparison of different tensor decompo-sitions and the search for the optimum numberof latent components is work to be done to takeadvantage of relational data, that due to semanticweb technologies is not restricted only to formalscientific documents and it is available for varioustype of data. Also, the proposal of (Tresp et al.,2017) of considering the knowledge graphs assemantic and episodic memories allows havinga framework that links computational memorywith the biological one. Its capacities and defectsneed to be explored. Curiously, the etymology of“topic” comes from the Greek topos or place, thatas memory is other of the known hippocampuscognitive functions.Finally, from the results obtained is evident

that sleep and memory are the most relevant words of the selected papers, these words and sloware the few words that are the result from queries too with RESCAL decomposition. The nsNMFdecomposition gives for any number of components the same words, then it is more robust to thechange in the number of components.Methods and MaterialsData extractionThe meta-data from 11 articles from different publishers (Table 4) related to “Stimulation duringNREM sleep” in PDF files was obtained using the software CERMINE (Tkaczyk et al., 2015) andstored in JATS format. After, with a Python script, the own title, authors and abstract were extractedand also the title and authors of references inside the time range 2008-2018. Later, the titles and

1https://jats.nlm.nih.gov/

21


Table 4. Most probable word with NMF decomposition.Year Authors Title2016 Batterink et. al Phase of Spontaneous Slow Oscillations during Sleep Influences Memory-Related Processing of Auditory Cues2016 Weigenand et. al Timing matters: open-loop stimulation does not improve overnight consolidation of word pairs in humans2017 Besedovsky et. al Auditory closed-loop stimulation of EEG slow oscillations strengthens sleep and signs of its immune-supportive function2017 Lafon et. al Low frequency transcranial electrical stimulation does not entrain sleep rhythms measured by human intracranial recordings2017 Leminen et. al Enhanced Memory Consolidation Via Automatic Sound Stimulation During Non-REM Sleep2017 Lustenberger et. al High-density EEG characterization of brain responses to auditory rhythmic stimuli during wakefulness and NREM sleep2017 Oyarzun et. al Targeted Memory Reactivation during Sleep Adaptively Promotes the Strengthening or Weakening of Overlapping Memories2017 Kinzing et. al Odor cueing during slow-wave sleep benefits memory independently of low cholinergic tone2018 Ashton et. al No effect of targeted memory reactivation during slow-wave sleep on emotional recognition memory2018 Debellemaniere et. al Performance of an Ambulatory Dry-EEG Device for Auditory Closed-Loop Stimulation of Sleep Slow Oscillations in the Home Environment2018 Ezzyat et. al Closed-loop stimulation of temporal cortex rescues functional networks and improves memory

abstracts were tokenized and semantically tagged, using nltk library, to extract the adjectives andnouns that are considered the principal terms of the articles. For de-duplicating authors, all namesare formatted to “(Last name) (First name initial.) (Middle name initial.)” For de-duplication of titlesand words, all words were transformed to lowercase and special characters were eliminated.For each year, a zeros square matrix Xk ∈ ℝ(na+nt+nw)×(na+nt+nw) was populated with weighted and

directed values using the next rules only in the k-year correspondent to relations:Autℎori co-wrote with Autℎorj xkij+ = 2

Autℎori cited Autℎorj xki,j+ = 1Autℎori wrote Articlej xki,j+ = 2Autℎori cited Articlej xki,j+ = 1Autℎori wroteW ordj xki,j+ = 1Articlei cited Articlej xki,j+ = 1

Articlei containedW ordj xki,j+ = 1W ordi is in the same document ofW ordj xki,j+ = 1

This approach for expressing the relations simplifies the tensor representation because thedimension correspondent to predicate are intrinsic on the weighted values and allows the use ofRESCAL factorization. (Ma et al., 2018) explain other tensor decomposition methods that could beused to get the latent components.Cosine similarityThe cosine similarity is an adequate distance metric for vectors where the magnitude is dependenton the size of the sample, as the frequency of words in a document.

s =√x ⋅ y

‖x‖2‖y‖2 (2)Tensor memory hypothesisA fourth order tensor could be decomposed as

X ≈ G ×1 As ×2 Ap ×3 Ao ×4 At (3)The probability of the existence of the relationship between the entities of a quad is given by

P ((s, p, o, t)) = sig(�s,p,o,t), (4)Where

sig(x) = 11 + e−x

(5)

22


�s,p,o,t = f e(aes , aep , aeo , aet ), (6)

f e(aes , aep , aeo , aet ) =r∑

r1=1

r∑r2=1

r∑r3=1

r∑r4=1

aes ,r1aep ,r2aeo ,r3aet ,r4ge(r1, r2, r3, r4). (7)

The analysis of tensors, as for matrices, is possible to perform using a reduced form obtained byfactorization. One popular factorization method of tensors is the Tucker representation, however,there are other matrices and tensor decomposition algorithms. Here, I used RESCAL and theconstruction of the tensor with weighted values allow to omit the predicate dimension, then thecharacteristic function becomes

f e(aes , aeo , aet ) =r∑

r1=1

r∑r2=1

r∑r3=1

aes ,r1aeo ,r2aet ,r3ge(r1, r2, r3). (8)

RESCALThis tensor decomposition was proposed by (Nickel, 2013). The decomposed tensor needs to havetwo dimensions of the same size, i.e., X ∈ ℝn×n×m and the results are a matrix A ∈ ℝn×r and a tensorR ∈ ℝr×r×m.

X ≈ R ×1 A ×2 A,Xk = ArkAT .

(9)The algorithm is an alternating least squares (ALS) procedure where the outputs are updated with:

A←

(m∑k=1

XkARTk +X

Tk ARk

)(m∑k=1

RkATART

k + RTkA

TARk + �AI

)−1

, (10)

Rk ← V(P ∗ UTXkU

)V T , (11)

Where Rk is a slice of the tensor R and for optimization a singular value decomposition of matrix Ais employed. P is the matrix such that diag(vec(P )) = S, which can be constructed by rearrangingthe diagonal entries of S via the inverse vectorization operator vec−1r (⋅)

A = UΣV T , (12)Then, for regularization, the Kronecker product of the diagonal matrix is employed.

S = Σ⊗ Σ (13)

Sii =Sii

S2ii + �R(14)

Non-negative Matrix Factorization (NMF)This matrix factorization method finds two matricesW ∈ ℝn×r andH ∈ ℝr×m which multiplicationminimizes the Froebenius norm with the original matrix X ∈ ℝn×m.

X ≈ WH, (15)The updates using the algorithm proposed by (Lee and Seung, 2001) are:

W ← W XHT

WHHT + �, (16)

H ← H W TXW TWH + �

. (17)

23


Non-smooth Non-negative Matrix Factorization (nsNMF)This decomposition is a modification of NMF proposed by (Kang and Lin, 2018).

X ≈ WSH, (18)Where

S = (1 − �)I + �k11T , (19)

D =

(m∑j=1

Hi,j

)I, (20)

And usingW = WℎD

−1S−1, (21)Finally, the matrix decomposition could be expressed as

X ≈ WD−1S−1SDH. (22)FundingThis work was supported by Beca Doctorado Nacional Conicyt, Folio No 21180640.ReferencesAlshareef AM, Alhamid MF, El Saddik A. Recommending Scientific Collaboration Based on Topical, Authors andVenues Similarities. 2018 IEEE International Conference on Information Reuse and Integration (IRI). 2018; p.55–61. https://ieeexplore.ieee.org/document/8424687/, doi: 10.1109/IRI.2018.00016.Bizer C, Heath T, Berners-Lee T. Linked data: The story so far. In: Semantic services, interoperability and webapplications: emerging concepts IGI Global; 2011.p. 205–227.

Eichenbaum H. Memory on time. Trends in Cognitive Sciences. 2014; 17(2):81–88. doi:10.1016/j.tics.2012.12.007.Memory.Eichenbaum H. Time cells in the hippocampus: A new dimension for mapping memories. Nature ReviewsNeuroscience. 2014; 15(11):732–744. doi: 10.1038/nrn3827.Griffiths TL, Steyvers M. Finding scientific topics. Proceedings of the National academy of Sciences. 2004;101(suppl 1):5228–5235.Kang Y, Lin Kp. Topic Diffusion Discovery based on Sparseness-constrained Non-negative Matrix Factorization. .2018; doi: 10.1109/IRI.2018.00021.Lee D, Seung H. Algorithms for non-negative matrix factorization. Advances in neural information processing sys-tems. 2001; (1):556–562. http://papers.nips.cc/paper/1861-algorithms-for-non-negative-matrix-factorization,doi: 10.1109/IJCNN.2008.4634046.Ma Y, Tresp V, Daxberger E. Embedding Models for Episodic Memory. . 2018 jun; http://arxiv.org/abs/1807.00228.Nickel M. Tensor Factorization for Relational Learning. . 2013; p. 161. http://nbn-resolving.de/urn:nbn:de:bvb:19-160568.Tkaczyk D, Szostek P, Fedoryszak M, Dendek PJ, Bolikowski Ł. CERMINE: automatic extraction of structuredmetadata from scientific literature. International Journal on Document Analysis and Recognition (IJDAR). 2015;18(4):317–335.Tresp V, Ma Y. The Tensor Memory Hypothesis. . 2017; http://arxiv.org/abs/1708.02918.Tresp V, Ma Y, Baier S, Yang Y. Embedding learning for declarative memories. Lecture Notes in ComputerScience (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 2017;10249 LNCS:202–216. doi: 10.1007/978-3-319-58068-5_13.Wei T, Li M, Wu C, Yan XY, Fan Y, Di Z, Wu J. Do scientists trace hot topics? Scientific Reports. 2013; 3:3–7. doi:10.1038/srep02207.Zeng A, Shen Z, Zhou J, Wu J, Fan Y, Wang Y, Stanley HE. The science of science: From the perspective ofcomplex systems. Physics Reports. 2017; 714-715:1–73. https://doi.org/10.1016/j.physrep.2017.10.001, doi:10.1016/j.physrep.2017.10.001.

24


AIS: Artificial Intelligent Soccer1

Ricardo Alfaro1*, Maximiliano Aubel1*, Pablo Yañez1*, Pablo Reyes1, Tomás2Rodenas1, Nicolás Hernandez1, Felipe Pinto1, Sung Hee Kim1, Tania Barrera1,3Daniel Torres1, Ignacio Vicencio1, Jorge Alvarez1, Felipe Osorio1, Sebastian4Castillo15

*For correspondence:[email protected];[email protected]; [email protected]

1Innovación y Robótica Estudiantil, Universidad Técnica Federico Santa María6

7

Abstract This paper describes the current development status of our SSL team, AIS. Throughout8this document, we present the design and implementation we have got so far, showing the9electrical, mechanical and software topics involved in our work, which were designed according to10satisfy the RoboCup rules. In addition of giving details on the current development, we present11some insights on the main challenges that have been identified and tackled in recent years with the12consolidation of this team, in order to foster other forming groups around the globe.13

14

Introduction15Innovación y Robótica Estudiantil, which is the affiliation of all members on this team, has been16founded in 2001 and corresponds to a self-organized group of students from several faculties, such17as Electronics, Informatics and Mechanical Engineering departments at the University (UTFSM). This18RoboCup team belongs to one of several projects from this aggroupation and is conformed by19students with different specialization areas such as computer science, control, and automation, or20power electronics, but also on a multidisciplinary approach including students from mechanical21engineering, as well as industrial engineering students.22This SSL team follows the nature of the host students initiative, starting from its multidisci-23

plinary constitution, the self-organization and motivation with professor advises when required but24managed independently from any professor funding project, and trascendence over generations25renewing its members with a constantly growing development and enhancement, and making26both research and development works, like [1] where a previous generation of the team applied27reinforcement learning on the goalkeeper task.28This document describes our design and the implementation we have got so far, showing all the29

work made in the different areas involved in this category.30In particular, we describe the mechanical design, electronics design for different devices and31

algorithms implementation for the (robotics) team coordination, also including the expected imple-32mentation we are planning to reach by the time of the competition.33

Mechanical Design34The material selected for the chassis structure is chosen by means of priorizing the collision35resistance, so an aluminum base is used, while supports for the wheel motors also consists on36four aluminum blocks and a second floor of polymethyl-methacrylate (PMMA) which stands for37supporting the battery. Then, a third floor is designed also of PMMA, with the aim of supporting the38PCB and also isolating the battery and PCB. Finally, the cover is 3D-printed on ABS material.39-Height: 150 mm.40-Diameter: 180 mm.41-Maximum coverage of the ball: 18%.42

25


Figure 1. Omnidirectional wheel back view Figure 2. Robot assembled

Drive System43Mechanical locomotion of robots is based on 4 omnidirectional wheels, which are currently 3D-44printed in PLA. Each wheel is designed with 55 mm of diameter and 15 sub-wheels of 13.5 mm of45diameter, so the robot can move in all directions. Also, each one of the 55 mm diameter wheel has46a set of 15 mini metal V grove guide pulley rail ball bearing wheels. Each wheel is driven by a Maxon47EC45 30 Watts motor [7] and an L6235 driver for 3-phase brush-less DC motor [6], which enables us48to program a velocity control for each motor, ensuring that the robot moves to our desired setpoint49speed.50

Hardware51Each robot is controlled by a PIC MX440F256H using Pinguino Development board. This model was52chosen because of its simplicity, versatility and peripherals features. It has shown an acceptable53performance, letting us accomplish communication, movement, and playing skills. The peripherals54also replace a lot of external electronics needed to control the motors and dribbler.55

Peripherals56ADC conversion57The ease of implementation of this kind of modules allows us to control wheel speed and orientation58through an L6235 driver using DAC conversion. Additionally using ADC conversion we can measure59the wheel speed, allowing us to implement a PID control on velocities for each wheel.60

I2C61As mentioned before, we use an L6235 driver, which communicates with the PIC through its I2C62module.63

UART64The UART module allows us to develop serial communication between the PIC and our APC22065RF module, which will send and receive data from the centralized decision maker placed on the66computer. Additionally, we use an FTDI connected to our UART communication module to watch67data of interest.68

GPIO69The general purpose Input/Output pins let us program easily, in general, any other significant70settings, i.e. set the wheel break and direction pins or activate the kick routine.71

Kicker72The circuit that is shown in Figure 3 is used for the kicking system. This consists of a chip charger73controller with regulation which is a controller of flyback of high voltage, raising the voltage from7424 [V] to 100 [V] on a capacitor of 2400uF and, therefore, storing an energy of 244 [J]. The time of75charge of the capacitor takes up to 3 seconds to reach the voltage setpoint and can be regulated to76kick with different intensities.77

26


GND

GND

GND

GND

GND

APF30212A

Solenoid

Relay

SignalCHARGE

8

CLAMP9

VCC13

DONE7

FAULT6

UVL012

OVL013

UVL024

OVL025

GN

D1

7*3

RB

G1

6

RV

_T

RA

NS

1

RDCM20

RV_OUT18

HVGATE15

LVGATE14

CSP12

CSN11

FB10

4

12

3

11

2

10

1

9

7

6

T1

M1

R10

R9

C4C3C2C1

R1

R3

R2

R5

R7

R6

R4

R8

C6

C5D1 D2

R11

K1

21

COM NO

VCC

VCC

VCC

VCC

V_TRANS

V_TRANS

LT3751

FLUX SHLD

+

Figure 3. Kicker circuit

Dribbler78According to RoboCup rules, the robot is allowed to cover up to 20% of the ball. Experimentally, it79has been proved that it is easier to catch the ball when the dribbler has a slight curve to center the80ball on its own. So, this design involves two diameters, D1 and D2, and based on this information,81maximum height possible is calculated obtaining the following expression:82

H =√

14(D2(2d +D2) +D1(4pd − 2d −D1) + 4pd2(1 − p)) + d

2, (1)

where d and p corresponds to the ball diameter and maximum coverage of the ball, whose83relation is illustrated in Figure 4.84Our team uses the engine MAXON EC 16, BRUSHLESS, 30 WATT, SENSORLESS, handled by an85

ESC LettleBee opto 6s.86Both the engine and roller join using a gear system with ratio 1:1, configuration that let us drive87

Figure 4. Relation between variables involved indribbler design

Figure 5. View of dribbler assembly

27


Figure 6. Robot wheel speed control system

and automatically center the ball with a 3D-printed support structure.88

Communication89For communication, we use an RF module consisting of an APC220 which is a low-cost NRF Athena90that integrates an embedded high-speed microprocessor and high-performance IC that creates91a transparent UART/TTL interface. It is a 430 MHz system capable of transmitting up to 100092meters. We send every single robot data through a common channel as hexadecimal packages93in order to achieve better transmission bytes. Each robot receives and decodes the data in a pic94microcontroller.95

Kinematic model and wheel speed control96In order to maintain the expected velocity and position, we applied PID control [2] on every wheel97once the setpoint speed is calculated for every robot. To accomplish this task, our control system98sends a velocity vector V = [vx, vy, v�]T to each robot, multiplying then its kinematic model matrix99W , defined by the geometry of the robots, obtaining100

u = V TW = [u1, u2, u3, u4]T = �r,

where u is the wheel velocity vector, which divided by r (radius of the wheel), it is possible to101obtain the angular velocities of the wheels, �. This is the reference variable to the control speed102system shown in Figure 6, and by obtaining direct measure from the hall sensors of the motor we103can obtain the input error variable to the PID. Then, the controller generates a PWM as output in104order to set the wheel speed. It is important to note that � is treated as an absolute value because105the direction is set by enabling or disabling certain pins on the driver controller.106

Software107Diagram depicted in Figure 7 shows a general overview of the system, where we implement a108high-level AI decision making in order to decide which is the best action to take from a set of high109level preprogrammed plays based on the game state that comes just from the vision receiver.110Then we have a low-level path planning algorithm to choose the best path in order to execute the111play avoiding obstacles. This is implemented on the desktop computer in charge of making the112centralized decisions for every robot.113

High level AI114The higher order AI level computes at each processing cycle the best actions to be performed for115each robot. This action is chosen by selecting a game-play from a pre-defined static pool. This116fundamental part of the system’s architecture is shown in Figure 8, introducing four identifiable117processes. First, there is a SceneRater which analyze and encapsulates all the relevant information118from the game field for choosing a game-play. Then, that information is used for actually selecting119the specific game-play through the block named as PlayChooser, weighting each detected event for120deciding whether an attacking strategy or a defensive one should be used, and which one in detail121

28


Figure 7. General diagram of the system

Set of pre-defined plays

SceneRater

Game-plays

PlayChooser

RoleAssigner

Executer

Low levelPath-planning

Events vector(field data)

Role vector for robot assignment

Role vectorsfor action performing New

play?

yes

no

High-level AI

Vision/Referee data

Robot control system data

Figure 8. High-Level Artificial Intelligence Architecture Diagram

must be performed. Once a game-play is chosen, then a RoleAssigner block is in charge of coherently122distribute the roles associated to that game-play, as well as selecting which robots should assume123each position. Finally, each position must be run by the robots, a process managed through time by124an Executer block.125Then, as shown in Figure 8 which depicts the diagram of the algorithm implemented as the top126

hierarchy intelligence architecture, the processing cycle starts with the receiving of new data either127from the vision system or the referee. As illustrated, four blocks are implemented and processed128in order: SceneRater, PlayChooser, RoleAssigner, which is executed just if the current play has129changed, and the Executer block.130Specifically, the SceneRater evaluates conditions as which team has the ball, whether a team has131

or not high and middle chances of making an annotation, the partial position of the ball in the field,132which team is closer to the ball, among others relevant features.133The PlayChooser, after receiving the vector of detected events, evaluates all pre-defined game-134

plays, each one of them rating differently the events detected. Offensive plays rate with higher135values the detection of the ball in the enemy team area, and even more if the ball is close to the136enemy goal area. Coherently, defensive plays strongly rate when the enemy team has the ball and137even more if they have an opportunity for shooting to the team goal area.138Each game-play is described by a set of roles, one role for each agent, introduced in a priority139

29


order in case of using fewer robots than the maximum allowed. Each role considers a set of actions140to be developed by the agent, as moving, receiving or giving a pass or shooting to the goal area.141To do so, the architecture includes the Executer block, which is in charge of evaluating if either142

an action has finished or not, managing the changing of steps and computing the next step of an143action execution for each robot.144

In order to simulate the robotic team coordination, and test different multi-agent algorithms,145we make use of GrSim [5], software that has been very helpful to test game strategies.146

Low-level path planning147Under the high-level plays, we run a path planning algorithm to find the best way of executing these148tasks. We have tested different methods looking for a suitable algorithm which gives good results149at the moment of avoiding obstacles.150The first method tested was Potential Field algorithms [4]. This proposes a potential field151

representation for obstacles and target, using sources for the prior and sink for the latter. In this152way, vector trajectories are generated avoiding obstacles and leading the agent to the target, as we153let a ball fall down. A disadvantage of this method is that we could obtain local minimums without154reaching the target.155The best method tested was Rapidly-Exploring Random Trees (RRTs)which consists on expanding156

a tree on the target zone, avoiding to add nodes that could produce collisions with targets. The157added points to the tree are randomly chosen with probability p in a straight line to the target,158and with probability (1 − p) selecting a random point on the space, making more exploration and159avoiding to get stuck on a different location to the target.160For improving its performance, we have implemented and tested some of the algorithms based161

on RRT, way-points, smoothing and some extensions like RRT* presented on 2011 [3].162

Conclusions163Although we tackled several challenges prior to our participation at the Robot Soccer World Cup1642018, such as discarding low-level path planning algorithms due to local optimums according165to some remarks described throughout this document, and despite efforts about testing game166strategies on a simulated environment, a newcomer team into the league should consider several167other challenges that should be faced at the time of participation, such as:168

• Full integration with referee box: it is not enough to communicate this software with the169central unit for the team decision-making system, given that all rule cases should be covered.170

• Number of robots: although each team is allowed to participate with a smaller number of171robots, it is important to maximize the number of available prototypes in case of structural172damage (chassis should be prepared to receive very strong shoots).173

• Number of team members: in case of belonging to a self-organized group of students (such174as our case), budget and funding search for the tournament participation should include at175the very least six members, given that captain is constantly being called for meetings within176the contest, and two other members have to serve as referees for other matches.177

• Precision and time delays: although splitting of teams into Division A and Division B lower the178game complexity in order to face similar teams in terms of experience and tasks achievement,179there are very experimented teams which have mastered the motion control of robots, so it is180important to be prepared to chase and face the ball in the most efficient possible manner.181

Next challenges for this team in order to qualify for the upcoming world cup in addition to fundrais-182ing for participation, include fine-tuning of robots which were already capable of playing matches183and even winning one of them, following with the appropriate prototypes replication.184

30


References[1] Ahumada et al. “Accelerating Q-Learning through Kalman Filter Estimations Applied in a

RoboCup SSL Simulation”. In: Robotics Symposium and Competition (LARS/LARC), 2013 LatinAmerican. IEEE. 2013, pp. 112–117.

[2] Åström et al. PID controllers: theory, design, and tuning. Vol. 2. Instrument society of AmericaResearch Triangle Park, NC, 1995.

[3] Bry et al. “Rapidly-exploring random belief trees for motion planning under uncertainty”. In:Robotics and Automation (ICRA), 2011 IEEE International Conference on. IEEE. 2011, pp. 723–730.

[4] Ge et al. “New potential functions for mobile robot path planning”. In: IEEE Transactions onrobotics and automation 16.5 (2000), pp. 615–620.

[5] Monajjemi et al. “grsim–robocup small size robot soccer simulator”. In: Robot Soccer World Cup.Springer. 2011, pp. 450–460.

[6] Vincenzo Marano. “L6235 three phase brushless dc motor driver”. In: Application Note, ST(2003).

[7] AG Maxon Motor. EC-Powermax 30 Catalogue Information. 2008.

31


A Theoretical Strategy for EnhancingLearning Through MetacognitivePractices on a ConstructivistMethodologyCamila Angel1* and Cristóbal J. Nettle2

*For correspondence:[email protected] (CA) 1Departamento de Ingeniería en Diseño, Universidad Técnica Federico Santa María;

2Centro de Innovación y Robótica

Abstract While there has been a shift to an education based in the development ofcompetences, the current approaches following a constructivist educative model (described asideal for competence acquisition) still lack a clear conjunction with metacognitive practices. Theintroduction of metacognition improves self-cognitive processes that are fundamental for assuringaware and long-standing learning, among other essential characteristics for later applications onsocial and professional contexts. Here, following a state-of-the-art evaluation of Latin Americaneducation, we present a conceptual model to form competent people which integrates key factorsfrom metacognitive development linked to a constructivist approach. Theoretically, the proposedmodel improves competence development through self-knowledge as what students are capableof, and what is the best attitude to accomplish their aspirations. Final conclusions depict steps forpractical applications, describing also how stronger efforts have to get done in a social andinstitutional scale for actually reaching a solid competence development.

IntroductionCurrent societies are strongly affected by technological and scientific advances in a dramatic way,with demands of specialized knowledge but, at the same time, a very practical one, requiring fastand efficient responses without knowing to much, given that information is reachable for everybody.This implicates that people must be flexible, self-critic and open to change (Rejas et al., 2013).

In order to form competent persons, universities had have to reformulate their curriculum and,along with it, their teaching models and methodologies. In consequence, an important portionof the occidental educational entities has adopted a formation based on the development ofcompetences. Specifically, in Latin America there is a socio-formative approach, where there is afocus in forming an ethic compromise with the self, with society and the environment that surroundus (Tobón et al., 2010).Competences are defined as a set of knowledge, skills, values, and attitudes, either specific

or transversal (or generic), that a graduate has to acquire in order to fully satisfy the social andprofessional demands. This set is put into play when a problem is confronted, and its especiallyinteresting when it is an uncertain, complex problem. The specific competences (or technicalones) are such that are related to a knowledge about a particular area, and must be developedthrough learning from a specific discipline. On the other side, generic competences (also knownas soft or transversal) are the ones that tackle the formation of a person as a socially inserted

32


individual, and that makes reference to personal attributes with cognitive, social, attitudinal orvaluable characteristics, which enrich professional behavior, and increases the value of the specificcompetences (Bellocchio Albornoz, 2009).Several authors have proposed that for developing competences is necessary to adopt a con-

structivist educative model, based in the formative process, aiming to promote a meaningfullearning to ensure this competence acquisition (see for example the Tuning Latinoamérica project1),leaving behind the approaches based on behaviorism.However, there is a disjointed relationship between the developing of competences and the

constructivist educative model, which has been considered as ideal for competence acquisition.The bridge between them can be based on a metacognitive development, a fundamental elementfor ensuring the appearance of ideal competences. Metacognition refers to the knowledge andregulation of self-cognitive processes, i.e., the regulation of how to acquire knowledge, how toperform, how to react to a specific situation, when to perform certain actions, etc.Then, here we present a search and later description of the factors that play a role in metacogni-

tive development as an integral part of the teaching-learning process, with the goal of achievingin students a permanent reflective and self-evaluative state. This, for introducing in them anawareness of what do they learn, and how to transfer that knowledge to other situations.This article proceeds as follows: first, we describe the bases for meaningful learning, and how

constructivist methodologies and metacognitive considerations can enhance learning. Then, adescription of the proposed strategy is presented, including a detailed systematical view to facilitateits application. Finally, conclusions and possible extensions of this work are depicted.Teaching for learningIn the seek of meaningful learningNowadays, the main focus on pedagogical actions is to develop in the student the capacities forproducing meaningful learning by their-self, in a wide range of situations and circumstances, i.e.,learning how to learn (Coll, 1988).David Ausubel, who introduced the concept of meaningful learning, proposed a theoretical

explanation of the learning process from a cognitive point of view, while taking into account motiva-tional and affective factors. According to him, learning involves the organization and incorporationof cognitive information in a person, which happens when new information received by a subjectis related (i.e., interact) with relevant ideas that support it, which are already part of the cognitivestructure of the subject. This pre-existent structure is called integrator concept. In this sense, Ausubelet al. (1980) understands the storage of information as a highly organized process inside the humanbrain, where more specific elements of knowledge are anchored to more general and inclusiveknowledge, a process which is called assimilation. Then, the cognitive structure is a hierarchicalstructure of concepts, which is a product of the experience of the subject.Mechanical learning differentiates with meaningful learning because it takes into account the

incorporation of new information without establishing any relationship with previous concepts orpropositions, being stores in an arbitrary way.

In order to achieve meaningful learning, several conditions have to be met: the new informationhas to be related in a non-arbitrary and substantial fashion, which depends on other factors suchas disposition, motivation, and attitude from the learner subject, and the nature of the matter beingstudied, or its contents (Díaz and Hernández, 1999). Moreover, the promotion of an auto-criticattitude on the student is necessary for ensuring meaningful learning, to acquire precise andintegrated meanings across learning (Arancibia, 2008).The predisposition of learning is also essential (Bruner et al., 1966; Gagné, 1985), and it can

be achieved by mechanisms for triggering, sustaining and directing learning. This, by setting asituation with a certain degree of uncertainty (trigger), showing the benefits of exploring alternatives

1Information about the Tuning Latinoamérica project can be found at http://www.tuningal.org/.

33


higher than the associated risks (sustain), and leading this discovering according to the ending goaland relevance of knowledge (direction). Bruner et al. (1966) and Gagné (1985) described also theimportance of evoking what has been learned in situations of different domains than the originalone. This reuse of the learned subject has to incorporate a feedback for measuring how well andprecise was the learning.The constructivist model of learning, developed at the ends of the XX century, and currently

predominant in school environments, takes into account the described concepts for inducingmeaningful learning.Contructivism: foundations and fundamental characteristicsConstructivismwas build as a need for understanding and integrating the complexmultidimensionalprocesses involved in learning. It aims to solve problems such as: identification and attentionto the diversity of interests, needs, and motivations of the students in relation with the teaching-learning process; a re-description of the scholar curriculum for learning to learn; a recognition ofdifferent methods and types of learning, considering intellectual, affective and social dimensions;the relevance of promoting teacher-student and between students interactions; the role of theteacher and the student; among others (Díaz and Hernández, 1999).By definition, constructivism is an approach that integrates the subject with cognitive, social and

affective variables, factors that build the self day by day. Following this perspective, knowledge isnot directly ground truth, but instead, a construction of the self that is built through its relation withthe environment (Carretero, 2000).The core characteristics of this educative model are that: it defines the learning process as an

active situation, centered in the student; it promotes the interaction among persons; the teacheradopts a decentralized but active role; it promotes meaningful learning in all its shapes (self-learning, learning by discovery and reception); and, it establish that knowledge is organized in abase of conceptual structures that have to me remove or reshaped in function of later cognitiverequirements (Bellocchio Albornoz, 2009).Even while there is no constructivism without metacognition, the last one is often passed by

(Tobón et al., 2010). Both, the teacher and the students, have to be conscious of the tools that arein use during the process of acquiring and constructing knowledge. This improves self-learning anddidactic interactions (Tobón et al., 2010).Metacognition as a tool for enhancing learningMetacognition is the knowledge of self-cognitive processes (e.g., being able to determinate howcapable is one for memorizing or for how difficult is a task) and the regulation of them (e.g., knowinghow many times one have to read a sentence in order to remember it) (Flavell et al., 1970; Flavell,2000). The use of metacognition impact in the development of diverse methodologies, techniquesand didactic strategies in the process of learning. In any case, one uses at least an intuitive approachfor it, which allows us to have an idea of the necessary means to achieve learning of a specificmatter (Bellocchio Albornoz, 2009).Metacognition is composed of three fundamental components, which allow us to act and control

our self-cognitive activity: knowledge, experience and skills (Flavell, 2000). The metacognitiveknowledge is, at the same time, separated as declarative knowledge (knowledge about the generalskills that we possess), procedural knowledge (about how effective we are on solving problems)and a conditional or attitudinal knowledge (about when to apply specific strategies) (Sperling et al.,2004; Mateos, 2001). The use of metacognitive skills and strategies are useful and needed foracquisition, use, and control of our knowledge and other cognitive skills, which includes, e.g., skillsfor planning and regulating the effective use of our own cognitive resources (Brown and Smiley,1977). Then, our metacognitive skills allow us to direct, monitor, evaluate and modify our learningand thought. Also, metacognitive experiences are conscious experiences that are focused somehowon our own cognitive performance. These experiences are thoughts, sensations or feelings that can

34


be interpreted consciously and that accompanies the cognitive activity (Mateos, 2001). However,the line between metacognitive knowledge and metacognitive experiences is not clear (Arancibia,2008).Then, the process of metacognition makes explicit the process of knowledge construction,

implicit on learning, in order to achieve the desired performance and to know when and how thatknowledge should be applied, in a specific context. Also, metacognition must be used in any didacticstrategy applied in an educative context, given that a poor metacognitive formation in teachersand students blocks the success of any constructivist methodology. This idea is supported by themain focus of education, learn to learn, a complex function that is the base of all the other ways oflearning: to be, to do and to act (Bellocchio Albornoz, 2009).

It is important to note that metacognition is not necessarily a conscious act, but it has to beexplicit at the beginning for adopting its practices, which eventually becomes an automatizedbehavior, a habit, keeping its importance on performance (Martí, 1995).Also, an important clarification is that the cognitive and metacognitive processes cannot be seen

as separated entities, but as complementary for learning processes, taking into account where is anactivity being developed and its features, constraints, and specificities. In this dynamic process ofexternal regulation and self-regulation is that appears mechanisms of interiorization (abstractionand awareness, according to Piaget) and exteriorization (explicitation and accessibility to knowledge,according to Vygotsky) that can be supported by cultural means or devices (computers, writing, etc.)that are used as facilitators for regulating the cognitive activity (Martí, 1992).Integrating metacognitive practices in a contructivistic environmentAs exposed, it is highly relevant to take into consideration the practice of metacognitive activitiesduring learning processes and, therefore, relevant for the development of school strategies forenhancing learning and later knowledge applications, at the student’s personal and professionallife.There are several specificmetacognitive strategies (Tobón, 2013) as, e.g., themeta-conceptualization,

which considers knowing how much do we know about a specific concept, or theory, for under-standing a wider, more complex phenomenon, helping to decide what actions or precautions hasto be taken for accomplish a knowledge acquisition. But, even with the current, wide establishmentof different strategies, there is still a lack of integration in a transverse fashion over the processof teaching-learning of competences, specially at college education, which is reflected by the lowaptitudes that students present when they are requested to recognize their knowledge levels, skills,and competences acquired during their formative process. This lack of abilities have a profoundimpact on the students intrinsic motivations, according to an analysis based on surveys answered bystudents from different levels from the Product Design Engineering career, at Universidad TécnicaFederico Santa María, Chile (data not shown). The students also presented a reduced ability (alongthe years) for regulating their motivations. From these, one can infer that this happens mainlydue to the great uncertainty experienced at the moment of applying their learned abilities in realindustry scenarios, where they have to answer properly to the market needs.Based on this perceived issues, here a model is proposed which can be extended later as

a methodology aiming to accompany students throughout their formative process, encouragingawareness of the transference of knowledge, skills, and competences, into other domains. This, withthe goal to properly educate students for their professional life, reaching a high level of knowledgeabout what are they capable of, and what is the best attitude for accomplishing their aspirations(context-dependent), at a personal and professional level.Developing a competenceIn order to develop a competence, it is necessary a knowledge transference that could articulatetheory and action. There is a shift from a general knowledge to a concrete extramental situation,which affects our conduct efficiently to “do something”. Then, a competent action is such that is

35


Figure 1. Concep-tual model forcompetence devel-opment. Theoreticalmodel that artic-ulates knowledgeand action throughthe development ofcompetences, whichare mediated by thedifferent metacog-nitive dimensionsfor achieving aware-ness and knowledgetransference.

COMPETENCEACQUISITION

TRANSVERSALAND SPECIFIC

DEMANDED BY THE SOCIETY AND THE

PROFESSIONAL MARKET

ON

EXPERIENTALKNOWLEDGEDemonstration

(either internal or external) done

previous the real application of the

acquired competence

TRAJECTORY

FUEL

SELF-REFLECTION

SELF-MONITORING

SELF-EVALUATION

Monitoring ofcognitive components

of behavior

METACOGNITIVEmethodology

MOTOR

Knowing

how to BE / ACT

Knowinghow to

DO

Knowing how to

understand

performed by a subject who master it in all its domains: what has to be done (cognitive knowledge),how it has to be done (procedural knowledge), how do Imust react (attitudinal knowledge) and,at which moment the action has to be started (conditions). This is all shown by how a subjectperforms.Following this, a competent subject is a person who is able to use all these relevant knowledge

in a pertinent, efficient, and long-lasting fashion (Le Boterf, 2001).An integrative modelTaking all the relevant concepts here described, associated to achieving knowledge acquisition andtransference, and development of competences, here we present a conceptual, theoretical modelas a base for developing a metacognitive methodology, transversal to the comprehensive learningprocess, aiming to improve the use of competences in social and professional contexts. The modelis presented in figure 1.The model articulates in conjunction the knowledge that composes a competence (to know,

to do and to act) as amotor for an experiential knowledge. This knowledge introduces a specificway of evaluating and proof to the self (the student), and the surrounding community (teachersand classmates), the competences acquired in the integral formative process of a given course.The experiential knowledge is also able to feedback what has been learned previously, producingawareness about the competences that are mastered by the student, acquired through its educa-tive process, and its meaningful learning in all the addressed areas. It prepares the student forcompetence behaving, which is required once their formative process ends.The fuel that makes possible this swinging between knowledge is a metacognitive methodology

that must involve a reflective and self-critic attitude, through self-questioning what is known (re-flective dimension), what is controllable (managing dimension) and, what can be modifiable andvaluable (evaluative dimension). This three dimensions, besides integrating procedural and cog-nitive elements, integrate emotional, affective, attitudinal elements, among others (Tovar-Gálvez,2005). It is important to clarify that while these three dimensions are generally called as “moni-

36


Project Start

Identify the challenge (or

sub-challenges)

Identify goals and objectives of

the challenge

What skills (or processes) should I know how to apply?

What attitude should I have?

What knowledge do I have to know?

Do I lack skills?

Is my attitude the right one?

Do I lack knowledge?

Formulate an action plan.

META Strategy

Put the plan into action

Evaluate resultsWere they the expected

ones?

NEXT CHALLENGE

Learn what I need to know

Learn how to do

Change my attitude

To consider:TimeResourcesSpacesPeopleMethodsExternal Conditions

Why?

Yes

No

No

No

Yes

Yes

Yes

No

REFLECTIVE DIMENSION

ADMINISTRATIVE DIMENSION

EVALUATIVE DIMENSION

Figure 2. Flow diagram for applying the metacognitive methodology. The model is presented under a systematic view to show how to applythe monitoring of cognitive components in a non-specific learning situation, starting by a challenge that can be analyzed in a higher (macro) level,or by decomposing it on sub-challenges.

toring of cognitive components”, they are a set of elements with multiple featuring components:methodologies, procedures, processes, skills, motivations, interests, etc.Finally, what triggers the dynamics of learning, called here as the button, is the need for develop-

ing competences (either generic or transversal and technical or specific). This needs are directlyassociated with what is required in the social and professional context, and have to be developedinside a given course.ApplicabilityTo describe how to apply the model, figure 2 introduces a flow diagram with separated steps andquestions to self-analyze, prepare and complete the acquisition of knowledge. The reflective dimen-sion has as objective to raise awareness of acquired theories and knowledge. The administrativedimension has to guide for understanding how to use the acquired knowledge. The evaluativedimension has to invite to analyze tangible results (experiential knowledge) to consolidate thelearning process. While the three dimensions are crosswise to the whole experience, the systematicview proposed simplifies the planning of the learning experience.

It is important to note that the diagram exposed in figure 2 can be complemented with othermethodologies, tools or strategies in all its stages, in order to foster the effects on each dimension(e.g., learning strategies as creative thought (De Juanas Oliva and Lozano, 2008), or 10metacognitiveactions (Tobón, 2013)).Also, while the model can be fully or partly applied in different contexts, given the extension,

complexity, and amount of concatenated challenges in a project-based methodology, the steps andanalyses fit better in that context.ConclusionsThe described approach is the first step for a metacognitive methodology that can be transferredto diverse contextual scenarios. But, to achieve that, it is important to promote the conceptsof participatory education, stimulate the redesign of educative curriculum and their evaluativesystems, and promote as a high priority the research on education and the training of teachers aseducative entities. Also, there has to be a permanent supervision and advising in this processes of

37


transformation.A metacognitive methodology builds upon the proposed model might act as a driver and

promoter for meaningful learning which, according to what is exposed herein, would end upwith an integral formation of the student, with incorporated metacognitive skills, facilitating eachconfrontation of future challenges. The main highlight of the proposed strategy is that generallythe tools for acquiring metacognitive skills are introduced to the students in a parceled fashion,and not as a complete, single and continuous strategy.A natural extension of this work is the development of a concrete methodology, which could

incorporate specific tools depending on the learning process. This, introducing the two fundamentalinstances for the developing of competences: an individual instance, and a group instance. Also,this work can be extended introducing the roles played by the teacher and the student following thisnew metacognitive dynamic. Some examples for doing these extensions are the use of tools like aportfolio, activities with gaming cards, or a personal book with established activities, supportingintrospection (personal stage). In a second instance, the tools have to integrate social elementssupporting interpersonal feedback, as methods including simulations and games, confrontingreal case situations or, fomenting multidisciplinary projects of innovation and entrepreneurship inbachelor studies for consolidating the development of competences in personal valuable situations(interpersonal stage).ReferencesArancibia V. Manual de psicología educacional. Ediciones UC; 2008.Ausubel DP, Novak JD, Hanesian H. Psicologia educacional. Interamericana; 1980.Bellocchio Albornoz M. Educación basada en competencias y constructivismo un enfoque y un modelo parala formación pedagógica del siglo XXI. No. F/378 B4; 2009.Brown AL, Smiley SS. Rating the importance of structural units of prose passages: A problem of metacognitivedevelopment. Child development. 1977; p. 1–8.Bruner JS, et al. Toward a theory of instruction, vol. 59. Harvard University Press; 1966.Carretero M. Constructivismo y educación. Editorial Progreso; 2000.Coll C. Significado y sentido en el aprendizaje escolar. Reflexiones en torno al concepto de aprendizajesignificativo. Infancia y aprendizaje. 1988; 11(41):131–142.De Juanas Oliva Á, Lozano MPF. Competencias y estrategias de aprendizaje. Reflexiones sobre el procesode cambio en el EESS/Competency and learning strategies. Think about the process of change in the EEES.Cuadernos de trabajo social. 2008; 21:217.Díaz F, Hernández G. Estrategias de enseñanza para la promoción de aprendizajes significativos. F Díaz Barriga,Estrategias docentes para un aprendizaje significativo. 1999; p. 79–111.Flavell JH. El desarrollo cognitivo. Visor; 2000.Flavell JH, Friedrichs AG, Hoyt JD. Developmental changes in memorization processes. Cognitive psychology.1970; 1(4):324–340.Gagné ED. The cognitive psychology of school learning. Little, Brown; 1985.Le Boterf G. L’ingénierie des competénces. Ingeniería de las competencias; 2001.Martí E. Aprender con ordenadores en la escuela. ICE Universitat de Barcelona; 1992.Martí E. Metacognición: entre la fascinación y el desencanto. Infancia y aprendizaje. 1995; 18(72):9–32.Mateos M. Metacognición y educación. Aique Buenos Aires; 2001.Rejas LP, Ponce ER, Ponce JR. La influencia de la gestión del conocimiento sobre la eficacia organizacional: Unestudio en instituciones públicas y empresas privadas. Revista Facultad de Ingeniería. 2013; (47):218–227.Sperling RA, Howard BC, Staley R, DuBois N. Metacognition and self-regulated learning constructs. EducationalResearch and Evaluation. 2004; 10(2):117–139.Tobón S. Formación integral y competencias: Pensamiento complejo, diseño curricular, didáctica y evaluación.Bogotá: Ecoe Ediciones. 2013; .Tobón ST, Prieto JHP, Fraile JAG. Secuencias didácticas: aprendizaje y evaluación de competencias. Pearsoneducación México; 2010.Tovar-Gálvez JC. Evaluación metacognitiva y el aprendizaje autónomo. Tecné Episteme y Didaxis TE. 2005; p.196.

38

SENA Tecnoacademia Risaralda and Caldas as a Collaborative Learning Scenario in Robotics

Margarita María Vallejo-Jiménez 1 , John Jairo Martínez-Puerta 2 , Sebastián Bedoya Agudelo 1 , Nicole D. Salgado 2 .

1 SENA Tecnoacademia Risaralda; 2 SENA Tecnoacademia Caldas

Abstract: The Research, Technological Development and Innova�on System of SENA (SENNOVA) of Colombia, has the purpose of strengthening the standards of quality and relevance, through programs and projects as Tecnoacademias, defined as a STEM learning scenario, equipped with emerging technologies to develop innova�on-oriented skills, through project training, to students of basic and secondary educa�on, in courses such as Mathema�cs, Physics, Chemistry, Biology, applied sciences such as Robo�cs, Nanotechnology, Biotechnology and Virtual Technologies.

This work presents some of the ac�vi�es carried out by the appren�ces through the Educa�onal Robo�cs in Tecnoacademia Risaralda and Tecnoacademia Caldas sites, based on Industrial and Mechatronic Design methodologies, using LEGO MINDSTORM EV3 kits and Design Thinking for educators and LEGO , successfully applied in the EducarChile program. It is based on three fundamental pillars, which are empathy, collabora�on and experimenta�on, which are presented in the five (5) phases of the methodology. It should be noted that the tools of innova�on and prototyping per se, do not serve much if the team that executes them is not immersed in a culture of tolerance, teamwork, leadership and if there is no feedback and if the capaci�es are not taken into account and strengths of the work team. All this was achieved through different prototypes of robots of light and robust type originated in a PON scenario (problem, opportunity, needs).

Introducción In Colombia, the Ministry of Na�onal Educa�on MEN (2008) proposes: to train in technology, by encouraging scien�fic curiosity for the solu�on of problems and needs of the environment; to propi�ate the development of cri�cal thought and reflec�on, for the control of technology in society; to provide tools for innova�on and crea�vity, in the solu�on of problems from different points of view. One of these tools is the Robo�cs, and since the seven�es (Ruiz, 1987), a new area of study called "Pedagogical Robo�cs" is generated, which uses these artefacts, which


SENA Tecnoacademia Risaralda andCaldas as a Collaborative LearningScenario in RoboticsMargarita María Vallejo-Jiménez1*, John Jairo Martínez-Puerta2*, SebastiánBedoya Agudelo1, Nicole D. Salgado1

*For correspondence:[email protected];[email protected]

1SENA Tecnoacademia Risalda; 2SENA Tecnoacademia Caldas

39

have elements of electronics, programming and mechanics for didac�c purposes; relying on teaching and learning methodologies, changing the tradi�onal role of the teacher and taking the student to an ac�ve role (2010_Pinto-Salamanca). Robots naturally awaken the interests and curiosity of children, excite them to explore their ideas through their inquiries and test their hypotheses, make new discoveries and develop their knowledge through real-world experiences, by using a Technologically and computa�onally improved tool (Eguchi, 2017). As a STEM strategy (Science, Technology, Engineering and Mathema�cs), Educa�onal Robo�cs allows the genera�on of learning environments based on the ini�a�ve and ac�vity of students, for the solu�on of problems that arise in the areas previously exposed (Marquez, 2014 ) and skills in innova�on, crea�vity and real-life problem solving are developed (Ghi�s, 2014). The iden�fica�on, applica�on and valida�on of different mechatronic design tools through the use of pedagogical robo�cs to solve a need, becomes the star�ng point of an educa�on that truly achieves an impact and a change of mentality, taking advantage of the scenarios and the exis�ng infrastructure in the Na�onal Service of Learning SENA (2018), a state en�ty that provides STEM courses of 140 hours, including that of Robo�cs Recrea�on, in its 10 Educa�on Centers called Tecnoacademia. Methods The methodologies described are supported in SENA's project-based learning strategy (2007, Carrera, 2011, GIZ), which allows the applica�on of knowledge and the development of thinking skills, knowledge and the development of biophysical skills, in doing and developing basic skills such as ethics, asser�ve communica�on, and teamwork. Methodology for the Domestic Robot Prototype: The educa�onal robo�cs workshop at Tecnoacademia Risaralda was conducted face-to-face, in two courses of 20 students each, divided into 5 groups of 4 students. Every session lasted an average of 4 hours, for a total of 10 sessions and 140 hours. The students made a robot prototype, first of low fidelity in cardboard and paper, then using the LEGO Mindstorm EV3 kit and the 5 phases of Design Thinking for Educators (IDEO, 2012):

1. Discovery: Through observa�on, students discover that some tasks related to

housework in their homes generate feelings of discomfort.

2. Interpreta�on: The students performed technological surveillance on mechanical structures, sensors, displacement of robots, how they can be programmed and what is the cost of their parts in local stores.


40

3. Idea�on: Brainstroming is done to choose the viable ideas for the solu�on and

are valued with a score of 0 to 5 depending on how well it meets the criteria: · It should be easy to manipulate. · It should be easy to program. · Its construc�on should not be very expensive. · It should be able to move easily. · It should be beau�ful. Then, through a process of co-crea�on, each team joins the winning criteria to realize their idea of robot.

4. Experimenta�on: Each team was given a set of 20 materials (paper, cardboard, ballons, rope and others) and the students made the mechanical and electronic parts (sensors) ini�ally with sketches, as well as describing the programming (behaviors) of the robot, everything by using a prototyping canvas.

5. Evolu�on: The tool to share the history of the robot is designed, for the documenta�on of the process and its valida�on; Each group presented their work to students of the Chemistry line, who evaluated the robot with the same criteria men�oned in point 3. Then, each team was given a LEGO Mindstorms EV3 kit and an expansion kit to build the func�onal prototype, choose the mechanical configura�on, the type of displacement, the sensors and the sequence of movements of the robot, to then be validated by the above criteria.

Methodology for the Space Robot Prototype: The previously described methodology was applied, this �me with a single group of 5 students aged between 15 and 17 years old, members of the Robo�cs line of Tecnoacademia Manizales, who were selected for their high performance in the ac�vi�es of the line .

1. In the Discovery phase, the challenge was to build a robot for explora�on and

liquid sampling in irregular terrain.

2. Interpreta�on: The students performed technological surveillance on locomo�on systems, stability mechanical structures and, displacement of robots on irregular terrains.

3. The Idea�on phase were carried out in the manner already described for the Domes�c Robot Prototype.

4. For the Experimenta�on phase the LEGO Mindstorms EV3 kit was used to design and build the different robot mechanisms: suspension, steering wheels, robot body, probe arm, syringe drive for samples.


41

5. Regarding the last phase, the prototype is in con�nuous evolu�on, with the aim of giving the possibility to new students to depart based on what was learned by their predecessors and from there to give new contribu�ons to the project.

Results

The prototypes of STEM mobile robo�cs are described, which gave solu�on to a design challenge in context (training project), made by students of the Robo�cs Lines of Tecnoacademia Risaralda and Tecnoacademia Caldas, in Colombia. Students developed a variety of lightweight prototypes before developing func�onal prototypes. Domestic Robot Prototypes: In Tecnoacademia Risaralda the students designed and built low fidelity prototypes, which they transformed into func�onal prototypes using the LEGO Mindstorms EV3 kit, as shown in Figure 1.

Figura 1. Domestic Robot Prototypes.

a) b) c)

d) e) f) Figura 1. Domestic robot prototypes: a) Sketches. b) Light prototype front part. c) Light prototype rear part. d) Func�onal prototype front. e) Func�onal prototype rear. f) Collabora�ve work.


42

Space Robot Prototype : Using LEGO Mindstorms EV3 parts, the students built the structures of the wheels based on NASA´s Roker-Bogie system. The best designs were implemented in a prototype that remains in evolu�on, it is a robot for explora�on and taking liquid samples in irregular terrains, as shown in Figure 2. A first prototype was presented at the III Interna�onal Astrobiology Congress in Manizales (Colombia).

Figura 2. Space Robot Prototype

a) b)

c) d)

e) Figura 2. Space robot prototype . a) "Rocker-Bogie" suspension system. b) Arm for taking samples. c) Assembling the systems d) Prototype tests. e) Displacement in irregular terrain.

Discussion The exercise of carrying out a great variety of prototypes using different phases of a design methodology becomes a challenge for the appren�ces, who see how the


43

original idea takes shape step by step, in an orderly, consistent and documented way. The methodology can be applied by using any educa�onal robo�cs kit; In this case, the Lego Mindstorms EV3 kit was chosen due to its availability in the Tecnoacademias. The applica�on of the Design Thinking methodology and LEGO Roles allows students to plan their work systema�cally using different tools for this, to which they are not accustomed; even so, documen�ng what they do in an orderly manner gives them sa�sfac�on and they feel proud; some of them take photographs of this planning and the first sketches to show to their parents. The valida�on and exposure of these prototypes in public, although ini�ally it causes them anxiety, allows others to know the ac�vi�es they do and therefore in the end, they feel proud, understanding that the robot must sa�sfy the needs of the user. In addi�on, they iden�fied their strengths, tastes and interests; This is a star�ng point so that later on they acquire the necessary skills so that they can con�nue with their training at higher levels or perform jobs related to science and technology that are useful to the community. In the design phase, despite its abstrac�on, ideas flow and can some�mes become overwhelming, but with the proposal of the mechatronic approach, to divide the robot into the mechanical, electronic (sensors) and programming sub-systems, they go specifying the op�ons. In the construc�on, the integra�on of the mechanics with the sensors allows a coordinated work on the part of the appren�ces and the facilitator, giving shape to the abstract. In programming, as the appren�ces define it, the robot is given "life", crea�vity and collabora�ve work flow again to arrive at a defined programming. Finally, tes�ng, documen�ng and sharing is a rewarding experience for facilitators and students, as it allows other people to understand the process carried out and realize the importance of applying this knowledge in context. The SENA evalua�on system consists of verifying whether or not students achieve certain learning outcomes, wich are defined in their training program. For this, evalua�on instruments are defined that value knowledge, produc�on and product; The prototypes are evaluated by applying such instruments. There are s�ll more tools to be designed and applied on each of the phases of the methodology, these will be carried out in future courses and will help to improve the experience with the students; For their part they could design the mechanical parts in a so�ware and print them in 3D, and they could even design their own electronic components.


44

Acknowledgments We thank the Na�onal System of Research, Technological Development and Innova�on SENNOVA of the Na�onal Service of Learning SENA, who finance the research; the directors of Risaralda Regional and Caldas Regional and, especially the subdirectors sof the centers an the leaders of the Tecnoacademias of Risaralda and Caldas 2018, who believe in the power to transform the lives of young people through science, technology and research. References Brown, R., Brown, J., Reardon, K., & Merrill, C. (2011). Understanding STEM: current percep�ons. Technology and Engineering Teacher, 70(6), 5. Eguchi, A. (2017). Bringing Robo�cs in Classrooms: Redesigning the Learning Experience. Robotics in STEM Education (pp. 3-31). Springer, Cham, Suiza. Ghi�s, T., & Vásquez, J. A. A. (2014). Los robots llegan a las aulas. Infancias imágenes, 13(1), 143-147. IDEO (2012). Design Thinking para educadores y herramientas. 2ª Edición, version translated by the portal educarchile, of the Ministry of Educa�on and Fundación Chile. Retrieved from: h�ps://designthinkingforeducators.com/. Consulted in February 2018. Márquez, D., Jairo, E., Ruiz, F., & Javier, H. (2014). Robó�ca Educa�va aplicada a la enseñanza básica secundaria. Didác�ca, innovación y mul�media, (30), 1-12. MEN Ministerio de Educación Nacional (2008). Car�lla No. 30:. Ser competente en tecnología. Una necesidad para el desarrollo y Orientaciones generales para la educación en tecnología. Ministerio de Educación Nacional de Colombia. Ruiz, E., & Sánchez, V. (1987). La robó�ca pedagógica. Centro de Estudios sobre la Universidad CESU, Universidad Nacional Autónoma de México.

Salamanca, M. L. P., Lombana, N. B., & Holguín, W. J. P. (2010). Uso de la robó�ca educa�va como herramienta en los procesos de enseñanza. Ingeniería Inves�gación y Desarrollo: I2+ D, 10(1), 15-23.

Servicio Nacional de Aprendizaje SENA (2018). Página oficial. Retrieved from: h�p://www.sena.edu.co/es-co/formacion/Paginas/tecnoacademia.aspx. Consulted in April 2018.


45


CRABOT: A six-legged platform forenvironmental exploration andobject manipulationOscar A. Silva1*,2, Pascal Sigel1, Warren Eaton1, Cristian Osorio1, EduardoValdivia1, Nicolás Frois1, Felipe Vera1

*For correspondence:[email protected] (O.A.S.)

1Innovación y Robótica Estudiantil, Universidad Técnica Federico Santa María; 2Centro deInnovación y Robótica

Abstract The advantage of using legs instead of wheels are several, including better mobility andmaximizes energy consumption. Moreover, a leg can be taken into account as a useful arm if thedesign promotes it. Following this idea, here we describe a six-legged platform called CRABOT, a3D-printed robot able to be self-sustained by at least four of its legs, leaving other two forrobot-object interactions. Integrating a 4D camera, CRABOT has been designed for exploration ofenvironments, while is capable of manipulating objects with any of its legs. This article is focusedon the hardware design of the platform and its inverse kinematics required for achievingmovement. As possible future extensions, there is still an open issue on the design of differentmanipulators attachable to the platform.

IntroductionLegged robotic platforms are well known for being highly useful for displacement on unknown, com-plex environments (Bekker, 1960). While the use of wheels is easier in terms of control dynamics,the use of legs can improve not only the reachable space, but also it can increase the displacementvelocity, energy use, mobility, among others (Song and Waldron, 1989). Instead, wheels platformsare severely limited by the roughness of the terrain, environments commonly present under searchand rescue situationsMurphy et al. (2008). Torres Redondo (2015) has presented a recent analysisintroducing the advantages of using legs in outer-space.Currently, there are several types of legged robotic platforms, usually identified by the number

of legs, commonly ranging between one to eight (Torres Redondo, 2015). There are, among others,biped humanoid robots, quadrupeds, hexapods and octopods, insect-like (e.g., Chen et al. (2018))or bigger animals (e.g., Kalouche (2017)). Despite the advantages for displacement, we believe thatthe legs are also significantly meaningful when a platform has to interact with the environment,given that the legs by itself can act as manipulators. In that sense, here we propose the designof a six-legged robotic platform called CRABOT, which has been designed for a later integrationof different tools that can be attached and used on any of its legs, becoming a Leg-Arm Hybridrobot (Kajita and Espiau, 2008). Then, the platform would be potentially capable of doing sampleacquisitions or other complex object manipulations, among several other interactions, using toolscontained by each leg. The tools could be either different or equals, which automatically reducesthe problem of positioning the platform in an exact position near the object in use.

46


Figure 1. CRABOT, a six-legged 3D-printed platform designed for exploration and object manipulation. Platformfeatures are listed in table 1.

Design of the platformStructural design and characteristicsIn the seek of developing a multiple-purpose platform which could integrate the benefits using oflegs, as described before, we have built a hexapod robot, which is similar to a crab (shown in figure1). All the structural parts of the robots are 3D-printed, including its chassis and legs.This proposal considers a six-legged platform for an easily positioning and supporting of the

robot using generally four legs, leaving at least two of them free for object interaction. Similar tothe MELMANTIS-1 platform (Koyachi et al., 2002), CRABOT is able to transit in rough terrain while itcan use a manipulator when interactions with objects are necessary. CRABOT is able to stand infour or five of its legs, which leaves one or two manipulators free of use, only depending on whatare the goals of the manipulation. It can interact with objects with maximum size and weight as theones detailed in table 1. All the designs of the platform parts are open, and available at a Githubrepository:https://github.com/osilvam/CrabotFigure 2 show the rotational points of the designed legs, without including any ending manipula-

tor. Each leg has four degrees of freedom (DOF), whose last three DOF are used for movements fordisplacement, being enough for letting CRABOT move around in almost any kind of environment.The last DOF, which is attached to the platform center (i.e., the torso) is in charge of increasing thereaching space for reaching and manipulation purposes, acting as a rotating shoulder for each limb.While currently each leg ends into a tip, its internal space is aimed for tool storage and deploy-

ment on later developments to the platform (see figure 2). Following that design, the platformcould incorporate a maximum of six different manipulators, used also as standing points when

Object ManipulationObject weight 100 [gr]Object size 5 × 5 [cm]

Body characteristicsRobot weight 2.5 [kg]Carrying weight 300 [gr]

Autonomy and displacementBattery duration 15 [min] (Battery: 5000A a 6 × 1.3 [Ah])Walking speed 5 [cm/s]

Table 1. Platform characteristics

47


needed. Given the radial symmetries of the platform design, the addition of manipulators (e.g., aplatform with eight legs/manipulators) only imply architectural modifications at the torso.Hardware componentsMobile actuatorsEach limb of the platform is composed of four Smart Serial Servo from Dynamixel1 (model AX-18a,with a maximum torque of 1.8 [Nm]). Prior tests using servos from the same series but with 1.2[Nm] produced an unstable gate lacking the strength for self-sustain the weight of the platform.These servomotors are easily connected and controlled by their serial capability. Also, as their nameintroduces, these servos are smart as they include an internal driver which not only allows them toreach an exact positioning but also provides to an external controller meaningful information asinternal motor temperature, the level of current consumption, the instant torque being applied,among others. This motors also include a gearbox necessary for applying the levels of mechanicaltorque required for moving a platform as CRABOT.SensorsCRABOT includes an inertial measurement unit (IMU), which gives information about the absoluteinclination of the robotic torso, computed by estimating pitch, yaw and roll, and its accelerationon each of the three common coordinate axes. The platform also includes a 4D Kinect sensor,incorporating an RGB camera and proximity sensing, information that allows the platform toexplore and reconstruct unknown environments. The Kinect is replaceable with any other suitable4D camera, in terms of the task.The platform also includes sensors integrated to its servo motors. These sensors give CRABOT

online information about how each actuator is interacting with the environment, information thatcan be used for computing the strength applied in a given task, mean and maximum currentconsumption, supported load, among others.Drivers and controllersFor managing all the robot motor signals, the platform includes the CM-700 driver (and its com-plementary board CM-700 SUB board), which is developed by ROBOTIS specifically designed forcontrolling Dynamixel servo motors. The use of this drivers allows to easily manage and controlseveral motors in parallel. This board is connected to a central controller unit. As the CM-700board is connected through TTL/RS485, to be used with any computer an interface to USB has tobe considered, as the USB Downloader LN-101 from ROBOTIS.

1Dynamixel is a line of actuators from ROBOTIS. More information can be found at http://en.robotis.com/.

Leg section to be replaced in a later version with a dual purpose ending, leg and manipulator.

Figure 2. Leg design. Each leg is composed by four degrees of freedom, which are shown here by the orangesticks.

48


The central controller unit in charge of the behavior of the platform is a microcomputer that isplaced inside the torso. CRABOT was developed for using the ODROID board from Hardkernel 2,but any computer unit could be attached to the platform.Movement algorithmsSimilar to a person and other animals, a robotic platform has to perform several actions in orderto complete a task. For just displacement, we can think in moving forward, turning, changing thecenter of mass for allowing platform inclinations, among others. For any required movement, thereare common descriptions that need to be addressed: the platform forward and inverse kinematics.Direct kinematicsIn order to reach any place at the space, the different motors have to act and rotate. To address thisproblem, which has infinite possibilities and is solved by inverse kinematics, we will start describinga representational framework to acquire the direct kinematics of the platform. Figure 3 shows arepresentation of each link and a different coordinate axes at each joint. Then, at each coordinateaxes, each joint has to rotate with respect of its own coordinates. Then, the movement of the tipof each leg can be thought as a function f that depends on a set of angles corresponding to thedifferent DOF of a limb, and which gives as a result a point in the space with coordinates (x, y, z), as

f (�) = X (1)where � = (�1, �2, ..., �n) is the set of angles and X = (x, y, z). Moreover, f (�) can be expressed asa matrix multiplication, using a transform matrix considering rotation and translation for eachcoordinate axis.Then, from the base to the tip of each limb, the ending point can be computed as

f (�)R1(�1)R2(�2)T1(d1)R3(�3)T2(d2)R4(�4)T3(d3) (2)where the R(�) and T (d) correspond to a matrix rotation by a given angle � and a matrix translationgiven a distance d, respectively, as shown in Box 1.Then, by using the representation in equation (1), the position of the tip of any leg of the platform

can be obtained just knowing the rotational angles of each of motor of that leg.Now, with this information, we are not capable of computing angles when we need to reach a

point, but rather we are able to now the position of the tip given certain rotational angles. In orderto know how to reach any point, we need the inverse of equation (1), finding the inverse kinematicsof the platform.

2Specifications and more information about the ODROID board can be found at https://www.hardkernel.com.

Figure 3. Diagram showing the representation of each coordinate axes of a leg, one for each joint plus onerepresenting the traslation from the last joint to the tip.

49


Box 1. Example of rotation and translation matricesHere, examples of rotationalmatrices are shown foreach coordinate: Rot(x, �),Rot(y, �) and Rot(z, �).Also, a translation matrixT rans(x0, y0, z0) is presented.These matrices are thebases for representing theposition of the tip of a leg bythe relative coordinates ofthe robot.

Rot(x,�)=

⎡⎢⎢⎢⎢⎢⎢⎢⎣

1 0 0 01 cos(�) −sin(�) 01 sin(�) cos(�) 00 0 0 1

⎤⎥⎥⎥⎥⎥⎥⎥⎦

Rot(y,�)=

⎡⎢⎢⎢⎢⎢⎢⎢⎣

cos(�) 0 sin(�) 00 1 0 0

−sin(�) 0 cos(�) 00 0 0 1

⎤⎥⎥⎥⎥⎥⎥⎥⎦

Rot(z,�)=

⎡⎢⎢⎢⎢⎢⎢⎢⎣

cos(�) −sin(�) 0 0sin(�) cos(�) 0 00 0 1 00 0 0 1

⎤⎥⎥⎥⎥⎥⎥⎥⎦

T rans(x0 ,y0 ,z0)=

⎡⎢⎢⎢⎢⎢⎢⎢⎣

1 0 0 x00 1 0 y00 0 1 z00 0 0 1

⎤⎥⎥⎥⎥⎥⎥⎥⎦

Inverse kinematicsIn order to find rotational angles that would position the tip of a leg in a certain position in thespace, we can express equation (1) using Taylor’s series. Then, each coordinate can be expressed as

x = f (�0) +)f (�))�1

|||�0 ⋅ (�1 − �01 ) +⋯ + )f (�))�n

|||�0 ⋅ (�n − �01 ) + error((� − tℎeta0)2

) (3)Now, if we express all the coordinates following equation (3), we can use the matrix form

⎡⎢⎢⎢⎣

xyz

⎤⎥⎥⎥⎦=⎡⎢⎢⎢⎣

f1(�0)f2(�0)f3(�0)

⎤⎥⎥⎥⎦+⎡⎢⎢⎢⎣

)f1)�1⋯ )f1

)�n)f2)�1⋯ )f2

)�n)f3)�1⋯ )f3

)�n

⎤⎥⎥⎥⎦�0

⎡⎢⎢⎢⎣

�1 − �01⋮

�n − �0n

⎤⎥⎥⎥⎦

(4)

There, we have removed the error as its value is not meaningful on the results. Then, equation(4) can be rewritten as

X = F (�0) + J�F (�) |�0 ⋅ (� − �0) (5)where J�F (�) is the Jacobian matrix of the spatial coordinates of the direct kinematics. Then, it ispossible to inverse this equation, obtaining �:

� = J−1�F (�)|||�0

(X − F (�0)

)+ �0 (6)

Moreover, we can take the first derivative of equation (6) to obtain the angular velocity necessaryfor optimal movement, considering a desired (known) linear velocity:

)�)t

= J−1�F (�)|||�0 ⋅

)X)t

(7)Using equations (6) and(7), we are able to calculate angles for reaching a certain point, and to

control the movement velocity in order to not produce wrong, possible harmful, movements ofeach leg.

In order to keep all the movements smooth, for any given point, the algorithm that reach a placetakes into consideration the initial point and the final point. Using this two points, a trajectory isconstructed (which depends on the given task, but could be linear), from which a closer point tothe initial position is extracted. The ending point for computing the new motor angles, followingequations (6) and (7), is the closer one extracted from the constructed trajectory. Once that the newangles are obtained, the movement of a leg is triggered and performed.The developed software for controlling the platform, including a program that runs in a server

for remote control using a web-based application usable in mobile devices, are also available at theonline repository ( https://github.com/osilvam/Crabot).

50


ConclusionsCRABOT is a multipurpose platform developed for remote exploration and object manipulation.The current advances allows to control the platform, with described mathematical descriptions ofits direct and inverse kinematics. The platform is capable to sustain up to two legs in the air forobject interaction, which makes it a very versatile legged robot.While the necessary hardware and mathematical approaches for controlling the platform are

here described, there is still an open problem about the specific design of different manipula-tors that could be attached to its legs. Extending the platform with an analysis and design ofmanipulators can contribute, as explained here, to the development of robots with lower energyconsumption ratios while exploiting the mobility advantages of using legs.ReferencesBekker MG. Off-the-road locomotion: research and development in terramechanics. University of MichiganPress; 1960.Chen Y, Doshi N, Goldberg B, Wang H, Wood RJ. Controllable water surface to underwater transition throughelectrowetting in a hybrid terrestrial-aquatic microrobot. Nature communications. 2018; 9(1):2495.Kajita S, Espiau B. Legged robots. In: Springer handbook of robotics Springer; 2008.p. 361–389.Kalouche S. GOAT: A legged robot with 3D agility and virtual compliance. In: Intelligent Robots and Systems (IROS),2017 IEEE/RSJ International Conference on IEEE; 2017. p. 4110–4117.

Koyachi N, Adachi H, Izumi M, Hirose T, Senjo N, Murata R, Arai T. Multimodal control of hexapod mobilemanipulator MELMANTIS-1. In: Proceedings of 5th International Conference on Climbing Walking Robots; 2002. p.471–478.Murphy RR, Tadokoro S, Nardi D, Jacoff A, Fiorini P, Choset H, Erkmen AM. Search and rescue robotics. In:Springer handbook of robotics Springer; 2008.p. 1151–1173.

Song SM, Waldron KJ. Machines that walk: the adaptive suspension vehicle. MIT press; 1989.Torres Redondo J. Analysis and optimization for a hexapod walking robot for planetary missions. PhD thesis,Industriales; 2015.

51

Author Index

Alfaro, Ricardo 25

Allende, Héctor 11

Alvarez, Jorge 25

Angel, Camila 32

Aubel, Maximiliano 25

Barrera, Tania 25

Bedoya-Agudelo, Sebastián 39

Castillo, Sebastián 25

Eaton, Warren 46

Frois, Nicolás 46

Hernández, Nicolás 25

Kim, Sung H. 25

Martínez-Puerta, John J. 39

Nettle, Cristóbal J. 32

Ollino, Franco 11

Osorio, Cristian 46

Osorio, Felipe 25

Pinto, Felipe 25

Reyes, Pablo 25

Rodenas, Tomás 25

Salgado, Nicole D. 39

Sigel, Pascal 46

Silva, Oscar A. 46

Solis, Miguel A. 11

Torres, Daniel 25

Torres, Felipe 18

Valdivia, Eduardo 46

Vallejo-Jiménez, Margarita M. 39

Vicencio, Ignacio 25

Vera, Felipe 46

Yañez, Pablo 25

53

Congress on Robotics and NeuroscienceCRoNe is a focused, multidisciplinary event part of the Latin American Robotics Week organized by the Innovation and Robotics Students group at UTFSM. It is a meeting point for people from engineering, human and biological sciences for developing and understanding complex intelligent systems.

During its 4th version, the Congress on Robotics and Neuroscience was focused on four different areas:

Inside this volume, you will find:

ISBN: 978-956-09282-0-7.Copyright ©2019 for the individual papers by the papers’ authors. Copying permitted only for private and academic purposes. This volume is published and copyrighted by its editors.

Batch Reinforcement Learning on a RoboCup SSL Keepaway Strategy Learning Problem 11-17Franco Ollino, Miguel A. Solis, Héctor Allende.Trending Topics on Science, a Tensor Memory Hypothesis Approach 18-24Felipe Torres.AIS: Artificial Intelligent Soccer 25-31Maximiliano Aubel, Ricardo Alfaro, Pablo Yañez, Pablo Reyes, Tomás Rodenas, Nicolás Hernández, Felipe Pinto, Sung Hee Kim, Tania Barrera, Daniel Torres, Ignacio Vicencio, Jorge Alvarez, Felipe Osorio, Sebastián Castillo.A Theoretical Strategy for Enhancing Learning Through Metacognitive Practices on a Constructivist Methodology 32-38Camila Angel, Cristóbal J. Nettle.SENA Tecnoacademia Risaralda and Caldas as a Collaborative Learning Scenario in Robotics 39-45Margarita María Vallejo-Jiménez, John Jairo Martínez-Puerta, Sebastián Bedoya-Agudelo, Nicole D. Salgado-Giraldo.CRABOT: A Six-legged Platform for Environmental Exploration and Object Manipulation 46-51Oscar A. Silva, Pascal Sigel, Warren Eaton, Cristian Osorio, Eduardo Valdivia, Nicolás Frois, Felipe Vera.

Development of meaning: with approaches from Developmental Robotics, Machine Learning, and Brain-based theories.Predictive coding for cognitive development: introducing analyses from Neuroscience, Computational Neuroscience, and AI.Multimodal cognition: under neuroscientific and psychological scopes.Experimental analyses and methodologies: presenting novel methodologies for closed-loop brain training, brain functional connectivity analyses and Machine Learning applications.

robotics and neuroscience · robotics, machine learning, and brain-based theories.-predictive...

Documents