Evolving Cutting Horse and Sheepdog Behavior

with a Simulated Flock

Chris Beacham

Computer Systems Research Lab 2009

Table of Contents

• Focus• Applications• Previous Research• Explanation of Problem• Flocking• Designing the Project• Running the Project• Results• Future Research Possibilities• Evaluation of Success• Questions

Focus

The focus of this project is attempting to evolve the behavior of a single agent or small group of agents so that they can effectively direct the movement of a much larger group which displays flocking behavior.

In English: Trying to produce a computer program that can herd like a sheepdog

Applications

Two main applications:Animation – Control direction of flocks in

animations, currently no good method exists

Livestock Herding: Could be used by a robotic agent to actually herd livestock. Robotic agent could reduce livestock stress.

Previous Research

There has been little development in this specific field.

Flocking algorithm is well established, but has no method to control movement.

Robotic Sheepdog Project: Proof of Concept for robotic handling of livestock

Explanation of Problem:Sheepdog and Cutting Horse

The sheepdog directs the movements of sheep, ducks, cows, goats and other herd stock. It accomplishes this by moving around the herd or flock in specific ways.

The cutting horse is used with herds of cattle. It’s job is to single out a single cow, and “cut” it off from the herd, thus allowing people to access it.

Flocking

The herd will be simulated with a flocking algorithm.

Three algorithms make up flocking behavior:SeparationCohesionAlignmentAssumption: Herds obey the same rules as flocks,

but are stable

Designing the Project:Genetic Algorithms

• Genetic Algorithms mimic evolution. • They provide unexpected and non-intuitive

solutions.• Capable of searching massive search space.• Require four things:

– Gene Pool– Heuristic (scoring system)– Breeding Algorithm– Time

Designing the Project: Genetic Algorithms

• 1000 agents per generation• Each agent is evaluated• 500 agents are placed into the breeding pool

– 15 copies each of the 10 best agents– 5 copies each of agents in places 11-50– 1 copy of each agent in places 51-200

• 500 new agents are bred randomly from the breeding pool

• All new agents and breeding pool agents are mutated. These comprise the next generation.

• Repeat

Designing the Project:Gene Pool

• Which behavior determination system to use?• Neural Networks• Cellular Automaton • Weighting

• Pros and Cons for all options

Designing the Project:Gene Pool

Weighting method chosen as best behavior determination system.

Genetic code adds weight to different vectors that determine behavior.

ProsMany strategies can be producedShould be reproducible in other situationsConsNowhere near the possibilities for innovation of

neural nets.

Designing the Project: Heuristic• 2 different methods – Time trials and Distance

trials• Both types evaluate a single agent• Time Trials

– Create a goal, agent and flock– Stop trial when flock is within certain distance of

goal– Score is the time taken to reach that distance– Should create fast solutions

• Distance Trials– Create a goal, agent and flock– Stop simulation after a certain amount of time– Score is the distance from the flock to the goal– Should create exact solutions

Running the Project: Creating the Simulation

Running the Project: Creating the Simulation

Running the Project: Creating the Simulation

Running the Project: Creating the Simulation

Running the Project:Now We Wait

• Lots of time need for repeated trials• Monopolized a server round-the-clock for

several weeks• 14 simultaneous populations• Over one million agents created and evaluated

Results

• Two strong successes• Herding behavior is surprisingly simple, relies on

only two vectors to determine movement.• Demonstration.

Results:Genetic Algorithms

2 – Fixed Error5 – Automated Testing, increased field size

Results:Genetic Algorithms

Results

• Essential difference between flocking and herding behavior – Specificity

• Flocking behavior is non-specific, and thus hard to judge.

• Herding behavior is goal oriented and specific. It requires tuning to get good results

Evaluation of Success:Are any Strategies Viable?

• Two applications: Animations and Livestock• Livestock – Strategy would likely work. Needs

feedback mechanisms and real life trials.• Animation – depends on application. Evolving a

specific agent would take too long, but a trial and error implementation could give good results.

Future Research

• Future research should focus on these areas:– Feedback mechanisms and fine tuning– Option of other vectors to determine behavior– Accuracy of Herd simulation– Actual livestock responses and trials

Questions?

Third Quarter

• Breeding program and Automated testing program produced.

• Breeding program designed for very heavy selection pressure

• Agents evaluated on how close they get the flock to the goal.

Third Quarter

• Several strategies predominate• “Bubble” strategy• “Slow” strategy• “Southeast” strategy• “Spinning wheel” strategy• “Wall-bounce” strategy – suspect disconnect

between graphical output and nongraphical output

Second Quarter

First generation sheepdog behavior is randomly generated. Movement is very chaotic.

First QuarterThe first quarter

was spent creating a flocking behavior simulation, and debugging it.

This is a screenshot of a flock flying to the right.

Second Quarter

Herding behavior established. Same algorithm as flocking.

Second Quarter

Herd reacting to ‘predator’

Sheepdog agent very simple at this point