the ball is not just orange

A B I N J O: U C L C R I

Hannes Schulz, Hauke Strasdat, and Sven Behnke

University of FreiburgInstitute of Computer Science

Nov 29, 2007

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

M

The ball

is small,

is easy to confuse with other objects

is the most important object on the field:You cannot play sensibly without knowing its position

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

M

The ball

is small,

is easy to confuse with other objects

is the most important object on the field:You cannot play sensibly without knowing its position

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

M

The ball

is small,

is easy to confuse with other objects

is the most important object on the field:You cannot play sensibly without knowing its position

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

M

The ball

is small,

is easy to confuse with other objects

is the most important object on the field:You cannot play sensibly without knowing its position

. We should put a lot of effort into finding the single real ball.

O

1 I B N-B

2 F B C

3 C B C

4 E

O

1 I B N-B

2 F B C

3 C B C

4 E

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D B L L?

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D B L L?

T E C

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D B L L?

L C

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D B L L?

M B

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D B L L?

CW L

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D B L L?

C R

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D N-B L L?

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D N-B L L?

H F

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D N-B L L?

O O

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D N-B L L?

O O

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D N-B L L?

F

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H D N-B L L?

F

O

1 I B N-B

2 F B C

3 C B C

4 E

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

C E YUV S

Actual ball color

Wider, brownish color

. Allows for motion blur

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

C E YUV S

Two ellipses for “orange”

Actual ball color

Wider, brownish color

. Allows for motion blur

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

C E YUV S

Two ellipses for “orange”

Actual ball color

Wider, brownish color

. Allows for motion blur

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

S C I

orange

white

green

(-candidate)

64 : 1

YUV Camera Image

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

S C I

orange

white

green

(-candidate)

64 : 1

1. Find Maximum

2. Find Weighted Mean

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

S C I

orange

white

green

(-candidate)

64 : 1

cut corresponding area

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

S C I

orange

white

green

(-candidate)

64 : 1

cut corresponding area

Box size depends on position in image

O

1 I B N-B

2 F B C

3 C B C

4 E

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

P C

. Projection changes with ellipses: Robust to changes inlighting conditions

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

P C

. Projection changes with ellipses: Robust to changes inlighting conditions

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

P C

. Projection changes with ellipses: Robust to changes inlighting conditions

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

P L

YUV-Image Y-Image Subsampled Y-Image

. Subtraction of mean: Robust to changes in lighting conditions

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H L F

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H L F

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

H L F

+

-

- -

-

+

-

- -

-

+

-

- -

-

+

-

- -

-

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

N N C

+

-

- -

-

1: Ball

0: No Ball

O

1 I B N-B

2 F B C

3 C B C

4 E

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 06 – T

H520 MHz ARM PocketPC

VGA HTC Camera

D S160 balls

440 non-balls

divided randomly in training set(80%) and test set (20%)

P (T S)100% of distractors classifiedcorrectly

1 ball out of 32 not recognized

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 06 – T

H520 MHz ARM PocketPC

VGA HTC Camera

D S160 balls

440 non-balls

divided randomly in training set(80%) and test set (20%)

P (T S)100% of distractors classifiedcorrectly

1 ball out of 32 not recognized

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 06 – R T

H520 MHz ARM PocketPC

VGA HTC Camera

R T T

Robot decides autonomouslywhich object to approach.

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 2007

H1.33 GHz PC

WVGA µEye Camera

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 2007

S D D S273 balls

548 non-balls

training set 62%, validation set13%, test set 25%

varying lighting conditions

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 2007

A SAvg Luminance

ball non-ball

Avg Orange-Greennessball non-ball

D S273 balls

548 non-balls

training set 62%, validation set13%, test set 25%

varying lighting conditions

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

NR KS 2007

A SAvg Luminance

ball non-ball

Avg Orange-Greennessball non-ball

D S273 balls

548 non-balls

training set 62%, validation set13%, test set 25%

varying lighting conditions

R T S91.1% accuracy

76.6% if stimuli flipped up/down.Drop suggests dependency ongradient.

88.2% if lighting in testset differs.Classifier seems indifferent.

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

C C

Classifier

Task Neural Net Linear Classifier KNN (k = 5)

Regular 91.1% 86.5% 88.5%Flipped 74.0% 70.0% 76.6%

Balls and Non-Balls Finding Ball Candidates Classification of Ball Candidates Experiments

C

The ball has properties aside from being orange

These properties are exploited by our Neural NetworkClassifiers

Changes in Lighting conditions can be dealt with by projectionto lines in YUV-space and Luminance.

The method introduced here can be generalized to othersmall-sized objects on the field.