wt10

1Weka Tutorial 10 - Neural Networks© 2009 – Mark Polczynski

All rights reserved

Weka Tutorial 10 –Neural Networks

TechnologyForge

www.technologyforge.net/tutorialsweka

Version 0.1

A fractionation column identifies components in a fluid by passing the fluid through the column and examining what comes out as a function of time.

time

Source for fractionation column example and dataset:

http://www.rmltech.com/

2Weka Tutorial 10 - Neural Networks

0.2 6.7 6.1 6.1 61.3 3.7 4.1 10.2 5.60.5 6.3 6.2 8.7 6.11.1 3.7 7 12.5 5.811 17.6 23.9 25.1 9.60.1 7 9.3 17.3 6.825 23.2 26.5 23.9 10.80.4 7.5 10 14.6 6.719 22.3 21.9 21.7 9.70.4 1.8 1.9 7.1 5.415.4 21.2 20.6 17.7 9.10.2 7.4 6.3 12.7 6.415.7 21.4 23 25.4 101.5 5.7 8 18.1 6.50.1 1.8 3.1 7.3 5.4

V0 V1 V2 V3 Z

Input attributesOutput

attribute Goal: Predict value of

Z from V’s.


Neural network regression model

V0-V3 raw data

Z corrected value

Weka Tutorial 10 - Neural Networks 4

V0

V1

V2

V3

Z

Z = (W0 * V0) + (W1 * V1) + (W2 * V2) + (W3 * V3) + T

Neural network structure for curve-fitting fractionation column calibration dataset

T = 1


2. Fire up MultilayerPerceptro

n

2. Fire up MultilayerPerceptro

n

1. Load up FractiuonationColumn.ar

ff

1. Load up FractiuonationColumn.ar

ff

Set up Weka to do curve-fitting


Set up Weka to do curve-fitting


Complicates interpretation of

results, so we won’t use here.

V0 V1 V2 V3 Y1.3 4.4 3.3 8.5 5.60.5 5.1 4.5 10.7 5.90.8 8.1 8.4 12.7 6.6

5.6565.968

6.689

Predicted values of Y


Apply neural network weights to instances to predict Z values

Apply neural network weights to instances to predict Z values


Weka’s calculation of predicted outputs vs.

actual Z values in dataset

How well did we do?


0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

1 2 3 4 5 6 7 8 9 10

Bin Frequency Cumulative2% 47% 47%4% 34% 81%6% 13% 93%8% 2% 95%

10% 2% 97%12% 1% 98%14% 1% 99%16% 0% 99%18% 1% 100%20% 0% 100%

Histogram of % absolute error

For 80% of the instances, the difference between the actual and predicted value of Z is less than 4%

For 80% of the instances, the difference between the actual and predicted value of Z is less than 4%


Fractionation column output: V0, V1, V2, V3Example: 8.0, 11.2, 12.9, 16.6

Corrected Z value: 7.6

Use neural network model to calculate modified

Z value


Segment of Fisher’s Iris DatasetInput OutputAttributes Attribute

Inst.Sepal

LengthSepal Width

Petal Length

Petal Width Species

1 5.1 3.5 1.4 0.2 setosa2 4.9 3 1.4 0.2 setosa3 4.7 3.2 1.3 0.2 setosa4 4.6 3.1 1.5 0.2 setosa5 5 3.6 1.4 0.2 setosa

Numerical Nominal

Class

Using Weka to do classification


SepalLength

SepalWidth

PetalLength

PetalWidth

1

VersicolorVersicolor

SetosaSetosa

VirginicaVirginica

Neural network to classify Fisher’s

iris dataset

Weka Tutorial 10 - Neural Networks

X1

X2

X0 = 1

W1

W2

W0

Y R

14

1

Y

R

0

Perceptron model for classification:

TrainTrain

Calculate

Calculate

Calculate

Calculate

1(1+exp(-Y)

1(1+exp(-Y)


Set up the classifier,then build the

neural network


One node for each of the three iris classes

Weka built one output node for each iris species


Weights for the three output nodessetosasetosa

versicolorversicolor

virginicavirginica


Inst.Sepal

LengthSepal Width

Petal Length

Petal Width Species

Setosa Y

Versicolor Y

Virginica Y

Setosa R

Versicolor R

Virginica R Class

1 5.1 3.5 1.4 0.2 setosa 6.57 -7.28 -57.61 0.999 0.001 0.000 setosa2 4.9 3.0 1.4 0.2 setosa 5.26 -1.73 -51.53 0.995 0.151 0.000 setosa3 4.7 3.2 1.3 0.2 setosa 5.96 -5.82 -53.40 0.997 0.003 0.000 setosa

Attribute values from dataset

Product of attribute values

and weights

Applysigmoidfunction

to Y

Predictedclass

They match!

1/(1+e-y)1/(1+e-y)Weights from previous slide

Sample calculation of

predicted class


Confusion matrix for classification of Fisher’s iris dataset


Outlook Temp. Humidity Windy Playsunny hot high false nosunny hot high true no

overcast hot high false yesrainy mild high false yesrainy cool normal false yesrainy cool normal true no

overcast cool normal true yessunny mild high false nosunny cool normal false yesrainy mild normal false yessunny mild normal true yes

overcast mild high true yesovercast hot normal false yes

rainy mild high true no

Input AttributesOutput Attribute

Class

Weather dataset with

all nominal values


Handles nominal attribute valuesHandles nominal attribute values


Overcast

Sunny

Rainy

One nominal attributewith three values

Three binary attributeswith values 0 and 1

Outlook

Play = yes

Play = no

Nominal values:•sunny,•overcast,•rainy

To handle nominalattributes, performnominal-to-binary

conversion.

If Outlook = sunny then Sunny = 1, else Sunny = 0


Outlook Temp. Humidity Windy Playsunny hot high false nosunny hot high true no

overcast hot high false yesrainy mild high false yesrainy cool normal false yesrainy cool normal true no

overcast cool normal true yessunny mild high false nosunny cool normal false yesrainy mild normal false yessunny mild normal true yes

overcast mild high true yesovercast hot normal false yes

rainy mild high true no

Instance = 1Input Sunny 1Nodes Overcast 0

Rainy 0Hot 1Mild 0Cool 0Humidity 1Windy 0

Output Yes 0Nodes No 1Nominal-to-binary

conversion for the first instance.


Results for classification of the weather dataset

with nominal values.


Sigmoid Node = yesInputs Weights UnknownThreshold -0.75243 1Outlook=sunny 2.98673 0Outlook=overcast -3.33561 1Outlook=rainy 1.066728 0Temp.=hot 0.665878 0Temp.=mild -2.09639 0Temp.=cool 2.071948 1Humidity -6.09779 1Windy 4.005785 1

Y = -4.108R = 0.016

Sigmoid Node = noInputs Weights UnknownThreshold 0.730992 1Outlook=sunny -3.00681 0Outlook=overcast 3.315451 1Outlook=rainy -1.08682 0Temp.=hot -0.66717 0Temp.=mild 2.092667 0Temp.=cool -2.07565 1Humidity 6.097738 1Windy -4.00574 1

Y = 4.063R = 0.983

New set of weather conditions: Outlook = overcast, Temperature = cool, Humidity = high, Windy = yes.

Prediction: Play = no

Importing model weights into Excel


Additional features of the multilayer

perceptron


How to stop training the network

Stop after 500

epochs

Stop after 500

epochsStop when error for validation set stops improving

Stop when error for validation set stops improving

Break dataset into: - training set, - test set, - validation set

Break dataset into: - training set, - test set, - validation set


Stopping training on validation set error


Stop onvalidationThreshold =

20


20

Stop ontrainingTime = 500

Stop ontrainingTime = 500


Layers and nodes: n = number of nodes in hidden layer n,n… = number of nodes in each hidden layer

Wildcards: i = attributes o = classes t = (attributes + classes) a = (attributes + classes)/2

1 hidden layer with (7/2) 3 nodes

1 hidden layer with (7/2) 3 nodes

Add some hidden layers!



20

0 hidden layers


20

0 hidden layers


20

1 hidden layerwith 3 nodes


20

1 hidden layerwith 3 nodes


Select GUI = TrueSelect GUI = True


Input nodesInput nodes

Hidden layer

Hidden layer

Output nodesOutput nodes

Weka bird hopping around

Weka bird hopping around


How to use the GUI to modify network structure.


Place cursor over node and

right-click.

Place cursor over node and

right-click.

Node is removedNode is

removed

Click Start to build networkClick Start to

build network

How to use the GUI to modify network structure.


Modify network

structure, then click Accept

Modify network

structure, then click Accept

GUI displays the Error per EpochGUI displays the Error per Epoch


Next Topic:

Genetic Algorithms


Weka Documentation:


Contact the Author:

Mark Polczynski, PhDThe Technology [email protected]

mailto:[email protected]

wt10

Documents