final year project report

APPLICATION OF NEURAL NETWORK IN FORECASTING

List of Abbreviations

ANN Artificial Neural Network

GWL Ground Water Level

RWH Rain Water Harvesting

h1 Hidden layer 1

h2 Hidden layer 2

NARX Nonlinear Autoregressive Exogenous Model

R Correlation Coefficient

MSE Mean Square Error

Department of Computer Science & Engg., FIET Bareilly


INTRODUCTION


CHAPTER 1


A prediction (Latin præ-, "before," and dicere, "to say") or forecast is a statement about the way

things will happen in the future, often but not always based on experience or knowledge. While

there is much overlap between prediction and forecast, a prediction may be a statement that some

outcome is expected, while a forecast is more specific, and may cover a range of possible

outcomes. Although guaranteed information about the future is in many cases impossible,

prediction is necessary to allow plans to be made about possible developments; Howard H.

Stevenson writes that prediction in business "... is at least two things: Important and hard."

Football (Soccer) has becoming increasingly popular over the years. During the last decade,

soccer’s biggest event, the FIFA World Cup, has attracted millions of fans worldwide; in fact, the

viewership of the FIFA World Cup match arguably surpasses the Super Bowl event as soccer is the

more widely played sport in the world. To showcase its international reach, the World Cup was

hosted in Japan and South Korea in 2002. South Africa hosted its first World Cup in 2010, and

Brazil will host the next World Cup in 2014. Even in the Middle East, soccer appeals to the

audience in that part of the world, with Qatar hosting the World Cup in 2022. The exposure of

soccer to different continents highlights the growing popularity for the sport that originated in

Europe.

With great viewership comes great opportunity for soccer viewers to bet on scores too. This is

especially true in Asia, where soccer betting is common among soccer fans. For instance, in

Singapore, the Singapore Pools, which is Singapore’s legalized gambling institution, allows

Singapore citizens to bet on almost anything, with soccer betting being one of its largest revenue

generators. These bets range from predicting the outcome of the score, the exact result of the score,

the winning margin and whether a certain player will score first. They are based on analyses

including whether a certain player is playing, the coach’s record against the opposition and the

strategies that the coach has been using. These factors are all based on human analyses with a tinge

of biasness in them, which is inevitable for any sport that involves human judgment.

In this experiment it will be shown how neural networks and Neuroph Studio are used when it

comes to problems of classification. Several architectures will be tried out, and it will be

determined which ones represent a good solution to the problem, and which ones do not.


1.1 An Overview

http://en.wikipedia.org/wiki/Future

http://en.wikipedia.org/wiki/Forecasting

http://en.wikipedia.org/wiki/Latin


1.2 Football : The Game

Football refers to a number of sports that involve, to varying degrees, kicking a ball with the foot

to score a goal. The most popular of these sports worldwide is association football, more

commonly known as just "football" or "soccer". Unqualified, the word football applies to

whichever form of football is the most popular in the regional context in which the word appears,

including association football, as well as football, Australian rules football, Canadian

football, Gaelic football, rugby league, rugby union, and other related games. These variations of

football are known as football codes.

Various forms of football can be identified in history, often as popular peasant games.

Contemporary codes of football can be traced back to the codification of these games at English

public schools in the eighteenth and nineteenth centuries. The influence and power of the British

Empire allowed these rules of football to spread to areas of British influence outside of the directly

controlled Empire, though by the end of the nineteenth century, distinct regional codes were

already developing: Gaelic Football, for example, deliberately incorporated the rules of local

traditional football games in order to maintain their heritage. In 1888, The Football League was

founded in England, becoming the first of many professional football competitions. During the

twentieth century, several of the various kinds of football grew to become among the most popular

team sports in the world.

1.3 Rules and facts of game

There are 17 laws in total, each one briefly summarized below.

Field of Play. The game can be played on either natural or artificial surfaces, the surface must be

green and rectangular in shape. The two long sides of the rectangle are called touch lines and the

two shorter sides are called goal lines. The field is divided in half by the halfway line.

Ball. Must be spherical, made of leather (or similar) 68-70 cm in circumference and of a certain

pressure.



Number of Players. Two teams of no more than 11 players (one of which is the goalkeeper). A

game cannot start if either team has less than 7 players. Players are divided into three

combination : Defender , midfielder , Forward player .

Equipment. Players must wear a jersey, shorts, stockings, shin guards and footwear.

Referee. The referee ensures the Laws of the Game are respected and upheld.

Assistant Referees. There may be at most 2 assistant referees.

Duration of the Match. The game is played in 2 halves consisting of 45 minutes each. The half

time interval must not exceed more than 15 minutes. At the discretion of the referee more time is

allowed to compensate for any stoppage during play e.g. Due to substitutions or care and attention

of injured players.

Start and Restart of Play. A kick-off starts play at the start of the match or after a goal. A kick-

off involves one player kicking the ball, from stationary, forward from the centre spot. All players

must be in their own half prior to kick-off. A coin is tossed pre-game, the team which loses the

toss are awarded the kick-off to start the game whilst the team that win the toss are allowed to

choose which direction they want to play. After half time the teams switch direction and the other

team will kick-off. After a goal is scored, the team which conceded the goal will kick-off to restart

play.

Ball in and Out of Play. The ball is out of play once a goal has been scored or when the referee

has stopped the game. The ball is in play at all other times.

Method of Scoring. The ball crosses the goal line inside the goal mouth.

Offside. It is an offence for a player to be in contact with the ball when they are closer to the

opponents' goal than both the ball and the second-last opponent. The offside rule exists to ensure

there are always opponents (generally the goal keeper and a defender) between a player receiving


APPLICATION OF NEURAL NETWORK IN FORECASTINGthe ball and the goal. Without the offside rule, play can become boring with repeated long balls

being kicked to a player stood next to the goalkeeper for an easy goal.

Fouls/Misconduct. These are many and varied, broadly speaking it is an offence to use excessive

force whilst playing the game either deliberately or undeliberately or to handle the ball (unless you

are a goal keeper). The referee may show the yellow card to caution players for less serious

offences and the red card for more serious offences resulting in the player being sent off. Two

yellow cards are equivalent to one red card.

Free Kicks. Are given by the referee for fouls and misconduct. A free kick can either be direct or

indirect. A goal can be scored directly from a direct free kick. A goal can only be scored from an

indirect free kick if it touches at least one other player first. The free kick must be taken from a

stationary position with that position varying depending on whether the free kick was given inside

or outside the goal area and whether it's direct or indirect. The opposing team must be a minimum

of 9.15 m from the ball when the free kick is taken.

Penalty Kicks. Are given against a team when they commit an offence which would normally be

awarded a direct free kick inside their goal area. The ball is kicked from stationary from the

penalty spot. The opposing team must be outside of the penalty area and at least 9.15 m from the

ball.

Throw-in. Used to restart play after the whole of the ball has crossed the touch line.

Goal kick. Used to restart play after a goal has been scored.

Corner Kick. Is given when the whole of the ball crosses the goal line and was last touched by a

member of the defending team (and no goal was scored). A corner kick is taken from inside the

corner arc closest to the point where the ball crosses the goal line. The defending team must be at

least 9.15 m from the ball when the corner kick is taken.



1.4 Role of prediction in Football

Predicting the outcome of sporting events is a business which has grown in popularity in recent

years. Handicappers predict the outcome of games using a variety of mathematical formulas,

simulation models or qualitative analysis. Early, well known sports bettors, such as Jimmy the

Greek, were believed to have access to information that gave them an edge. Information ranged

from personal issues, such as gambling or drinking to undisclosed injuries; anything that may

affect the performance of a player on the field.

Recent times have changed the way sports are predicted. Predictions now typically consist of two

distinct approaches: Situational plays and statistical based models. Situational plays are much more

difficult to measure because they usually involve the motivation of a team. Dan Gordon, noted

handicapper, wrote “Without an emotional edge in a game in addition to value in a line, I won’t

put my money on it”. These types of plays consist of: Betting on the home underdog, betting

against Monday Night winners if they are a favorite next week, betting the underdog in “look

ahead” games etc. As situational plays become more widely known they become less useful

because they will impact the way the line is set.

The widespread use of technology has brought with it more modern sports betting systems. These

systems are typically algorithms and simulation models based on regression analysis. Jeff Sagarin,

a sports statistician, has brought attention to sports by having the results of his models published in

USA Today. He is currently paid as a consultant by the Dallas Mavericks for his advice on lineups

and the use of his Winval system, which evaluates free agents. Brian Burke, a former Navy fighter

pilot turned sports statistician, has published his results of using regression analysis to predict the

outcome of NFL games. Ken Pomeroy is widely accepted as a leading authority on college

basketball statistics. His website includes his College Basketball Ratings, a tempo based statistics

system. Some statisticians have become very famous for having successful prediction systems.

Dare wrote “the effective odds for sports betting and horse racing are a direct result of human

decisions and can therefore potentially exhibit consistent error”. Unlike other games offered in a

casino, prediction in sporting events can be both logical and consistent

In short we can say that we can use prediction techniques to find out best combination for any

team by predicting individual performance of players and the total effect for their performances in


http://en.wikipedia.org/wiki/Ken_Pomeroy

http://en.wikipedia.org/wiki/Navy

http://en.wikipedia.org/wiki/Advanced_NFL_Stats

http://en.wikipedia.org/wiki/Dallas_Mavericks

http://en.wikipedia.org/wiki/Jeff_Sagarin

http://en.wikipedia.org/wiki/Regression_analysis

http://en.wikipedia.org/wiki/Sports_betting_systems

http://en.wikipedia.org/wiki/Jimmy_the_Greek

http://en.wikipedia.org/wiki/Jimmy_the_Greek

http://en.wikipedia.org/wiki/Qualitative_research

APPLICATION OF NEURAL NETWORK IN FORECASTINGthe team .Using this process we can select best combination for any playing condition and

opponent easily and effectively.

1.5 Neural Network

1.5.1 Biological neural networks

A neuron ( or nerve cell ) is a special biological cell that processes information (see Figure

1.8). It is composed of a cell body, or soma, and two types of out-reaching tree-like

branches: the axon and the dendrites. The cell body has a nucleus that contains information

about hereditary traits and plasma that holds the molecular equipment for producing material

needed by the neuron. A neuron receives signals (impulses) from other neurons through its

dendrites (receivers) and transmits signals generated by its cell body along the axon

(transmitter), which eventually branches into strands and sub strands. At the terminals of

these strands are the synapses. A synapse is an elementary structure and functional unit

between two neurons (an axon strand of one neuron and a dendrite of another), When the

impulse reaches the synapse’s terminal, certain chemicals called neurotransmitters are

released. The neurotransmitters diffuse across the synaptic gap, to enhance or inhibit,

depending on the type of the synapse, the receptor neuron's own tendency to emit electrical

impulses. The synapse's effectiveness can be adjusted by the signals passing through it so that

the synapses can learn from the activities in which they participate. This

dependence on history acts as a memory, which is possibly responsible for human

memory [3].



Figure1.1 Biological Neuron [3]

The cerebral cortex in humans is a large flat sheet of neurons about 2 to 3 millimeters

thick with a surface area of about 2,200 cm2, about twice the area of a standard

computer keyboard. The cerebral cortex contains about 10" neurons, which is

approximately the number of stars in the Milky Way." Neurons are massively connected,

much more complex and dense than telephone networks. Each neuron is connected to

103 to l04 other neurons.

In total, the human brain contains approximately 1014 to l015 interconnections.

Neurons communicate through a very short train of pulses, typically milliseconds in

duration. The message is modulated on the pulse-transmission frequency. This frequency

can vary from a few to several hundred hertz, which is a million times slower than the

fastest switching speed in electronic circuits. However, complex perceptual decisions such

as face recognition are typically made by humans within a few hundred milliseconds.

These decisions are made by a network of neurons whose operational speed is only a few

milliseconds. This implies that the computations cannot take more than about 100 serial

stages. In other words, the brain runs parallel programs that are about

100 steps long for such perceptual tasks. This is known as the hundred step rule.12 The

same timing considerations show that the amount of information sent from one neuron

to another must be very small (a few bits). This implies that critical information is not

transmitted directly, but captured and distributed in the interconnections-hence the

name, connectionist model, used to describe A’s .



1.5.2 Artificial Neural Network

Artificial neural network is a biologically inspired mathematical tool used widely

for prediction and classification. In basic Artificial neural network there are three

layers of artificial neurons given below:

1) Input layer: The function of this layer is to take input.

2) Hidden layer: This layer process that input as per the given activation function.

3) Output layer: After processing by hidden layer output layer provides the

output.

Figure 1.2 Artificial neural network

An Artificial neuron is a processing unit which process the input. The input is taken by

a input vector x. Each edge connecting two neurons have some weight Wij shows

the connection strength between two neurons. The weights are given in weight matrix W.



1.5.3 Models of a neuron

A neuron is an information-processing unit that is fundamental to the operation of a

neural network. We may identify three basic elements of the neuron model:

Figure 1.3 Nonlinear model of a neuron

1. A set of synapses, each of which is characterized by a weight or strength of its own.

Specifically, a signal by the synaptic weight at the input of synapse j connected to neuron k is

multiplied. It is important to make a note of the manner in which the subscripts of the synaptic

weight are written. The first subscript refers to the neuron in question and the second subscript

refers to the input end of the synapse to which the weight refers. The weight is positive if the

associated synapse is excitatory; it is negative if the synapse is inhibitory.

2. An adder for summing the input signals, weighted by the respective synapses of the neuron.


APPLICATION OF NEURAL NETWORK IN FORECASTING3. An activation function for limiting the amplitude of the output of a neuron. The activation

function is also referred to in the literature as a squashing function in that

it squashes (limits) the permissible amplitude range of the output signal to some finite value.

Typically, the normalized amplitude range of the output of a neuron is written as the closed unit

interval [0, 1] or alternatively [-1, 1].

4. The model of a neuron also includes an externally applied bias (threshold) wk0 = bk that has the

effect of lowering or increasing the net input of the activation function.

In mathematical terms, we may describe a neuron k by writing the following pair of equations:

or in a matrix form

Types of activation functions [3]

a) Threshold activation function (McCulloch–Pitts model)



Figure 1.4 Threshold activation function.

In this model, the output of a neuron takes on the value of 1 if the total internal activity

level of that neuron is nonnegative and 0 otherwise. This statement describes the all-

or-none property of the McCulloch–Pitts model.

The McCulloch–Pitts model of a neuron is sometimes used with the output either -1 or +1:

b) Piecewise-linear activation function

Figure 1.5 Piecewise-linear activation function



The amplification factor inside the linear region is assumed to be unity. The following

two situations may be viewed as special forms of the piecewise linear function:

1. A linear combiner arises if the linear region of operation in maintained without

running into saturation.

2. The piecewise-linear function reduces to a threshold function if the amplification factor

of the linear region in made infinitely large.



c) Sigmoid (logistic) activation function

Figure 1.6 Sigmoid (logistic) activation functions.

The sigmoid function is the most common form of activation function used in the

construction of artificial neural networks. Whereas a threshold function assumes the value of

0 or 1, a sigmoid function assumes a continuous range of values form 0 and 1. Note also that

the sigmoid function is differentiable, which is an important feature of neural network theory.

The derivative of the sigmoid function has a nice property, which makes is easy to calculate:



d) Hyperbolic tangent function

Figure 1.7 Hyperbolic tangent activation function

The hyperbolic tangent function can be easily expressed in terms of the logistic function: (2 ×

logistic function – 1). Its derivative is also easy to calculate:

e) Softmax activation function

One approach toward approximating probabilities is to choose the output neuron nonlinearity

to be exponential rather than sigmoidal and for each pattern to normalize the outputs to sum

to 1. Let c be the number of output neurons. Each output is generated by the following

activation function:

It is clear that the softmax activation function is a smoothed version of a winner-

take-all nonlinearity in which the maximum output is transformed to 1, and all others

reduced to 0.

The use of softmax is appropriate when the network is to be used for estimating probabilities.

For example, each output yi can represent a probability that the corresponding neural network


APPLICATION OF NEURAL NETWORK IN FORECASTINGinput belongs to class Ci.



1.5.4 Multilayer feedforward network

Figure 1.8 A 10-4-2 fully connected feedforward network.

The source nodes in the input layer of the network supply respective elements of the

activation pattern (input vector), which constitute the input signals applied to the neurons

(computation nodes) in the second layer (i.e. the first hidden layer). The output signals of the

second layer are used as inputs to the third layer, and so on for the rest of the network.

Typically, the neurons in each layer of the network have as their inputs the output signals of

the preceding layer only. The set of output signals of the neurons in the output layer of the

network constitutes the overall response of the network to the activation pattern supplied by

the source nodes in the input layer. For brevity, the network shown above is referred to as a

10-4-2 network in that it has 10 source nodes, 4 hidden neurons, and 2 output neurons. As

another example, a feed forward network with p source nodes, h1 neurons in the first

hidden layer, h2 neurons in the second layer, and q neurons in the output layer is referred to

as a p-hl-h2-q network [3].

A neural network is said to be fully connected if every node in each layer of the network is

connected to every other node in the adjacent forward layer. If, however, some of the

communication links (synaptic connections) are missing from the network, we say that the

network is partially connected.



1.6 Learning

The ability to learn is a fundamental trait of intelligence. Although a precise definition

of learning is difficult to formulate, a learning process in the ANN context can be viewed as

the problem of updating network architecture and connection weights so that a network

can efficiently perform a specific task. The network usually must learn the connection

weights from available training patterns. Performance is improved over time by iteratively

updating the weights in the network. ANNs' ability to automatically learn from examples

makes them attractive and exciting. Instead of following a set of rules specified by human

experts, ANNs appear to learn underlying rules (like input-output relationships) from the

given collection of representative examples. This is one of the major advantages of

neural networks over traditional expert systems. To understand or design a learning

process, you must first have a model of the environment in which a neural network

operates, that is, you must know what information is available to the network. We refer

to this model as a learning paradigm. Second, you must understand how network weights

are updated, that is, which learning rules govern the updating process. A learning algorithm

refers to a procedure in which learning rules are used for adjusting the weights .

There are three main learning paradigms: supervised, unsupervised, and hybrid. In

supervised learning, or learning with a “teacher,” the network is provided with a correct

answer (output) for every input pattern. Weights are determined to allow the network to

produce answers as close as possible to the known correct answers. Reinforcement learning is

a variant of supervised learning in which the network is provided with only a critique on the

correctness of network outputs, not the correct answers themselves. In contrast, unsupervised

learning, or learning without a teacher, does not require a correct answer associated with each

input pattern in the training data set. It explores the underlying structure in the data, or

correlations between patterns in the data, and organizes patterns into categories from these

correlations. Hybrid learning combines supervised and unsupervised learning. Parts of the

weights are usually determined through supervised learning, while the others are obtained

through unsupervised learning.

Learning theory must address three fundamental and practical issues associated with learning

from samples: capacity, sample complexity, and computational complexity.

Capacity concerns how many patterns can be stored, and what functions and decision


APPLICATION OF NEURAL NETWORK IN FORECASTINGboundaries a net-work can form. , Sample complexity determines the number of training

patterns needed to

train the network to guarantee a valid generalization. Too few patterns may cause “over-

fitting” (where in the network performs well on the training data set, but poorly on

independent test patterns drawn from the same distribution as the training patterns).

Computational complexity refers to the time required for a learning algorithm to estimate

a solution from training patterns. Many existing learning algorithms have high

computational complexity. Designing efficient algorithms for neural network learning is

a very active research topic.

1.6.1 Backpropogation learning

Consider a multilayer feed forward network, such as the three-layer network of Figure.

Figure 1.9 Three layer feedforward network

The net input to unit i in layer k + 1 is

(2.1)

The output of unit i will be

(2.2)

For an M layer network the system equations in matrix form are given by

(2.3)



(2.4)

The task of the network is to learn associations between a specified set of input-output pairs

The performance index for the network is

(2.5)

Where is the output of the network when the qth input, is presented, and

is the error for the qth input. For the standard backpropagation algorithm

we use an approximate steepest descent rule. The performance index is approximated by

(2.6)

where the total sum of squares is replaced by the squared errors for a single input/output

pair. The approximate steepest (gradient) descent algorithm is then

(2.7)

(2.8)

where is the leaming rate. Define

(2.9)


APPLICATION OF NEURAL NETWORK IN FORECASTINGas the sensitivity of the performance index to changes in the net input of unit i in layer k.

Now it can be shown, using (2.l), (2.6), and (2.9), that

(2.10)

(2.11)

It can also be shown that the sensitivities satisfy the following recurrence relation

(2.12)

Where

(2.13)

And

(2.14)

This recurrence relation is initialized at the final layer

(2.15)

The overall learning algorithm now proceeds as follows; first, propagate the input forward

using (2.3)-(2.4); next, propagate the sensitivities back using (2.15) and (2.12); and finally,

update the weights and offsets using (2.7), (2.8), (2.l0), and (2.11).

1.6.2 Marquardt-Levenberg modification

While backpropagation is a steepest descent algorithm, the Marquardt-Levenberg algorithm

is an approximation to Newton’s method. Suppose that we have a function V( ) which


APPLICATION OF NEURAL NETWORK IN FORECASTINGwe want to minimize with respect to the parameter vector I, then Newton's method would

be

(2.16)

Where is the Hessian matrix and is the gradient. If we assume that

is a sum of squares function

(2.17)

then it can be shown that

(2.18)

(2.19)

where J(x) is the Jacobian matrix

(2.20)And

(2.21)

For the Gauss-Newton method it is assumed that and the update (3.16) becomes

(2.22)

The Marquardt-Levenberg modification to the Gauss-Newton method is

(2.23)


APPLICATION OF NEURAL NETWORK IN FORECASTINGThe parameter is multiplied by some factor ( ) whenever a step would result in an

increased . When a step reduces , is divided by . Notice that when is

large the algorithm becomes steepest descent (with step ), while for small the

algorithm becomes Gauss-Newton. The Marquardt-Levenberg algorithm can be considered a

trust- region modification to Gauss-Newton.

The key step in this algorithm is the computation of the Jacobian matrix. For the

neural network mapping problem the terms in the Jacobian matrix can be computed by a

simple modification to the backpropagation algorithm. The performance index for the

mapping problem is given by (3.5). It is easy to see that this is equivalent in form to

(3.17), where

and N = Q x SM. Standard backpropagation calculates terms like

(2.24)

For the elements of the Jacobian matrix that are needed for the Marquardt algorithm we

need to calculate terms like

(2.25)

These terms can be calculated using the standard backpropagation algorithm with one

modification at the final layer

(2.26)

Note that each column of the matrix in (3.26) is a sensitivity vector which must be

backpropagated through the network to produce one row of the Jacobian.



1.7 Project Objective

Prediction is very useful thing for games .Here we are predicting the result of a football match.

Artificial neural network provide the better prediction result comparing other techniques.

Objective of this research work is given as:



PROJECT LITERATURE


CHAPTER 2


2.1 Literature Survey

A number of studies have been done on different forecasting. Various data mining

techniques have been used in these studies for prediction, and the calculated results are too

much close to the real values. Heesung Yoon et al. [8] done their study at coastal aquifer and

made a comparison between artificial neural network and support vector machine for

prediction. The results of the model performance show that root mean squared error

(RMSE) values of ANN models are lower than those of SVM in model training and

testing stages. However, the overall model performance criteria of the SVM are similar to or

even better than those of the ANN in model prediction stage.

Radha Krisna Balla [5] uses feed forward backpropogation artificial neural network for

result prediction at English Premier League in 2006-07. The model provided the good

fit and the predicted trend followed the observed data closely 65% correctness .This

proves the artificial neural network can be used for football match result prediction.

Monica Adya And Fred Collopy perform 48 studies , only eleven met all of criteria for

effectiveness of validation and implementation. Of the remaining 38, 17 presented effective

validations but suffered with respect to implementation. Eleven of these reported positive

results despite implementation problems. Altogether then, of the 48 studies, 22 contributed to

our knowledge regarding the applicability of NNs to forecasting and prediction. Nineteen

(86%) of these produced results that were favour-able, three produced results that were not.,

NNs, when they are effectively implemented and validated, show potential for forecasting and

prediction. Second, a significant portion of the NN research in forecasting and prediction lacks

validity.

Ahmad Abrishamchi et al. [7] investigate the capability of two ANN models to predict the

urban groundwater level using different sets of available input data, and then we compare the

results of these two models. A multi-input-single-output network has been trained using

Levenberg-Marquardt algorithm. The aforementioned models are evaluated using three

statistical performance criteria namely mean square error (MSE), root mean squared error

(RMSE), and efficiency (R2). This paper identifies the proficiency of ANN modeling

technique to capture the complex dynamics involved in urban groundwater level fluctuations.

The results show the importance of input data selection and its effect on prediction accuracy.



Sreenivasulu D and Deka P.C. [8] investigate the potential and applicability of Radial Basis

Function (RBF) for forecasting groundwater level with limited data. The results showed that

ANN with RBF network can be an effective tool for better groundwater level fluctuations

forecasting with limited data.

P. Sujatha1 and Dr. G.N. Pradeep Kumar [10] examined Performance of four types of

functionally different artificial neural network (ANN) models, namely Feed forward neural

network, Elman type recurrent neural network, Input delay neural network and Radial basis

function network and fourteen types of algorithms, namely Batch gradient descent (traingd),

Batch gradient descent with momentum (traingdm), Adaptive learning rate (traingda),

Adaptive learning rate with momentum (traingdx), Resilient backpropagation (trainrp),

Fletcher-Reeves update (traincgf), Polak-Ribiere update (traincgp), Powell-Beale restarts

(traincgb), Scaled conjugate gradient (trainscg), BFGS algorithm (trainbfg), One step

secant algorithm (trainoss), Levenberg-Marquardt (trainlm), Automated regularization

(trainbr) and Random order incremental training (trainr) were examined in order to

identify an efficient ANN architecture and algorithm that can simulate the water table

fluctuations using a relatively short length of groundwater level records. Results showed

that Feed forward neural network trained with training algorithm Levenberg-Marquardt

is suitable for accurate prediction of groundwater levels.

Ioannis N. Daliakopoulos et al. [11] investigate and compare seven different types of network

architectures and training algorithms in terms of model prediction efficiency and accuracy.

The different experiment results show that accurate predictions can be achieved with a

standard feedforward neural network trained with the Levenberg-Marquardt algorithm

providing the best results for up to 18 months forecast.

From all these studies we can conclude that feedforward neural network with Levenberg-

Marquardt training algorithm shown to be a promising tool for ground water level prediction

even with limited amount of data.



PROPOSED METHODOLOGY

& SIMULATION


CHAPTER 3


3.1 Proposed Methodology

Any result of football match mainly based on performance of participating teams. This

performance should be equal or best in different roles and areas of the game. For best

performance of any team their players should be fit and inform .For this purpose teams are

managing the performance and fitness data of the players and they evaluate this data to

select right person for right place in the team. Then they forecast the performance of the

team using historic data of players in different matches.

It was a old approach to use only evaluation of the individual performance of any player

here we does not relate the effect of individual performance in the team performance. So the

prediction of the match was very tricky because we were not used the overall effect of

performance by a player of the team, a department of the team (e.g. defense, forward,

midfielder, goalkeeper). It is very easy to maintain the data of performance but to relate the

data in the context of performance is difficult in different tools. We can use curve fitting and

other different statics tool but these tools are failed to maintain the best error free result.

Many models are based on mathematical calculations with dynamic programming, mainly in

these calculation they use some mathematical concept such as probability with dynamic way

and they predict result dynamically. For example WASP (Winning and Score Prediction) is

a technique which is used in cricket at New Zealand for prediction of score and winning

percentage of any team during the match.

In this model we include the performance rating of team in four main parts of team. In these

parts we include defense, goalkeeping, midfielder and forward. we collect the historic data

of performance of players and final result of match in season 2013 to 2014 in Barclays

premier league in match by match manner we randomly choose 106 matches .Then all these

data sets are implemented in training of neural network with performance of players in

different sectors as input and possible result of match as output. Then we can provide some

test set for known matches and if predicted result is same as real then model is ready to use.



3.2 Designing ANN models

Designing ANN models follows a number of systemic procedures. In general, there are

five basics steps: (1) Collecting data, (2) Preprocessing data, (3) Building the network, (4)

Train, and (5) test performance of model as shown in Figure.

Collecting data

Preprocessing data

Building the network

Train

Test Performance

Figure 3.1 Designing ANN model

Data Collection

Collecting and preparing sample data is the first step in designing ANN

models.as input. First we need data set. For this problem we choose result of Premier League

season 2011-12 . Because of great number of matches we randomly sampled 106 results.

Each result has 8 input and 3 output attributes. Input attributes are:

1. Home team goalkeeper rating

2. Home team defence rating

3. Home team midfield rating

4. Home team attack rating

5. Visitor team goalkeeper rating

6. Visitor team defiance rating

7. Visitor team midfield rating

8. Visitor team attack rating



Output attributes are:

1. Home team wins

2. Draw

3. Visitor team wins

Data pre-processing

After data collection, three data preprocessing procedures are conducted to train the

ANNs more efficiently. These procedures are:

a. Solve the problem of missing data,

b. Normalize data

c. Randomize data.

The missing data are replaced by the average of neighboring values during the same year.

Normalization procedure before presenting the input data to the network is generally a

good

practice, since mixing variables with large magnitudes and small magnitudes will

confuse the learning algorithm on the importance of each variable and may force it

to finally reject the variable with the smaller magnitude.

All input attributes are have integer values that can be very distant from each other. For

example, goalkeeper rating can have maximum value of 100, and attack rating can have

maximum of 300. In that case attack rating will influence more on problem than goalkeeper.

To prevent that we will normalize data set using Max-Min normalization formula.

B = (A - min(A)) / (max(A) - min(A)) * ( D - C ) + C

Where B is the standardized value, and D and C determines the range in which we want our

value to be. In this case, D = 1 and C = 0.

Normalized values are saved in PremierLeagueResults.txt file because they will be used for

training and testing neural network



Building the network

At this stage, the designer specifies the number of hidden layers, neuron in each

layer,

transfer function, weight/bias learning function, and performance function. In this

work multilayer feedforward network is used with Levenberg-Marquardt learning

algorithm.

Training the network

In order to neural network learn the problem we need traaining data set. The training data

set consists of input signals assigned with corresponding target (desired output). The

neural network is then trained using one of the supervised learning algorithms, which

uses the data to adjust the network's weights and thresholds so as to minimize the error in

its predictions on the training set. If the network is properly trained, it has then learned to

model the (unknown) function that relates the input variables to the output variables, and

can subsequently be used to make predictions where the output is not known.

In general, if you use a neural network, you will not know the exact nature of the

relationship between inputs and outputs – if you knew the relationship, you would model

it directly. The other key feature of neural networks is that they learn input/output

relationship through training. There are two types of training used in neural networks,

with different types of networks using different types of training. These are supervised

and unsupervised training, of which supervised is the most common. In supervised

learning, the network user assembles a set of training data. The training data contains

examples of inputs together with the corresponding outputs, and the network learns to

infer the relationship between the two. In other words, supervised learning is used for

classification. For an unsupervised learning rule, the training set consists of input training

patterns only. Unsupervised learning, on other hand, is used for clustering.

Training set can be created in two ways. You can either create training set by entering

elements as input and desired output values of neurons in input and output label, or you

can create training set by choosing an option load file. The first method of data entry is

time consuming, and there is also a risk to make a mistake when entering data. Since we

already have training set we will choose second way.



Testing the network

The next step is to test the performance of the developed model. At this stage unseen data are

exposed to the model. 147 samples are given to the network for the testing and evaluate the

performance. In order to evaluate the performance of the developed ANN model quantitatively

and verify whether there is any underlying trend in performance of ANN models, statistical

analysis involving the correlation coefficient (R), the Mean square error (MSE) were

conducted.

3.3 Creating the ANN model

In this model we used the Neuroph Studio which is Java Neural Network development

environment on top of NetBeans Platform based on Neuroph framework. It has been licensed

under the Common Development and Distribution License (CDDL) . we are using Neuroph

Studio 201207171143 (version 7.5) and java version 1.7 ,Java HotSpot(TM) Server VM 11.0 .

Complete procedure of the project is given below –

a. After normalizing all data we can start with Neuroph Studio. First we will

create new Neuroph project then a new project is created and it will appear in

the 'Projects' window, in the top left corner of Neuroph studio .

b. In order to neural network learn the problem we need training data set. The

training data set consists of input signals assigned with corresponding target

(desired output). The neural network is then trained using one of the

supervised learning algorithms, which uses the data to adjust the network's

weights and thresholds so as to minimize the error in its predictions on the

training set. If the network is properly trained, it has then learned to model the

(unknown) function that relates the input variables to the output variables, and

can subsequently be used to make predictions where the output is not known.

c. In general, if you use a neural network, you will not know the exact nature of

the relationship between inputs and outputs – if you knew the relationship,

you would model it directly. Training set can be created in two ways. You

can either create training set by entering elements as input and desired output



values of neurons in input and output label, or you can create training set by

choosing an option load file. The first method of data entry is time consuming,

d. and there is also a risk to make a mistake when entering data. Since we

already have training set we will choose second way

e. After the network is trained, we click 'Test', in order to see the total error, and

all the individual errors with respect to our training sets.

f. To predict the future value of any match we have to create a specific test set in

specified manner (same number and format of input and output variables as in

training data set )

Some pictures are given below that are showing the process



Figure 3.2 Creation of neuroph project

Figure 3.3 Creation of training set



Figure 3.4 training set with data



Figure 3.5 Learning parameters before training



Figure 3.6 network error graph



Figure 3.7 weights for different neurons for a training set



Figure 3.8 weights histogram for different neurons



3.4 Equivalent Java code for model

package predictingtheresultoffootballmatch;

import java.io.FileNotFoundException;

import java.io.IOException;

import java.util.Arrays;

import org.neuroph.core.NeuralNetwork;

import org.neuroph.core.learning.DataSet;

import org.neuroph.core.learning.DataSetRow;

import org.neuroph.nnet.MultiLayerPerceptron;

import org.neuroph.nnet.learning.MomentumBackpropagation;

import org.neuroph.util.TrainingSetImport;

import org.neuroph.util.TransferFunctionType;

public class PredictingTheResultofFootballMatch {

/* @param args the command line arguments */

public static void main(String[] args) {

String trainingSetFileName = "PremierLeagueResults.txt";

int inputsCount = 8;

int outputsCount = 3;

System.out.println("Running Sample");

System.out.println("Using training set " + trainingSetFileName);

// create training set

DataSet trainingSet = null;


APPLICATION OF NEURAL NETWORK IN FORECASTING try {

trainingSet = TrainingSetImport.importFromFile(trainingSetFileName, inputsCount,

outputsCount, ",");

} catch (FileNotFoundException ex) {

System.out.println("File not found!");

}

catch (IOException | NumberFormatException ex) {

System.out.println("Error reading file or bad number format!");

} // create multi layer perceptron

System.out.println("Creating neural network");

MultiLayerPerceptron neuralNet = new

MultiLayerPerceptron(TransferFunctionType.SIGMOID, 8, 30, 3);

// set learning parametars

MomentumBackpropagation learningRule = (MomentumBackpropagation)

neuralNet.getLearningRule();

learningRule.setLearningRate(0.3);

learningRule.setMomentum(0.7);

// learn the training set

System.out.println("Training neural network...");

neuralNet.learn(trainingSet);

System.out.println("Done!");

// test perceptron

System.out.println("Testing trained neural network");

testPredictingTheResultofFootballMatch(neuralNet, trainingSet);

}

public static void testPredictingTheResultofFootballMatch(NeuralNetwork nnet, DataSet

dset) {

for (DataSetRow trainingElement : dset.getRows()) {

nnet.setInput(trainingElement.getInput());


APPLICATION OF NEURAL NETWORK IN FORECASTING nnet.calculate();

double[] networkOutput = nnet.getOutput();

System.out.print("Input: " + Arrays.toString(trainingElement.getInput()));

System.out.println(" Output: " + Arrays.toString(networkOutput));

}

}

}

RESULT AND INTERPOLATION


CHAPTER 4


This chapter presents the best achieved results for feedforward backpropogation network

model. The network has 8 neurons in input layer, 2 to 25 neurons in hidden layer and 3 s

neuron for the predicted match result output. There are 106 samples are given to the network,

which divided by tool in 3 parts, training, validation and testing. After the completion of

training the Performance graph is given as:

Figure 4.1 network error graph for training set

The results are very impressive for the mean square error, and same as for the regression

coefficient as shown in the graphs



Figure 4.2 testing result for a testset



CONCLUTION AND FUTURE WORK


CHAPTER 5


5.1 Conclusion

In this model we simply create a architecture that can predict the result of a football match

(best result for Barclays premier League) . As we know that we can forecast and predict the

future values using neural network techniques and historical data . So we used the same

concept and created a simple model for forecasting of match result , to achieve this we collect

historical data ( performance rating of players for specific match with result of match ) from

genius website of premier league ( Barclays Premier League ) . After data collection we

proceed the preprocessing of data in this data cleaning, data normalization etc takes place .

Normalized data set was used to train the network algorithm , for this we create several

network architectures and and perform training to each architecture then after training and

testing of each network architecture we check the performance of each architecture .

Performance mainly based on minimum error (MSE).

After these steps we can say that which architecture gave us the best result, so we select that

best architecture and perform prediction for any upcoming match in the league. We can use

same architecture for other leagues and matches but the accuracy of the prediction may be

different from specified league.



Neural network is very healthy area for forecasting or prediction. Neural network contains several

different algorithms and techniques to forecast the values in various fields. Different techniques are

based on historical data and different algorithms of training can easily forecast the values. We know

that each technique of network needs data in its manner ( i.e. some format) for perfect prediction

and minimum error.

In this model we use 8 input variables which are mainly based on performance of players of both

teams but we know that the performance of player not only depends upon their rating, it also

depends on the place (match venue) , fitness of player ,weather , strategy of opposition etc . As we

know that we use historical data for these attributes so it is difficult to collect the data for above

given attributes from historic data but if we start collection of data of these attributes in present

then we can easily and effectively create a perfect forecasting or prediction model in future .

It is possible to create a model that can give best result for any match and any team, location,

situation with historical data of different attributes. We also know that for perfect result also based

on training algorithm and network architecture so we can say that modification and creation of new

algorithm and architecture can also change the result and can give best result . So we can perform

research work on it in future for various fields where forecasting and prediction can be applied.


5.2 Future work



REFERENCESE


[1] Kahn, J. (2003) \Neural Network Prediction of NFL Football Games, “World Wide Web

electronic publication, homepages.cae.wisc.edu/~ece539/project/f03/kahn.pdf.

[2] Official website of Barclays Premier league ,for dats collection ,Available:

http://www.premierleague.com/en-gb/matchday/results.html?paramComp_100=true&view

=.dateSeason, accessed on 5 April, 2014.

[3] M. Hajek, “Neural Networks” 2005, Available:

www.sciag.ukzn.ac.za/comp/notes/NeuralNetworks2005.pdf accessed on 4 May 2014

[4] Martin T. Hagan and Mohammad B. Menhaj “Training Feedforward Networks with the

Marquardt Algorithm”, IEEE Transactions on Neural Networks, Vol. 5, No.6, November

1994, pp 989-993.

[5] Alan McCabe, "An Artificially Intelligent Sports Tipper," in Proceedings : 15th Australian

Joint Conference on Artificial Intelligence (2002)

[6] A. P. Rotshtein, M. Posner and A. B. Rakityanskaya, “Football Predictions Based on a

Fuzzy Model with Genetic and Neural Tuning," Cybernetics and Systems Analysis Journal

(2005)

[7] P. D. Sreekanth, N. Geethanjali, P. D. Sreedevi, Shakeel Ahmed, N. Ravi Kumar and P. D.

Kamala Jayanthi “Forecasting groundwater level using artificial neural networks” , Current

science, vol. 96, no. 7, 10 april 2009.

[8] David, John A.; Pasteur, R. Drew; Ahmad,M. Saif; and Janning, Michael C. (2011)

"NFL Prediction using Committees of Arti_cial Neural Networks," Journal ofQuantitative

Analysis in Sports: Vol. 7: Iss.2, Article 9.

[9] ThePredictionTracker, (2007), “FootballPredictions", World Wide Web electronic

publication,www.thepredictiontracker.com

[10] Howard Demuth, Mark Beale, “Neural Network Toolbox, For use with Matlab” , User’s

Guide, Version 4, The MathWorks



[11] V. Nourani, A. Hosseini Baghanam, F. Daneshvar Vousoughi, M.T. Alami

“Classification of groundwater level data using som to develop ann-based forecasting model” ,

International Journal of Soft Computing and Engineering (IJSCE), ISSN: 2231-2307,

Volume-2, Issue-1, March 2012.

[12] Heesung Yoon, Seong-Chun Jun, Yunjung Hyun , Gwang-Ok Bae, Kang-Kun Lee, “A

comparative study of artificial neural networks and support vector machines for predicting

groundwater levels in a coastal aquifer”, Journal of Hydrology 396 (2011), pp 128–138.

[13] Faridah Othman and Mahdi Naseri “Reservoir inflow forecasting using artificial neural

network”, International Journal of the Physical Sciences Vol. 6(3), , 4 February, 2011, pp.

434-440.

[14] Purna C. Nayak,Y.R. Satyaji Rao and K. P. Sudheer “Groundwater Level Forecasting in

a Shallow Aquifer Using Artificial Neural Network Approach”, Water

Resources Management (2006) 20: pp 77–90.

[15] http://www.mathworks.in/matlabcentral/fileexchange/19997-neural-network-for-pattern-

recognition-tutorial/content/NN_tutorial.m, accessed on 11 May 2013.

[16] Chi Leung Patrick Hui. (11 April, 2011). “Artificial Neural Networks – Application”,

ISBN 978-953-307-188-6, InTec.


final year project report

Technology

football soccer

rules of football

game football

football codes

gaelic football

word football

football league

association football