CS-534 Artificial Intelligence ShowcaseNeural Networks: ASAP and 20Q

Andrew FoltanWPI CS534

Neural Networks

• Traditional computers are von-Neumann machines– suited to solving problems using well defined

algorithms– input data must be precise

• Neural Networks– well suited to situations with no clear algorithmic

solutions– predicts solutions based on imprecise input data

NN Applications

• Business– Stocks prediction

• Health– Breast cancer cell analysis

• Sports– Horse racing

• Science– Speech recognition– Weather forecasting– Solar Flare prediction

Neural Network Historical SummaryYear Event

1943 Simple neural network modeled using electrical circuits. Neurophysiologist Warren McCulloch and mathematician Walter Pitts

1950’s Simulation of a hypothetical neural network. Nathanial Rochester from the IBM research laboratories, first attempt failed.

1959 "ADALINE" and "MADALINE” models developed by Stranford. Bernard Widrow and Marcian Hoff

1962 Widrow & Hoff develop a learning procedure based on adjusting weight values1960’s Exaggeration of the potential of neural networks leads to decline in research and funding1972 Kohonen and Anderson developed a similar network independently, an array of analog ADALINE

circuits1975 The first multilayered network was developed1982 John Hopfield of Caltech proposes using bidirectional neurons.

A joint US-Japan conference on Cooperative/Competitive Neural Networks1986 Widrow-Hoff rule extended to multiple layers. Becomes known as back-propagation networks1990’s A.K.Dewdney, former Scientific American columnist, "Although neural nets do solve a few toy

problems, their powers of computation are so limited that I am surprised anyone takes them seriously as a general problem-solving tool."

2000’s Research on integrated circuits optimized for the application of neural networksPresent Recurrent neural networks and deep feedforward neural networks focused on pattern recognition

and machine learning.

Automated Solar Activity Predictor• Automated real-time prediction of Solar flares using image processing and

machine learning- based system– Near-real time detection and classification of sunspot groups– Forecasts solar flares and their intensity using Neural Networks

• Why? – Geomagnetic storms– Communications disruption– Satellite damage

• University of Bradford (UOB), Space Weather Research Group– 2007-2009, Rami Qahwaji / Tufan Colak– http://spaceweather.inf.brad.ac.uk/

NN Learning• Input Data Set

– National Geophysical Data Center (NOAA) publicly available sunspots and flares catalogues used to associate sunspots to flares

– The association is determined based on the location and timing information.

• The Neural Networks were optimized and trained using this association information– Input data comprised sunspot group classification (McIntosh) and

sunspot areas.• The Neural Networks are combined to produce a hybrid

system to give flaring probability and intensity of each sunspot group

The Flares Prediction Model

ASAP’sFlares

Prediction

“Automated Computer-Based Prediction of Solar Flares: The ASAP System”, Rami Qahwaji and Tufan Colak, Bradford University, England, UK

Results• Accuracy depends upon each stage

– ~95% on sunspot grouping– ~80% on sunspot group classification– ~90% on flare prediction depending on correct classification

• Overall success rate of ~70% on final flare prediction

• “AUTOMATED PREDICTION OF SOLAR FLARES: Integrating Image Processing and Machine Learning for the Creation of a Hybrid Computer Platform that Provides Real-Time Prediction of Solar Flares”, Amazon $78

20Q• Based on “20 Questions” game to identify person, place or thing• Predictive neural network• 20Q.net website created in 1995• 20Q started out knowing one object, a cat, and one question• Trained by online played its 44,000,000th game in September

2006• 20Q guesses correct answer 76 percent of the time (98 percent

of the time with 25 questions)• “Because 20Q does not simply follow a binary decision tree,

answering a question incorrectly will not throw it completely off” - 20Q.net

20Q GameThink of Something - “A bicycle”• Q1: Is it Animal, Vegetable, or Mineral? Mineral• Q2: Is it heavy? Y• Q3: Is it mechanical? Y• Q4: Can it protect you from the rain? N• Q5: Can you find it in a house? N• Q6: Does it shine? Y• Q7: Do you use it at night? Sometimes• Q8: Is it used in a sport? Y• Q9: Can you make sounds with it? N• Q10: Can it float? N• Q11: Can you walk on it? NI think your mind is giving me mixed signals…• Q12: Can you hold it when you use it? N• Q13: Do you put things in it? N• Q14: Would you be lost without it? N• Q15: Can it be easily moved? Y• Q16: Can it be refilled? N• Q17: Is it very large? NDrum roll please …It’s a bike!

Online at http://20Q.net

References

• Automated Solar Activity Predictorhttp://spaceweather.inf.brad.ac.uk• Neural Networks versus Conventional Computinghttp://www.neuralnetworksolutions.com• Neural Networks Historyhttp://www-cs-faculty.stanford.edu/~eroberts/courses/soco/projects/neural-networks/History/history1.html• Wikipedia: Neural Networkshttp://en.wikipedia.org/wiki/Neural_network• “Automated Computer-Based Prediction of Solar Flares: The ASAP System”, Rami Qahwaji and Tufan Colak, Bradford

University, England, UKhttp://www.google.com/url?sa=t&rct=j&q=3qahwaji.ppt&source=web&cd=1&cad=rja&ved=0CDUQFjAA&url=http%3A%2F

%2Fwww.spaceweather.eu%2Fro%2Frepository%2Fdownload%3Fid%3D3Qahwaji-1261139631.ppt%26file%3D3Qahwaji.ppt&ei=fb50UajqKvPy0QG4wIDADQ&usg=AFQjCNEAReW5MKb57Lm1loWRjnRfydMHCA&bvm=bv.45512109,d.dmQ

• Publication, “AUTOMATED PREDICTION OF SOLAR FLARES”, Amazon.com, $78http://www.amazon.com/AUTOMATED-PREDICTION-SOLAR-FLARES-Integrating/dp/3838370309• SpaceWeather.net - http://esa-spaceweather.net/sda/asap/• 20Q.net - http://www.20q.net/flat/history.html• Twenty Questions, Ten Million Synapses - http://scienceline.org/2006/07/tech-schrock-20q/• National Geophysical Data Center, Space Weather Data - http://www.ngdc.noaa.gov/stp/spaceweather.html• Sunspot Classification - http://theastronomer.tripod.com/SolarObserving202.html#sclassifications

McIntosh Sunspot Group Classification• Sunspot groups are classified by a three letter code.

– The first code letter deals with the group type. – The second code letter describes the penumbra of the largest spot of the group. – The third code letter describes the compactness of the spots in the immediate part of a group.

• Group Type:– A: Unipolar group without penumbra.– B: Bipolar group without penumbra on any spots.– C: Bipolar group with penumbra on one end of group, usually surrounding largest of leading umbra.– D: Bipolar group with penumbrae on spots at both ends of group and with longitudinal extent less than 10°.– E: Bipolar group with penumbrae on spots at both ends of group and with longitudinal length between 10° and 15°.– F: Bipolar group with penumbrae on spots at both ends of group and with longitudinal length more than 15°.– H: Unipolar group with penumbra.

• Penumbra of Largest Spot:– x: No penumbra (class A or B)– r: Rudimentary penumbra partly surrounds largest spot.– s: Small, symmetric penumbra, elliptical or circular and N-S size smaller than 2.5".– a: Small, asymmetric penumbra, irregular in outline and N-S size smaller than 2.5°.– h: Large, symmetric penumbra, N-S size larger than 2.5°.– k: Large, asymmetric penumbra, N-S size larger than 2.5.

• Spot Compactness:– x: Assigned to (but undefined for) unipolar groups (types A and H).– o: Open - few, if any, spots between leader and follower.– i: Intermediate - numerous spots between leader and follower, all without mature penumbra.– c: Compact - many large spots between leader and follower, with at least one mature penumbra.

• So, for example, if you were looking up data on a particular sunspot, and found the classification "Fki", you would know that this particular spot was:– F - bipolar (containing both a positive and negative charge) with penumbra on both ends exceeding a longitudinal length of 15 degrees.– k - the leader has a large, asymmetic penumbra with a N/S size larger the 2.5.– i - and that the group is intermediate with many spots between the leader and followers without mature penumbra.