1、CS 4700: Foundations of Artificial Intelligence,Prof. Carla P. Gomes gomescs.cornell.eduModule: Intro Neural Networks (Reading: Chapter 20.5),Neural Networks,Rich history, starting in the early forties with McCulloch and Pittss model of artificial neurons (McCulloch and Pitts 1943).Two views:Modelin
2、g the brain“Just” representation of complex functions (Continuous; contrast decision trees)Much progress on both fronts.Drawn interest from: Neuroscience, Cognitive science, AI, Physics, Statistics, and CS/EE.,Computer vs. Brain,Circa 1997,Computer processor speed (MIPS),Information or computer stor
3、age (Megabytes),Increasing Compute Power: Moores Law,In 1965, Gordon Moore, Intel co-founder, predicted that the number of transistors on a chip would double about every two years. (popularly known as Moores Law). Intel has kept that pace for nearly 40 years.,Computer Power / Cost,Computer processor
4、 speed (MIPS),Circa 1997,Neural Networks,Computational model inspired by the brain based on the interaction of multiple connected processing elements (Connectionism, parallel distributed processing, neural computation) .,Brains information and processing power emerges from a highly interconnected ne
5、twork of neurons.,Brain,Brain made up of neurons (1011),Inputs,Outputs,Connection between cells,Excitatory or inhibitoryand may change over time,Around 1011 neurons, 1014 synapses; a cycle time of 1ms-10 ms.,When inputs reach some threshold an action potential (electric pulse)is sent along the axon
6、to the outputs,Biological Neurons,The brain is made up of neurons which have A cell body (soma) Dendrites (inputs) An axon (outputs) Synapses (connection between cells) Synapses can be excitatory or inhibitory and may change over time When the inputs reach some threshold an action potential (electri
7、c pulse) is sent along the axon to the outputs There are around 1011 neurons, 1014 synapses; a cycle time of 1ms-10 ms. Signals are noisy “spike trains“ of electrical potential,Issue: The Hardware,The brain a neuron, or nerve cell, is the basic information processing unit (1011 ) many more synapses
8、(1014) connect the neurons cycle time: 10(-3) seconds (1 millisecond)How complex can we make computers? 108 or more transistors per CPU supercomputer: hundreds of CPUs, 1010 bits of RAM cycle times: order of 10(-9) seconds (1 nanosecond),Compute Power vs. Brain Power,In near future we can have compu
9、ters with as many processing elements as our brain, but:far fewer interconnections (wires or synapses)much faster updates (1 millisecond, 10-3 vs. 1 nanosecond 10-9)Fundamentally different hardware may require fundamentally different algorithms!Very much an open question.,Why Neural Nets?,Motivation
10、: Solving problems under the constraints similar to those of the brain may lead to solutions to AI problems that would otherwise be overlooked. Individual neurons operate very slowly massively parallel algorithms Neurons are failure-prone devices distributed and redundant representations Neurons pro
11、mote approximate matching less brittle,Connectionist Models of Learning,Characterized by: A large number of very simple neuron-like processing elements. A large number of weighted connections between the elements. Highly parallel, distributed control. An emphasis on learning internal representations
12、 automatically.,But of course the interconnectivity is not really at the brain scale,Autonomous Learning Vehicle In a Neural Net (ALVINN),ALVINN learns to drive an autonomous vehicle at normal speeds on public highways.,Pomerleau et al, 1993,ALVINN is a perception systems which learns to control the
13、 NAVLAB vehicles by watching a person drive.,ALVINN drives 70mph on highways,Each output unit correspond to a particular steering direction. The most highly activated one gives the direction to steer.,30 x 32 grid of pixelintensities from camera,What kinds of problems are suitable for neural network
14、s?,Have sufficient training dataLong training times are acceptableNot necessary for humans to understand learned target function or hypothesis, neural networks are magic black boxes,Tasks,Function approximation, or regression analysis, including time series prediction and modeling. Classification, i
15、ncluding pattern and sequence recognition, novelty detection and sequential decision making. Data processing, including filtering, clustering, blind signal separation and compression.,Example of Application Areas,Application areas include:System identification and control (vehicle control, process c
16、ontrol), Game-playing and decision making (backgammon, chess, racing), Pattern recognition (radar systems, face identification, object recognition, etc.) Sequence recognition (gesture, speech, handwritten text recognition), Medical diagnosis Financial applications Data mining (or knowledge discovery in databases, “KDD“), Visualization E-mail spam filtering.,
