8 results ·
● Live web index
E
en.wikipedia.org
article
https://en.wikipedia.org/wiki/History_of_artificial_neural_networks
* [(Top)](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#). * [3.1 LSTM](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#LSTM). * [5 Deep learning](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#Deep_learning). * [7.2 Transformer](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#Transformer). * [8.3 Deep learning](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#Deep_learning_2). * [11 Notes](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#Notes). * [Read](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks). * [Read](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks). popularized backpropagation.[[31]](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_note-32). They reported up to 70 times faster training.[[85]](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_note-86). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-fukuneoscholar_61-0)**Fukushima, K. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-wz1988_68-0)**Zhang, Wei (1988). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-wz1990_69-0)**Zhang, Wei (1990). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-70)**Fukushima, Kunihiko; Miyake, Sei (1982-01-01). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-LECUN1989_71-0)**LeCun _et al._, "Backpropagation Applied to Handwritten Zip Code Recognition," _Neural Computation_, 1, pp. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-73)**Zhang, Wei (1991). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-74)**Zhang, Wei (1994). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-Weng1992_75-0)**J. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-Weng19932_76-0)**J. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-Weng1997_77-0)**J. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-81)**Sven Behnke (2003). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-:62_88-0)**Ciresan, D. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-:9_91-0)**Ciresan, D.; Meier, U.; Schmidhuber, J. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-szegedy_94-0)**Szegedy, Christian (2015). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-101)**Linn, Allison (2015-12-10). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-olli2010_106-0)**Niemitalo, Olli (February 24, 2010). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-108)**Gutmann, Michael; Hyvärinen, Aapo. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-Cherry_1953_115-0)**Cherry EC (1953). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-118)**Fukushima, Kunihiko (1987-12-01). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-:12_121-0)**Soydaner, Derya (August 2022). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-122)**Giles, C. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-123)**Feldman, J. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-125)**Schmidhuber, Jürgen (January 1992). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-135)**Levy, Steven. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-138)**Kohonen, Teuvo (1982). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-139)**Von der Malsburg, C (1973). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-141)**Smolensky, Paul (1986). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-144)**Sejnowski, Terrence J. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-hinton2006_146-0)**[Hinton, G. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-hinton2009_147-0)**Hinton, Geoffrey (2009-05-31). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-149)**Watkin, Timothy L. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-150)**Schwarze, H; Hertz, J (1992-10-15). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-151)**Mato, G; Parga, N (1992-10-07). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-schmidhuber19922_153-0)**Schmidhuber, Jürgen (1992). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-154)**Hanson, Stephen; Pratt, Lorien (1988). **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-157)**Yang, J. **[^](https://en.wikipedia.org/wiki/History_of_artificial_neural_networks#cite_ref-158)**Strukov, D.
M
machinelearningcmu.github.io
article
https://machinelearningcmu.github.io/F23-10701/slides/lecture14.pdf
Durbin, Golden, Chauvin 1986—Rumelhart & Hinton Popularize Backpropagation, train larger nets Jordan Nets with Recurrent Nets 1986 Finding Structure in Time: Seeds of Language Modeling and Modern RNNs Yann LeCun trains ConvNets for OCR (1989) Yamaguchi introduce Max-Pooling (1990) • Applied in neural network for speech recognition (“speaker-independent isolated word recognition”) 1990s—“Textbook ML” comes into focus • Supervised learning Predict y given x • Unsupervised learning Uncover the structure of x, without pre-specifying any prediction task • Reinforcement learning Learn a policy to optimize a delayed reward signal 1991 LeNet Applied for OCR 1995 Adopted by Banks (for check-reading) 1997 — Invention of LSTM RNNs Hochreiter and Schmidhuber [1997] 2010—The Rise of Modern Deep Learning • 2008 Graves/Schmidhuber make strides in handwriting recognition/generation • 2010 Dahl/Hinton Win Kaggle Competition for predicting drug binding sites 2012 Khrizhevsky/Sutskever/Hinton win ImageNet Challenge “Human-level control through deep reinforcement learning” 2013 DeepMind’s AlphaGo Masters Go Industrial Applications in Healthcare Optimism rises for new era of self-driving https://www.youtube.com/watch?v=9e2x4dDRB-k 2014—Leaps in Commercial Speech Recognition (DeepSpeech) Hannun et al, 2014 Concerns arise about Fairness/Transparency/Privacy AI’s Generative Turn Sequence(-to-Sequence) Modeling 2012—Early Experiments with Deep RNNs + Language Modeling image credit: Karpathy, text from Sutskever Martens, Hinton 2012 (Fast-forward to 2023) Unaligned Seq-to-Seq Models for Natural Language Translation (2014) 2014/5 Image Captioning 2014 Generative Adversarial Networks Figure credit: Chris Olah Earlier GAN results Rapid progress in image quality https://www.youtube.com/watch?v=XOxxPcy5Gr4 Conditional Diffusion Models Prompt: Anthropomorphic majestic blobfish knight, portrait, finely detailed armor, cinematic lighting, intricate filigree metal design, 4k, 8k, unreal engine, octane render Image via https://www.blueshadow.art/midj ourney-prompt-commands/ From Narrow Purpose-Built Models to Webscale Capabilities The Rise of Foundation Models • 2017—ELMO pretrains forwards and backwards LSTMS for contextualized representations, fine-tunes on downstream tasks • 2018—BERT trained on web crawl to learn representations useful for downstream classification with surprisingly little fine-tuning • 2018—OpenAI releases GPT, a general web-scale language model • 2019—OpenAI releases GPT2 • 2020—OpenAI releases GPT3 • 2021—OpenAI releases
C
cs.stanford.edu
research
https://cs.stanford.edu/people/eroberts/courses/soco/projects/neural-networks…
| The Artificial Neuron History Comparison Architecture Applications Future Sources | Neural Network Header **History: The 1940's to the 1970's** In 1943, neurophysiologist Warren McCulloch and mathematician Walter Pitts wrote a paper on how neurons might work. In order to describe how neurons in the brain might work, they modeled a simple neural network using electrical circuits. MADALINE was the first neural network applied to a real world problem, using an adaptive filter that eliminates echoes on phone lines. It is based on the idea that while one active perceptron may have a big error, one can adjust the weight values to distribute it across the network, or at least to adjacent perceptrons. Despite the later success of the neural network, traditional von Neumann architecture took over the computing scene, and neural research was left behind. In the same time period, a paper was written that suggested there could not be an extension from the single layered neural network to a multiple layered neural network.
F
flagshippioneering.com
article
https://www.flagshippioneering.com/timelines/a-timeline-of-deep-learning
Research on neural networks stalls after MIT’s Marvin Minsky and Seymour Papert argue, in a book called “Perceptrons,” that the method would be too limited to be useful even if neural networks had many more layers of artificial neurons than Rosenblatt’s machine did. The backpropagation algorithm had been applied in computers in the 1970s, but now researchers put it to wider use in neural networks. Google researcher Ian Goodfellow plays two neural networks off each other to create what he calls a “generative adversarial network.” One network is programmed to generate data—such as an image of a face—while the other, known as the discriminator, evaluates whether it’s plausibly real. A deep learning system called AlphaGo beats human Go champion Lee Sedol after absorbing thousands of examples of past games played by people. The same team develops AlphaFold, a set of deep learning and generative neural networks to predict the structure of proteins from their amino acid sequences.
R
researchgate.net
research
https://www.researchgate.net/figure/Timeline-of-the-history-of-artificial-neu…
Timeline of the history of artificial neural networks and deep learning. Deep learning's peak corresponds with Hinton's et al breakthrough paper [50] and
Y
youtube.com
video
https://www.youtube.com/watch?v=AA2ettRM6_Q
Neural Networks Explained: From 1943 Origins to Deep Learning Revolution 🚀 | AI History & Evolution
The AI Guy
1400 subscribers
258 likes
10587 views
10 Jun 2024
Discover the fascinating history of neural networks, from their origins in 1943 to the groundbreaking deep learning advancements of today. Learn how pioneering scientists like Warren McCulloch, Walter Pitts, Frank Rosenblatt, John Hopfield, Geoffrey Hinton, and others contributed to this revolutionary field. Understand key developments like the perceptron, backpropagation, and the role of GPUs in transforming AI. Join us on this journey through time to see how neural networks have evolved to shape modern machine learning and artificial intelligence. 🚀 #NeuralNetworks #DeepLearning #AIHistory #MachineLearning #ArtificialIntelligence
9 comments
G
galileo-unbound.blog
article
https://galileo-unbound.blog/2025/02/05/a-short-history-of-neural-networks/
* ai, Artificial Intelligence, Attention mechanism, convolutional neural network, Deep Learning, History of Physics, Hopfield network, Machine Learning, neural networks, Neurodynamics, Nonlinear Dynamics, recurrent neural network, technology, van der Pol oscillator. Drawing from the work of McCulloch and Pitts, his team constructed a software system and then constructed a hardware model that adaptively updated the strength of the inputs, that they called neural weights, as it was trained on test images. PDP was an exciting framework for artificial intelligence, and it captured the general behavior of natural neural networks, but it had a serious problem: How could all of the neural weights be trained? The breakthrough that propelled Geoff Hinton to world-wide acclaim was the success of AlexNet, a neural network constructed by his graduate student Alex Krizhevsky at Toronto in 2012 consisting of 650,000 neurons with 60 million parameters that were trained using two early Nvidia GPUs. It won the ImageNet challenge that year, enabled by its deep architecture and representing a marked advancement that has been proceeding unabated today.
M
medium.com
article
https://medium.com/@chunklingo/a-short-history-of-neural-networks-52ff64d2025e
The First Neural Network: The Perceptron (1958). The first idea of a neural network came from Frank Rosenblatt in 1958, in his famous paper “The