8 results ·
● Live web index
L
labellerr.com
article
https://www.labellerr.com/blog/evolution-of-neural-networks-to-large-language…
Explore the evolution from neural networks to large language models, highlighting key advancements in NLP with the rise of transformer models. Neural network-based language models have revolutionized natural language processing (NLP) by enabling computers to predict and generate text with remarkable accuracy. Initial models like n-grams and Hidden Markov Models laid the foundation, but their limitations prompted the development of neural networks, including Recurrent Neural Networks (RNNs) and advanced versions like Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs). The Transformer architecture, introduced in the "Attention is All You Need" paper, has revolutionized NLP by leveraging attention mechanisms to process sequences in parallel, rather than sequentially as in previous models like RNNs or LSTMs. Before transformers, recurrent neural networks (RNNs), particularly Long Short-Term Memory (LSTM) networks, were the go-to models for sequential tasks. The introduction of encoder-decoder architectures and the Transformer model revolutionized language processing, enabling improved handling of sequential data and attention mechanisms.
I
internationaljournalssrg.org
research
https://www.internationaljournalssrg.org/IJCSE/2024/Volume11-Issue6/IJCSE-V11…
In NLP tasks like language modelling, text classification, emotion analysis, and machine translation, RNNs, CNNs, and transformer-based models have been used in new ways. Keywords - Deep Learning, Natural Language Processing, CNNs, Recurrent Neural Networks, NLP Models, Transformed based models. 5. Transfer Learning In NLP Transfer learning has revolutionized Natural Language Processing (NLP) by letting researchers and programmers employ pre-trained models later. It is found that transfer learning, attention processes, and self-supervised learning improve NLP models. [CrossRef] [Google Scholar] [Publisher Link] [14] Stephane Cedric Koumetio Tekouabou et al., “Reviewing the Application of Machine Learning Methods to Model Urban Form Indicators in Planning Decision Support Systems: Potential, Issues and Challenges,” Journal of King Saud University-Computer and Information Sciences, vol.
3
32931414.s21i.faiusr.com
article
https://32931414.s21i.faiusr.com/61/ABUIABA9GAAgxLqCuwYojOPTXQ.pdf
Neural Network Methods for Natural Language Processing Yoav Goldberg 2017 Syntax-based Statistical Machine Translation Philip Williams, Rico Sennrich, Matt Post, and Philipp Koehn 2016 Domain-Sensitive Temporal Tagging Jannik Strötgen and Michael Gertz 2016 Linked Lexical Knowledge Bases: Foundations and Applications Iryna Gurevych, Judith Eckle-Kohler, and Michael Matuschek 2016 Bayesian Analysis in Natural Language Processing Shay Cohen 2016 Metaphor: A Computational Perspective Tony Veale, Ekaterina Shutova, and Beata Beigman Klebanov 2016 Grammatical Inference for Computational Linguistics Jeffrey Heinz, Colin de la Higuera, and Menno van Zaanen 2015 iii Automatic Detection of Verbal Deception Eileen Fitzpatrick, Joan Bachenko, and Tommaso Fornaciari 2015 Natural Language Processing for Social Media Atefeh Farzindar and Diana Inkpen 2015 Semantic Similarity from Natural Language and Ontology Analysis Sébastien Harispe, Sylvie Ranwez, Stefan Janaqi, and Jacky Montmain 2015 Learning to Rank for Information Retrieval and Natural Language Processing, Second Edition Hang Li 2014 Ontology-Based Interpretation of Natural Language Philipp Cimiano, Christina Unger, and John McCrae 2014 Automated Grammatical Error Detection for Language Learners, Second Edition Claudia Leacock, Martin Chodorow, Michael Gamon, and Joel Tetreault 2014 Web Corpus Construction Roland Schäfer and Felix Bildhauer 2013 Recognizing Textual Entailment: Models and Applications Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto 2013 Linguistic Fundamentals for Natural Language Processing: 100 Essentials from Morphology and Syntax Emily M.
G
github.com
article
https://github.com/lschmelzeisen/talk-advanced-neural-network-architectures
The talk is designed to take roughly two hours. It first reviews deep learning basics, thereafter presents proven architectures in natrual language processing (
R
researchgate.net
research
https://www.researchgate.net/publication/399167067_The_Evolution_of_Deep_Lear…
The introduction of neural networks marked a shift toward data-driven representations, beginning with feedforward and recurrent architectures
A
arxiv.org
article
https://arxiv.org/abs/2503.20227
[Skip to main content](https://arxiv.org/abs/2503.20227#content). [](https://www.cornell.edu/). We gratefully acknowledge support from the Simons Foundation, [member institutions](https://info.arxiv.org/about/ourmembers.html), and all contributors.[Donate](https://info.arxiv.org/about/donate.html). [](https://arxiv.org/)>[cs](https://arxiv.org/list/cs/recent)> arXiv:2503.20227. [Help](https://info.arxiv.org/help) | [Advanced Search](https://arxiv.org/search/advanced). [](https://arxiv.org/). [](https://www.cornell.edu/). * [Login](https://arxiv.org/login). * [Help Pages](https://info.arxiv.org/help). * [About](https://info.arxiv.org/about). [View PDF](https://arxiv.org/pdf/2503.20227)[HTML (experimental)](https://arxiv.org/html/2503.20227v1). Cite as:[arXiv:2503.20227](https://arxiv.org/abs/2503.20227) [cs.CL]. (or [arXiv:2503.20227v1](https://arxiv.org/abs/2503.20227v1) [cs.CL] for this version). [https://doi.org/10.48550/arXiv.2503.20227](https://doi.org/10.48550/arXiv.2503.20227). From: Ngoc Quach [[view email](https://arxiv.org/show-email/97e204e7/2503.20227)]. [](https://arxiv.org/abs/2503.20227)Full-text links:. * [View PDF](https://arxiv.org/pdf/2503.20227). * [HTML (experimental)](https://arxiv.org/html/2503.20227v1). * [TeX Source](https://arxiv.org/src/2503.20227). [view license](http://creativecommons.org/publicdomain/zero/1.0/ "Rights to this article"). [cs](https://arxiv.org/abs/2503.20227?context=cs). [cs.AI](https://arxiv.org/abs/2503.20227?context=cs.AI). * [Semantic Scholar](https://api.semanticscholar.org/arXiv:2503.20227). Data provided by: [](https://arxiv.org/abs/2503.20227). [](http://www.bibsonomy.org/BibtexHandler?requTask=upload&url=https://arxiv.org/abs/2503.20227&description=Advancements%20in%20Natural%20Language%20Processing:%20Exploring%20Transformer-Based%20Architectures%20for%20Text%20Understanding "Bookmark on BibSonomy")[](https://reddit.com/submit?url=https://arxiv.org/abs/2503.20227&title=Advancements%20in%20Natural%20Language%20Processing:%20Exploring%20Transformer-Based%20Architectures%20for%20Text%20Understanding "Bookmark on Reddit"). Bibliographic Explorer _([What is the Explorer?](https://info.arxiv.org/labs/showcase.html#arxiv-bibliographic-explorer))_. * [Author](https://arxiv.org/abs/2503.20227). * [Venue](https://arxiv.org/abs/2503.20227). * [Institution](https://arxiv.org/abs/2503.20227). * [Topic](https://arxiv.org/abs/2503.20227). [**Learn more about arXivLabs**](https://info.arxiv.org/labs/index.html). [Which authors of this paper are endorsers?](https://arxiv.org/auth/show-endorsers/2503.20227) | [Disable MathJax](javascript:setMathjaxCookie()) ([What is MathJax?](https://info.arxiv.org/help/mathjax.html)). * [About](https://info.arxiv.org/about). * [Help](https://info.arxiv.org/help). * [Contact](https://info.arxiv.org/help/contact.html). * [Subscribe](https://info.arxiv.org/help/subscribe). * [Copyright](https://info.arxiv.org/help/license/index.html). * [Privacy Policy](https://info.arxiv.org/help/policies/privacy_policy.html). * [Web Accessibility Assistance](https://info.arxiv.org/help/web_accessibility.html). * [arXiv Operational Status](https://status.arxiv.org/).
S
slds-lmu.github.io
article
https://slds-lmu.github.io/seminar_nlp_ss20/introduction-deep-learning-for-nl…
## 2.1 Word Embeddings and Neural Network Language Models. These word embeddings are usually learned by neural networks, either within the final model in an additional layer or in its own model. A simple feed-forward network with fully connected layers for learning such embeddings while predicting the next word for a given context is shown in figure 2.2. Similarly, as mentioned before, one of the most common deep learning models in NLP is the recurrent neural network (RNN), which is a kind of sequence learning model and this model is also widely applied in the field of speech processing. In 5.2 and later subsections, some practical applications of CNN in the field of NLP will be further explored, and these applications are based on different CNN architecture at diverse level, for example, exploring the model performance at character-level on text classification research (see Zhang, Zhao, and LeCun (2015)) and based on multiple data sets to detect the Very Deep Convolutional Networks (VD-CNN) for text classification (see Schwenk et al.
T
towardsdatascience.com
article
https://towardsdatascience.com/3-neural-network-architectures-you-need-to-kno…
The hidden state of an RNN at time t takes in information from both the input at time t and activations from hidden units at time t-1, to calculate outputs for time t. In an RNN, the same weight matrix is being multiplied with inputs and previous outputs, and hence the gradients explode and vanish. The output at each time step is derived both from the input, the previous output and the updated cell state. RNNs had the issue of vanishing and exploding gradients, so memory was severely short term. Also, LSTMs had to be fed input data sequentially as they needed outputs from the previous time step to calculate current outputs. Also as the attentions for each word are independent of other words, the attentions for each word can be calculated in parallel and be further processed in parallel, greatly increasing the computation time of these networks. If you are interested in learning the details of the transformer network, the original paper and this TDS article are very good resources.