Knowledge Tree
Recurrent Neural Network
Also known as: RNN, recurrent neural networks
Facts (11)
Sources
Neuro-Symbolic AI: Explainability, Challenges, and Future Trends arxiv.org Nov 7, 2024 3 facts
procedureThe implicit adjustment process in Deep Symbolic Regression (DSR) provides feedback to the RNN to guide expression generation, relying on gradient descent or optimization algorithms to adjust RNN weights.
claimIn the Deep Symbolic Regression (DSR) method, the internal weights, activation functions, and processed sequences of the Recurrent Neural Network act as an implicit intermediate representation that is not directly oriented to the end user. However, because the symbolic form is output explicitly as a mathematical expression, the process is considered partially explicit and interpretable.
procedurePetersen et al. (2019) proposed the Deep Symbolic Regression (DSR) method to recover mathematical expressions from data. The method proceeds by: (1) representing expressions as node sequences in symbolic expression trees containing mathematical operators and operands; (2) using a Recurrent Neural Network (RNN) to predict the next operator or operand based on the existing sequence; (3) calculating the degree of fit of the generated expression on a specific dataset; and (4) using the fit as feedback to guide the RNN's subsequent generation process.
A Survey on the Theory and Mechanism of Large Language Models arxiv.org Mar 12, 2026 3 facts
referenceThe paper 'Rwkv: reinventing rnns for the transformer era' (arXiv:2305.13048) proposes a method to reinvent recurrent neural networks for the transformer era.
claimThe research paper 'Learning to (learn at test time): rnns with expressive hidden states' (arXiv:2407.04620) introduces recurrent neural networks with expressive hidden states designed for test-time learning.
claimWen et al. (2024) theoretically showed that introducing a single Transformer layer into an RNN is sufficient to enhance its in-context retrieval capability and close the representation gap with Transformers.
Practices, opportunities and challenges in the fusion of knowledge ... frontiersin.org 2 facts
claimThe transformer architecture was created to address the limitations of Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks in managing long-range dependencies in sequential data.
referenceThe development of Large Language Models (LLMs) evolved from traditional rule-based and statistical models like n-grams (Brown et al., 1992) and Hidden Markov Models (Rabiner and Juang, 1986) to Recurrent Neural Networks (RNNs) and Long Short-Term Memory (LSTM) networks (Sherstinsky, 2020).
The Synergy of Symbolic and Connectionist AI in LLM-Empowered ... arxiv.org Jul 11, 2024 1 fact
claimLong Short-Term Memory (LSTM) networks were developed in the 1990s to address the limitations of traditional recurrent neural networks (RNNs) by using gating mechanisms to handle long-term dependencies in sequential data.
The cognitive neuroscience of self-awareness: Current framework ... pubmed.ncbi.nlm.nih.gov 1 fact
referenceThe paper 'Predictive coding is a consequence of energy efficiency in recurrent neural networks' published in Patterns (2022) posits that predictive coding arises as a result of energy efficiency requirements in recurrent neural networks.
Construction of Knowledge Graphs: State and Challenges - arXiv arxiv.org 1 fact
claimLong Short-Term Memory (LSTM) networks, a type of recurrent neural network (RNN), are used for Named Entity Recognition (NER) tasks.