stochastic gradient descent
7 episodes mention this concept
lexfridmanAug 29, 2021Wojciech Zaremba on OpenAI's GPT-3, Codex, the Nature of Intelligence, Consciousness, and the Future of AI
TechnologyArtificial Intelligence (AI)GPT-3OpenAI CodexGitHub Copilot- lexfridmanJan 19, 2019
Tomaso Poggio on Brains, Minds, Machines, and the Nature of Intelligence
ScienceTheory of RelativityGedanken experimentArtificial General Intelligence (AGI)Biological neural networks - lexfridmanSep 27, 2016
Deep Learning for Natural Language Processing: Foundations, Word Vectors, and Recurrent Neural Networks
TechnologyNatural Language Processing (NLP)Deep LearningArtificial Intelligence (AI)Linguistics - lexfridmanSep 27, 2016
Sequence to Sequence Deep Learning: From Smart Reply to Attention Mechanisms
TechnologySequence to Sequence Learning (Seq2Seq)Recurrent Neural Networks (RNNs)Encoder-Decoder ArchitectureAttention Mechanism - lexfridmanSep 27, 2016
Foundations of Deep Learning: Feedforward Neural Networks, Backpropagation, and Training Techniques
TechnologyFeedforward Neural NetworksInput VectorHidden LayersPre-activation - lexfridmanSep 27, 2016
Foundations and Challenges of Deep Learning: Compositionality, Generalization, and Optimization
TechnologyCurse of dimensionalityCompositional modelsDistributed representationsDepth (neural networks) - lexfridman
Recurrent Neural Networks and the Fundamentals of Backpropagation for Steering Through Time
TechnologyRecurrent Neural Networks (RNNs)BackpropagationVanishing GradientsExploding Gradients
Knowledge Graph
Related concepts — line thickness indicates connection strength. Click any node to explore.
Related concepts
Concepts that appear alongside stochastic gradient descent across episodes.
- deep learning
- weight tying
- word vectors (embeddings)
- distributional semantics
- big data vs. n=1 learning
- prior information (in learning)
- named entity recognition
- natural language processing (nlp)
- optimization algorithms
- artificial neural networks
- wordnet
- recurrent neural networks (rnns)
- face recognition (neural mechanisms)
- non-linearities
- visual cortex
- sentiment analysis
- hyperparameters
- coreference resolution
- compositionality
- equipotentiality of the brain