stochastic gradient descent (sgd)
5 episodes mention this concept
lexfridmanAug 29, 2021Wojciech Zaremba on OpenAI's GPT-3, Codex, the Nature of Intelligence, Consciousness, and the Future of AI
TechnologyArtificial Intelligence (AI)GPT-3OpenAI CodexGitHub Copilot- lexfridmanSep 27, 2016
Foundations and Challenges of Deep Learning: Compositionality, Generalization, and Optimization
TechnologyCurse of dimensionalityCompositional modelsDistributed representationsDepth (neural networks) - lexfridmanSep 27, 2016
Foundations of Deep Learning: Feedforward Neural Networks, Backpropagation, and Training Techniques
TechnologyFeedforward Neural NetworksInput VectorHidden LayersPre-activation - lexfridmanSep 27, 2016
Sequence to Sequence Deep Learning: From Smart Reply to Attention Mechanisms
TechnologySequence to Sequence Learning (Seq2Seq)Recurrent Neural Networks (RNNs)Encoder-Decoder ArchitectureAttention Mechanism - lexfridman
Recurrent Neural Networks and the Fundamentals of Backpropagation for Steering Through Time
TechnologyRecurrent Neural Networks (RNNs)BackpropagationVanishing GradientsExploding Gradients
Knowledge Graph
Related concepts — line thickness indicates connection strength. Click any node to explore.
Related concepts
Concepts that appear alongside stochastic gradient descent (sgd) across episodes.
- recurrent neural networks (rnns)
- backpropagation
- local minima
- chain rule
- saddle points
- batch normalization
- backpropagation through time (bptt)
- rectified linear activation function (relu)
- action space (problem-solving)
- convex optimization
- auto-differentiation
- hidden layers
- compression (as intelligence)
- backward pass
- sequence to sequence learning (seq2seq)
- deep learning
- neural networks
- activation functions (sigmoid, relu)
- long-term dependencies
- beam search decoding