attention mechanism
4 episodes mention this concept
lexfridmanDec 2, 2020DeepMind's AlphaFold 2: Solving Protein Folding and Its Profound Impact on AI and Life Sciences
ScienceProtein FoldingAlphaFold 2Structural BiologyX-ray Crystallography- lexfridmanJan 17, 2019
Deep Learning State of the Art (2019): Breakthroughs in NLP, Applied AI, and Reinforcement Learning
TechnologyDeep Learning State of the ArtNatural Language Processing (NLP)Encoder-Decoder ArchitectureRecurrent Neural Networks (RNNs) - lexfridmanSep 27, 2016
Sequence to Sequence Deep Learning: From Smart Reply to Attention Mechanisms
TechnologySequence to Sequence Learning (Seq2Seq)Recurrent Neural Networks (RNNs)Encoder-Decoder ArchitectureAttention Mechanism - lexfridmanSep 27, 2016
Foundations and Challenges of Deep Learning: Compositionality, Generalization, and Optimization
TechnologyCurse of dimensionalityCompositional modelsDistributed representationsDepth (neural networks)
Knowledge Graph
Related concepts — line thickness indicates connection strength. Click any node to explore.
Related concepts
Concepts that appear alongside attention mechanism across episodes.
- encoder-decoder architecture
- recurrent neural networks (rnns)
- sequence to sequence learning (seq2seq)
- computational resources
- natural language processing (nlp)
- contextual embeddings
- transformer architecture
- softmax
- hyperbolic tangent
- generative adversarial networks (gans)
- model selection (underfitting/overfitting)
- deep reinforcement learning
- deep learning state of the art
- dilated/atrous convolutions
- synthetic data
- self-attention
- greedy decoding
- tokenization and normalization
- word vectors
- automated machine learning (automl)