Skip to content

Commit

Permalink
Merge pull request #646 from Alejandro-Casanova/patch-14
Browse files Browse the repository at this point in the history
fixed wrong link
  • Loading branch information
mrdbourke authored Oct 5, 2023
2 parents eac3c57 + 0167c39 commit 25bbdc9
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion 08_pytorch_paper_replicating.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4377,7 +4377,7 @@
"* There have been several iterations and tweaks to the Vision Transformer since its original release and the most concise and best performing (as of July 2022) can be viewed in [*Better plain ViT baselines for ImageNet-1k*](https://arxiv.org/abs/2205.01580). Depsite of the upgrades, we stuck with replicating a \"vanilla Vision Transformer\" in this notebook because if you understand the structure of the original, you can bridge to different iterations.\n",
"* The [`vit-pytorch` repository on GitHub by lucidrains](https://github.com/lucidrains/vit-pytorch) is one of the most extensive resources of different ViT architectures implemented in PyTorch. It's a phenomenal reference and one I used often to create the materials we've been through in this chapter. \n",
"* PyTorch have their [own implementation of the ViT architecture on GitHub](https://github.com/pytorch/vision/blob/main/torchvision/models/vision_transformer.py), it's used as the basis of the pretrained ViT models in `torchvision.models`.\n",
"* Jay Alammar has fantastic illustrations and explanations on his blog of the [attention mechanism](https://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/) (the foundation of Transformer models) and [Transformer models](https://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/). \n",
"* Jay Alammar has fantastic illustrations and explanations on his blog of the [attention mechanism](https://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/) (the foundation of Transformer models) and [Transformer models](https://jalammar.github.io/illustrated-transformer/). \n",
"* Adrish Dey has a fantastic [write up of Layer Normalization](https://wandb.ai/wandb_fc/LayerNorm/reports/Layer-Normalization-in-Pytorch-With-Examples---VmlldzoxMjk5MTk1) (a main component of the ViT architecture) can help neural network training.\n",
"* The self-attention (and multi-head self-attention) mechanism is at the heart of the ViT architecture as well as many other Transformer architectures, it was originally introduced in the [*Attention is all you need*](https://arxiv.org/abs/1706.03762) paper.\n",
"* Yannic Kilcher's YouTube channel is a sensational resource for visual paper walkthroughs, you can see his videos for the following papers:\n",
Expand Down

0 comments on commit 25bbdc9

Please sign in to comment.