Skip to content

Commit

Permalink
Merge pull request #645 from Alejandro-Casanova/patch-13
Browse files Browse the repository at this point in the history
fix typo
  • Loading branch information
mrdbourke authored Oct 5, 2023
2 parents 501cbba + 6466ba8 commit 909cf24
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion 08_pytorch_paper_replicating.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4374,7 +4374,7 @@
"source": [
"## Extra-curriculum\n",
"\n",
"* There have been several iterations and tweaks to the Vision Transformer since its original release and the most concise and best performing (as of July 2022) can be viewed in [*Better plain ViT baselines for ImageNet-1k*](https://arxiv.org/abs/2205.01580). Depsite of the upgrades, we stuck with replicating a \"vanilla Vision Transformer\" in this notebook because if you understand the structure of the original, you can bridge to different iterations.\n",
"* There have been several iterations and tweaks to the Vision Transformer since its original release and the most concise and best performing (as of July 2022) can be viewed in [*Better plain ViT baselines for ImageNet-1k*](https://arxiv.org/abs/2205.01580). Despite of the upgrades, we stuck with replicating a \"vanilla Vision Transformer\" in this notebook because if you understand the structure of the original, you can bridge to different iterations.\n",
"* The [`vit-pytorch` repository on GitHub by lucidrains](https://github.com/lucidrains/vit-pytorch) is one of the most extensive resources of different ViT architectures implemented in PyTorch. It's a phenomenal reference and one I used often to create the materials we've been through in this chapter. \n",
"* PyTorch have their [own implementation of the ViT architecture on GitHub](https://github.com/pytorch/vision/blob/main/torchvision/models/vision_transformer.py), it's used as the basis of the pretrained ViT models in `torchvision.models`.\n",
"* Jay Alammar has fantastic illustrations and explanations on his blog of the [attention mechanism](https://jalammar.github.io/visualizing-neural-machine-translation-mechanics-of-seq2seq-models-with-attention/) (the foundation of Transformer models) and [Transformer models](https://jalammar.github.io/illustrated-transformer/). \n",
Expand Down

0 comments on commit 909cf24

Please sign in to comment.