Skip to content

Commit

Permalink
Merge pull request #648 from Alejandro-Casanova/patch-15
Browse files Browse the repository at this point in the history
fixed typo
  • Loading branch information
mrdbourke authored Oct 5, 2023
2 parents 909cf24 + e7f1614 commit d6ff056
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion 08_pytorch_paper_replicating.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -4361,7 +4361,7 @@
"4. Try repeating the steps from excercise 3 but this time use the \"`ViT_B_16_Weights.IMAGENET1K_SWAG_E2E_V1`\" pretrained weights from [`torchvision.models.vit_b_16()`](https://pytorch.org/vision/stable/models/generated/torchvision.models.vit_b_16.html#torchvision.models.vit_b_16).\n",
" * **Note:** ViT pretrained with SWAG weights has a minimum input image size of `(384, 384)` (the pretrained ViT in exercise 3 has a minimum input size of `(224, 224)`), though this is accessible in the weights `.transforms()` method.\n",
"5. Our custom ViT model architecture closely mimics that of the ViT paper, however, our training recipe misses a few things. Research some of the following topics from Table 3 in the ViT paper that we miss and write a sentence about each and how it might help with training:\n",
" * ImageNet-22k pretraining (more data).\n",
" * ImageNet-21k pretraining (more data).\n",
" * Learning rate warmup.\n",
" * Learning rate decay.\n",
" * Gradient clipping."
Expand Down

0 comments on commit d6ff056

Please sign in to comment.