Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

About the experimental results #4

Open
huybery opened this issue Aug 30, 2021 · 2 comments
Open

About the experimental results #4

huybery opened this issue Aug 30, 2021 · 2 comments

Comments

@huybery
Copy link

huybery commented Aug 30, 2021

I directly ran the code of the code base without any modification. The results are as follows

08/28/2021 06:50:45 [Epoch 100] dev acc: 0.70696 (took 220s)
08/28/2021 06:50:45 checkpoint: tmp/dtfixup
08/28/2021 06:50:45 best dev accuracy: 0.72340
08/28/2021 06:50:45 checkpoint: tmp/dtfixup

The best dev accuracy is only 72.3%, Maybe I missed something?
For the Experiment Configuration, I found that the batch in the code is 32 and the batch in the paper is 16. Is this the reason for my failure?

@billy-inn
Copy link
Collaborator

In practice, the experimental results have some variances. The best results reported/submitted in the paper is the best one from 5 runs.

@huybery
Copy link
Author

huybery commented Sep 1, 2021

Thanks for your rely, I will run more times.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants