Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

I0313 15:10:55.695348 140095594268544 learning.py:507] global step 2290: loss = 24.0642 (4.784 sec/step) #381

Open
haobabuhaoba opened this issue Mar 14, 2020 · 0 comments

Comments

@haobabuhaoba
Copy link

I0313 15:10:55.695348 140095594268544 learning.py:507] global step 2290: loss = 24.0642 (4.784 sec/step)
INFO:tensorflow:Saving checkpoint to path ./train_model/model.ckpt
I0313 15:11:12.685011 140091937531648 supervisor.py:1117] Saving checkpoint to path ./train_model/model.ckpt
INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.DataLossError'>, corrupted record at 12
[[{{node pascalvoc_2007_data_provider/parallel_read/ReaderReadV2}}]]
I0313 15:11:19.223580 140091971102464 coordinator.py:224] Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.DataLossError'>, corrupted record at 12
[[{{node pascalvoc_2007_data_provider/parallel_read/ReaderReadV2}}]]
INFO:tensorflow:Caught OutOfRangeError. Stopping Training. 2 root error(s) found.
(0) Out of range: FIFOQueue '_6_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[node fifo_queue_Dequeue (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
(1) Out of range: FIFOQueue '_6_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[node fifo_queue_Dequeue (defined at /usr/local/lib/python3.6/dist-packages/tensorflow_core/python/framework/ops.py:1748) ]]
[[fifo_queue_Dequeue/_321]]
0 successful operations.
0 derived errors ignored.

Originally posted by @haobabuhaoba in #366 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant