-
Notifications
You must be signed in to change notification settings - Fork 525
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] (tf backend) Print NAN when training dipole model with reference data #4536
Comments
It's related to |
The filename is |
I thought this should standardize the target key? deepmd-kit/deepmd/utils/data.py Lines 568 to 571 in e5eac4a
maybe we need something similar in |
How about this: ChiahsinChu@e214ee9 Then the users do not need to change their conventions about naming. |
The filename is defined here deepmd-kit/deepmd/utils/data.py Line 610 in e5eac4a
from deepmd-kit/deepmd/utils/data.py Lines 504 to 508 in e5eac4a
It seems to me that line 569 doesn't affect the filename to be loaded. |
correct, line 569 just makes it compatible for both |
I think this might work. @ChiahsinChu Can you create a PR with your fix? |
Fix bug mentioned in #4536 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Bug Fixes** - Updated atomic property and weight label naming conventions across the machine learning training and loss components to ensure consistent terminology. - Corrected placeholder key references in the training process to match updated label names. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
Bug summary
When training dipole models with deepmd v3, the loss printed is nan even in the presence of reference data.
DeePMD-kit Version
DeePMD-kit v3.0.0
Backend and its version
TensorFlow v2.18.0-rc2-4-g6550e4bd802
How did you download the software?
pip
Input Files, Running Commands, Error Log, etc.
Input files (
deepmd-kit/examples/water_tensor/dipole/dipole_input.json
but use datasets with atomic dipoles only):Running commands:
Error log:
lcurve.out
:For comparison,
lcurve.out
from pt backend in the same setup:Steps to Reproduce
Further Information, Files, and Links
No response
The text was updated successfully, but these errors were encountered: