Add calibration to CQL as in CalQL paper arXiv:2303.05479

[BFAnas]

• I have marked all applicable categories:
  • exception-raising fix
  • algorithm implementation fix
  • documentation modification
  • new feature
• I have reformatted the code using make format (required)
• I have checked the code using make commit-checks (required)
• If applicable, I have mentioned the relevant/related issue(s)
• If applicable, I have listed every items in this Pull Request below

[MischaPanch]

If possible, pls wait until #912 is merged, as it will certainly introduce conflicts. It is now ready to be merged (CI is green), so I hope it will be quick

[BFAnas]

Yes sure no problem. I close this one for now until after your merge.

[MischaPanch]

Thank you, appreciate it!

[MischaPanch]

You can go for it now ;)

[BFAnas]

Awsome, congrats on the big Merge!

[codecov-commenter]

Codecov Report

Merging #915 (6f4689e) into master (6449a43) will increase coverage by 0.01%.
The diff coverage is 100.00%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

@@            Coverage Diff             @@
##           master     #915      +/-   ##
==========================================
+ Coverage   90.74%   90.75%   +0.01%     
==========================================
  Files          72       72              
  Lines        5292     5321      +29     
==========================================
+ Hits         4802     4829      +27     
- Misses        490      492       +2     

Flag	Coverage Δ
unittests	90.75% <100.00%> (+0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
tianshou/data/batch.py	92.03% <ø> (ø)
tianshou/data/buffer/base.py	98.96% <100.00%> (+<0.01%)	⬆️
tianshou/data/buffer/her.py	90.52% <100.00%> (+0.20%)	⬆️
tianshou/data/collector.py	93.47% <100.00%> (ø)
tianshou/data/types.py	100.00% <100.00%> (ø)
tianshou/policy/base.py	80.33% <100.00%> (+0.33%)	⬆️
tianshou/policy/imitation/cql.py	100.00% <100.00%> (ø)
tianshou/policy/modelbased/psrl.py	97.56% <100.00%> (+0.03%)	⬆️
tianshou/trainer/base.py	97.76% <100.00%> (+0.02%)	⬆️

... and 2 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[MischaPanch]

@BFAnas I will try review this in the near future. Could you pls resolve the conflicts with master? We now have a new built and the newest formatting (for py3.11)

[BFAnas]

Hi @MischaPanch,
Sorry I was a little busy this week, I merged your changes now. I fixed some pep8 errors but now the tests are failing because of override "Module not found error", not sure if it's worth have it as a dependency. I think if we can do without it it'll be better.
Btw I don't find the "Allow anyone to contribute checkbox" in my fork settings.

[MischaPanch]

@BFAnas Thanks! The overrides dependency will be added in #950 anyway, so might as well add it here. Pls run poetry add overrides and commit the pyproject and poetry.lock file (after a round of poe format).

In python 3.12 override will become part of the language, so might as well start using it now. For tianshou it's even particularly useful. After that, the MR is ready for merge, thank you a lot for the contribution!

[BFAnas]

@BFAnas Thanks! The overrides dependency will be added in #950 anyway, so might as well add it here. Pls run poetry add overrides and commit the pyproject and poetry.lock file (after a round of poe format).

In python 3.12 override will become part of the language, so might as well start using it now. For tianshou it's even particularly useful. After that, the MR is ready for merge, thank you a lot for the contribution!

Done!

[MischaPanch]

@Trinkle23897 this one is ready to merge, but the GPU runner seems to have problems, and I can't click the merge button. Could you do that, or fix the GPU runner? Alternatively, you could change the repository settings such that I can merge without all checks being green.

EDIT: almost ready to merge, see comments below