Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate grepping scripts to this repo #35

Open
jmdelahanty opened this issue Mar 25, 2023 · 0 comments
Open

Migrate grepping scripts to this repo #35

jmdelahanty opened this issue Mar 25, 2023 · 0 comments

Comments

@jmdelahanty
Copy link
Contributor

The data grepping/NWB table of trial structure information/other scripts for Austin's project I wrote a while ago and went unused are present here. Given the sheer number of recordings present and potential to miss broken files/conversions/etc, checking everything by hand is both quite time consuming and, frankly, annoying. I've also missed a couple apparently on accident while double checking. Unfortunately it would only work across projects if people agree on file structures/naming conventions.

So the grepping stuff should be migrated here and written so it can be used on linux cluster machines. It should be able to:

  • Check that raw 2p data folders from Bruker are intact, so environment files, xml files, and recordings are all present for each recording. Huge bonus if it can go into the snapshots and try to find and rsync missing contents.
  • Check that there are folders of tiffs present. Bonus if it can spawn a container to start conversion if necessary
  • Check that there are raw csvs present. Bonus if it can spawn a converter to start conversion if necessary
  • Check that there are events files present. Bonus if it can spawn the script for getting timestamps
  • For any discrepancies, log them in a human readable format. Logging in general during search should be done and output to file.
  • Additional bonus: Select different specific mice in a GUI or something, reference metadata for mice, etc
  • Even more additional bonus: Do this in a threaded manner so it's way faster to grep for all this stuff. That would be much much harder to get right though...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant