Icebox: Move to new Zenodo API

[jdangerx]

See also catalyst-cooperative/pudl#2939 -

Zenodo migrated to their new API, we need to use it if we want to keep creating new archives.

• use f"/records/{record_number}/files/{name}" in the datapackage.json
• hit the correct endpoints in the ZenodoDepositor
• upgrade to frictionless>=5.0?

Some details on what the "correct endpoints" are: it seems like the basic "make a draft, edit it, publish it" flow is the same as before, which is good.

We have the existing operations defined on our ZenodoDepositor:

• create_deposition - now create a draft record
• get_deposition - this gets the latest record associated with a concept DOI, we need to test if get a record works with concept DOIs
• get_record - this just gets the single record associated with a DOI - so just get a record
• get_new_version: this makes a new draft, copies the files, and updates the version. Maybe we want to break out the end point hits in here into their own functions.
  • I think create a new version + link files from a previous version should do the trick, though I'm not sure how we set the version in that workflow
  • other option is edit a published record and then update a draft record, though we need to check that edit a published record actually makes a snapshot like we expect.
• publish_deposition: publish a draft record
• delete_deposition: delete / discard a draft record
• create_file: start draft file upload(s), upload a draft file's content, complete draft file upload
  • unclear if we need to do the whole start/complete thing.
• delete_file: delete a draft file
• update_file: this one is defined in terms of create_file and update_file so I think we're good.
• add community?

Sandbox changes:

• API values have changed to match regular DOI values with a 5281 prefix, and all sandbox archive DOI values have changed. These will need to be updated.

[zaneselvans]

Automatically adding the catalyst-cooperative community would be great -- or maybe, add it if the previous version of the record was in the community. Right now the fact that a record is in a community doesn't seem to be inherited by subsequent versions which is very annoying, and leads to the most recent records often not showing up.

[zaneselvans]

I notice that we do not list frictionless as one of the dependencies in this module, but it does get installed, because we depend on catalystcoop.pudl which depends on catalystcoop.ferc-xbrl-extractor which depends on frictionless.

We also don't use frictionless in this package, even though we redefine (Pydantic) classes which mirror the Resource and DataPackage concepts. This seems like a recipe for ending up with outputs that look like data packages, but may or may not actually conform to the specification.

[jdangerx]

Ooooh yeah. I noticed that dependency funniness but did not connect the dots with "we don't actually use frictionless for data packages." We definitely should update that - could be a separate PR, though.

[e-belfer]

According to email communications with Zenodo this API will not be launched for another year and it's behavior is expected to change. So let's pause on this.