Catalyst Cooperative
Next data release for updated EIA-923 data #1845
-
|
It looks like the EIA just released an updated versions of the 2014-2020 EIA-923 files on August 11 (https://www.eia.gov/electricity/data/eia923/correction.php). Will PUDL produce a new data release on zenodo with this updated data? If so, what is the anticipated timeline for that? |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 4 replies
-
|
@cmgosnell has been churning through (and I've been reviewing) the Early Release EIA-860/923 data for 2021 (and EIA-860m through 2022-06). See PR #1834. That included generating a new archive for all the years of inputs, but those archives were snapshots from July 27th to August 2nd. See EIA-860, EIA-860m, and EIA-923 on Zenodo. It's easy to make a new snapshot. How long it takes to go from snapshot to database depends on how much has changed in the structure and contents of the data. Those changes look extremely minor, which both means that it would be easy, and the benefits would be small. |
Beta Was this translation helpful? Give feedback.
-
|
Okay I uploaded a new archive here: https://zenodo.org/record/7002112 |
Beta Was this translation helpful? Give feedback.
-
|
D'oh. Corrupted zipfile in 2020. Fixing. |
Beta Was this translation helpful? Give feedback.
-
|
I've made PR #1846 to integrate the new EIA-923 raw data archive. I don't know if you'd be up for this @cmgosnell but if you're going to be running the full ETL a bunch locally while working on #1836, might you also want to try merging that PR branch into your branch and seeing if the revised data works? |
Beta Was this translation helpful? Give feedback.
-
|
Thanks, Zane! I was actually curious about the timeline for the next full PUDL data release (https://zenodo.org/record/6349861#.Yv0a9BzMJhE). I'm sure this probably depends not only on updated data inputs but also updates to the pudl code. |
Beta Was this translation helpful? Give feedback.
-
|
This is currently waiting on our removal of the EIA API calls from our code (see #1491) and implementation of a new system for estimating redacted fuel prices (See #1708). But given where we're at right now it'll also end up including the 2021 Early Release data. Hopefully before the end of August! |
Beta Was this translation helpful? Give feedback.
@cmgosnell has been churning through (and I've been reviewing) the Early Release EIA-860/923 data for 2021 (and EIA-860m through 2022-06). See PR #1834. That included generating a new archive for all the years of inputs, but those archives were snapshots from July 27th to August 2nd. See EIA-860, EIA-860m, and EIA-923 on Zenodo.
It's easy to make a new snapshot. How long it takes to go from snapshot to database depends on how much has changed in the structure and contents of the data. Those changes look extremely minor, which both means that it would be easy, and the benefits would be small.