PUDL pandas style guide

[bendnorman]

Any interest in developing a Catalyst/PUDL pandas style guide? My return to data wrangling with the gridstatus work reminded me how many ways there are to do an operation in pandas. Some examples:

• built in operators or pandas methods: + vs .add()
• filtering data: df.query("a = 1") , df[df.a == 1], df[df.a.eq(1)]
• accessing columns via strings or attributes: df["my_column"], df.my_column
• probably many others I'm not thinking of

I think we all have different styles right now which sometimes makes it challenging to learn about a new section of data wrangling code.

If we wanted to go wild we could even create a pre-commit linter.

[cmgosnell]

hey ben! i think this is a good idea overall! I'm curious if there are any resources that we can just point to instead of making these decisions ourselves. On a quick search I found this one which is relatively minimal but I like it overall. (of your examples though, it does seem to only answer your df["my_column"] (good) vs df.my_column (bad apparently) question) (also the last suggestion in this guide i think we break all the time)

Also, we did a million years ago take a first crack at an attempt to develop some code standards. This does have a small pandas section but it was never really fleshed out.

[zaneselvans]

It's not very extensive, but there's also the pandas-vet flake8 extension, most of which has been integrated into Ruff.

[bendnorman]

These seem like great starting points! Looks like we already use some of pandas-vet ruff rules. I'll look into how to extend the rules.