ENH: Clear authentication defaults with more fine-grained control

[max-sixty]

We have about 220 lines of code which handles authentication - do we need that? Generally the answer to "do we need something we do a lot of" is 'Yes', but asking regardless

My prior is that we could:

1. Check if creds are passed by the user
2. Otherwise pass nothing through to the Google libraries, and let them manage the defaults

That would reduced the code we needed to maintain and conform to standards - Google have very reasonable defaults, and we make it harder for those to flow through - e.g. PANDAS_GBQ_CREDENTIALS_FILE is non-standard, every other implementation uses GOOGLE_APPLICATION_CREDENTIALS, so users need to have an additional setting to use this library

[tswast]

I definitely think there is room for improvement here. My ideal flow would be:

1. Use credentials as explicitly passed by the user.
   • Passing in the bytes of the JSON file as currently done for backwards compatibility.
   • Passing in a Credentials object from google-auth.
2. Try to load cached user-auth credentials.
3. Use service account from PANDAS_GBQ_CREDENTIALS_FILE env var. (Should we keep doing this?)
4. Try to create default credentials using google-auth.
5. Do three-legged OAuth to get user credentials.

Note that google-auth default credentials does not do three-legged OAuth, so I think that's an area where pandas-gbq will need to keep some auth code around.

Edit: Swapped items (1) and (2) so that explicitly passed credentials are always used first. Also, current behavior does default credentials before PANDAS_GBQ_CREDENTIALS_FILE. If we keep PANDAS_GBQ_CREDENTIALS_FILE around, we should prefer it to default credentials.

[max-sixty]

And if we just didn't pass any creds along - we literally did nothing there - what would happen then? How much would the google.cloud.bigquery-like libraries do?

[tswast]

The google-cloud-bigquery library just does (3) default credentials.

[max-sixty]

OK, thanks. I'll ponder

[caiolopes]

Just a small suggestion on this subject, I really enjoy how this library handles the authentication:

https://github.com/nithinmurali/pygsheets

It gives me more freedom than pandas-gbq right now.

It let the user pass a custom credentials object and decide where to save the oauth store instead of ~./config/pandas-gbq/bigquery_credentials.dat

pandas-gbq is giving me problems when running at appengine right now because it ignores that I set PANDAS_GBQ_CREDENTIALS_FILE and uses the default service account for my appengine app.

EDIT:

I saw now that pygsheets uses oauth2client which is deprecated, but anyway, I liked how it gives more control over the authentication.

[caiolopes]

For example, the method get_application_default_credentials gets my project's default service account at appengine first instead of the json file at PANDAS_GBQ_CREDENTIALS_FILE.

And the worst part is that it doesn't allow me to modify this with any parameter in the read_gbq, to_gbq.

[tswast]

@caiolopes Yes, that's good feedback. I actually forgot about PANDAS_GBQ_CREDENTIALS_FILE. If we keep that around, it should definitely take precedent before default credentials.

I should also swap (1) and (2) in my above comment. If ever supplied a google-auth Credentials object, we should always use that.

[tswast]

I have fleshed out a design that accounts for this feedback at #161. Closing this issue as a duplicate of that proposal.