User credentials for BigQuery

[tswast]

Now that #1528 is implemented, it's possible to send any kind of credentials to use with BigQuery. 🎉 But, it's still a bit difficult to use user credentials for BigQuery.

The "application default credentials" used by the BigQuery client library use:

1. The metadata server to get credentials on App Engine, Compute Engine, ...
2. A service account key file, whose path is described by the GOOGLE_APPLICATION_CREDENTIALS environment variable (what Google Cloud recommends in their authentication getting started guide).
3. User credentials from the Cloud SDK gcloud command-line tool, but these credentials now raise a warning in the Google Auth layer (mostly because they don't work for some APIs such as the Vision and Speech APIs).

I propose Ibis add a fourth fall-back option (or even skip 3 altogether) to do end-user authentication itself.

The reason for this is that is makes the getting started experience a lot easier. All they'll need is Python and the library installed and they can use Ibis with BigQuery with their own credentials. No gcloud installation or service account needed.

I've created a API credentials called "Ibis Library" in the Ibis GBQ project that we can use for this.

The code at Pandas GBQ
https://github.com/pydata/pandas-gbq/blob/993fe55f7e1f44a15b282b415284774310fc530c/pandas_gbq/auth.py#L113-L175
will be useful as a model to follow.

[cpcloud]

Interesting. That module looks useful outside of pandas-gbq. How realistic would it be to move it into its own tiny library? Alternatively, we copy that code into ibis, but that seems like a poor choice for the community.

[tswast]

How realistic would it be to move it into its own tiny library?

Realistic. If I make it, then it'll be a bit of red tape per Google's OSS policies, but still doable. Alternatively, someone could seed a repo with an appropriate LICENSE file and we can move the user auth code over.

[tswast]

I think this would best fit in the PyData org rather than under Google's umbrella, so I've put in a request at https://groups.google.com/d/topic/pydata/A0tPgegh9og/discussion If that doesn't go through, I'll go through the Google internal channels to create a repo in the GCP org.

[tswast]

I've started the auth-only fork of pandas-gbq over at https://github.com/pydata/pydata-google-auth

I'm making some adjustments to pandas-gbq auth over at googleapis/python-bigquery-pandas#161 so I'll use this as an opportunity to do it right for both Ibis & Pandas at the same time.

Currently I'm thinking we'll still want each package to have it's own client ID and secret. The reason being is that potentially the user might want different scopes for the two different packages, but I could probably be convinced to consolidate them all under one PyData banner if we think that's not going to be an issue.

[tswast]

I'm converting pandas-gbq to use pydata-google-auth in googleapis/python-bigquery-pandas#241.

For Ibis to use it, we'll want to call pydata_google_auth.default() just before we create the BigQuery client if the credentials argument is None.

We should use the client ID and client secret from the ibis-gbq GCP project.

[cpcloud]

Going to get this in for 1.0.0, which we are shooting to release next week.