Warn when using user credentials from the Cloud SDK

[theacodes]

No description provided.

[max-sixty]

Hi @theacodes & @crwilcox,

Each time we run a BigQuery request we now get this warning. Does this warn every time a user makes a request with their credentials? If so, is that balancing information and noise well? My understanding is that there are lots of legitimate uses of user accounts.

I'm also a committer of https://github.com/pydata/pandas-gbq under @tswast, where we intentionally fully support user credentials - potentially conflicting with this warning.

Thanks

[tswast]

One reason I made googleapis/python-bigquery-pandas#161 is to make it easier to explicitly use user credentials when those are desired.

In pandas-gbq you can disable default credentials by setting the GOOGLE_APPLICATION_CREDENTIALS environment variable to empty string to avoid this warning.

[theacodes]

This warning shows up every time you call google.auth.default() and the credentials resolved are associated with the Cloud SDK's client ID. You can cache the return value or filter the warning using filterwarnings.

[tswast]

Yeah, it's also on my list to cache the credentials in pandas-gbq. :-)

[max-sixty]

This warning shows up every time you call google.auth.default() and the credentials resolved are associated with the Cloud SDK's client ID

Can you help me understand why this warns? We use this code path for all our research. (then when we deploy to GKE it resolves to the service account)

I can understand that running servers on user auth is bad, but presumably lots of non-server applications are going to hit this code path?

[theacodes]

I summarized over at #271 (comment)

The trouble with the Cloud SDK is that all credentials that are issued from it share the same client ID, and thus, share quota. This doesn't work well for a significant portion of our APIs and often leads to users being confused and frustrated when things don't work.

Also requiring users to install a 300mb SDK that they may not need just to auth for a script is massive overkill.

@maxim-lian are you using pandas-gbq? If so, it's already equipped to do the right thing™ here and obtain authorization itself.

[max-sixty]

That comment makes sense on the intention.

Forgive me if these questions seem pedestrian (and @tswast has already spent lots of time hand-holding me through some auth questions): what would you recommend for us? We run a set of processes that interact with google services extensively - e.g. read & write to BQ, GCS. We run these on local containers when we're doing research, and then on GKE in production.

User accounts are extremely convenient (much more so than having everyone create a service account, and then managing permissions for those in parallel to user permissions).

Is this a ClientID issue? Should we be using a different one?

Feel free to point me towards documentation if it's out there.

Thanks

[tswast]

With local containers, my recommendation would be to let pandas-gbq do the user auth flow. Revoke the default credentials with

gcloud auth application-default revoke

to make pandas-gbq and other tools stop picking up the user auth from gcloud.

[max-sixty]

Thanks Tim

Would this generalize to using GCS, for example?

[tswast]

No, it doesn't generalize to GCS, unless we create a pandas-gbq-like wrapper for GCS, too.

The pandas_gbq.get_user_credentials function I propose in googleapis/python-bigquery-pandas#161 would work for creating a GCS client, but then your code would be explicitly using user credentials, unlike the "default" auth method.