Add AWS IAM Authentication

[yuriytkach]

This PR provides AWS IAM authentication mechanism for the Vault.

AWS client dependency is set as provided, so in order to use the authentication, the library must be on the application's dependency list. However, here I am not sure. Looks like the dependency must always be on classpath judging from the integration tests. Please, advice what would be the correct way to specify dependency here.

Integration test is added, which starts the Localstack container with STS and IAM services enabled. Please tell me if more tests are needed.

Relates to #39

[yuriytkach]

Hi @vsevel @sberyozkin @kdubb , any thoughts on this?

[yuriytkach]

Guys, I would need your help here since I am not sure if this feature (AWS auth) is possible without explicit dependency on AWS sdk if native compilation is required.

From the workflow run I could see that all tests pass in jvm mode with the AWS sdk dependency staying only in test scope since it is required by VaultTestExtension.

However, during native build in the vault-app module the class VaultInternalAwsIamAuthMethod cannot be created without the AWS sdk being present in compile scope. Maybe not the full sdk is needed and only some of its dependencies, but the apache-client definitely should be excluded (because of dependency on commons-logging).

Now, the vault-app project does not use the AWS auth for Vault, so AWS sdk should not be required. But it is required for native compilation to work.

Therefore a question: is there a possibility to have VaultInternalAwsIamAuthMethod included and be created at the build phase in native mode without AWS sdk classes present? Or maybe it can be excluded from build phase and move to runtime?

Sorry, if I am asking silly questions. But for now I don't know how to make dependency on AWS sdk optional if AWS auth is not required and native mode support is needed.

I would greatly appreciate any answer? Thanks

@vsevel

[vsevel]

should that be a separate extension then?

[kdubb]

How many REST calls are made to IAM? Is it something we can ourselves? If it's 2 or 3 simple REST calls then maybe just use the Vert.x WebClient.

I'm all for re-use but the AWS SDK is quite big and this might be a time for some constrained reproduction.

[yuriytkach]

How many REST calls are made to IAM? Is it something we can ourselves? If it's 2 or 3 simple REST calls then maybe just use the Vert.x WebClient.

I'm all for re-use but the AWS SDK is quite big and this might be a time for some constrained reproduction.

No calls are made to IAM from extension. The Vault itself will make a call to IAM. I am only using the Aws4Signer with some model classes from AWS sdk.

[yuriytkach]

How many REST calls are made to IAM? Is it something we can ourselves? If it's 2 or 3 simple REST calls then maybe just use the Vert.x WebClient.
I'm all for re-use but the AWS SDK is quite big and this might be a time for some constrained reproduction.

No calls are made to IAM from extension. The Vault itself will make a call to IAM. I am only using the Aws4Signer with some model classes from AWS sdk.

Ah, also the getting of aws credentials using the DefaultCredentialsProvider from AWS SDK).

[yuriytkach]

should that be a separate extension then?

I though about that, too, since I can't figure out the solution to get rid of aws sdk dependency. But then, I am not sure if it will be possible to depend on quarkus-vault extension and re-use it's internal classes to hook into authentication. Other way I thought that I can do aws auth in Vault completely by myself, then get the token from Vault and inject it into config so that quarkus-vault extension picks it up. Surely, the token can expire and I would need to re-auth, and that seems like duplicating functionality that you already have. Though, on the other hand, for something short-lived like aws lambda that might work.

[kdubb]

Forgive me, I have not gone through your PR yet. I was just suggesting that off the top of my head.

So basically we only need to get the AWS credentials?

AWS seems to have about 4000 methods for retrieving the credentials but the ones supported by most of the libraries we use are: env variables, config file and profile (which involves an HTTP call); in order of complexity to reproduce.

Since this is IAM authentication I'm going to assume profiles must be supported but IIRC it's just tagging a REST endpoint using an un-routable address (that I've forgetten).

[kdubb]

I think Go's AWS packages have a completely separate package for getting credentials and even one for using the credentials endpoint. Have you checked for that with Java?

[yuriytkach]

Frankly, I've never met a separate Java lib for getting aws credentials. In any case, there are as you said tons of possibilities and re-implementing them all seems like a overkill.

[kdubb]

@yuriytkach Both Aws4Signer and DefaultCredentialsProvider are in software.amazon.awssdk:auth which has a fairly small POM. The biggest dependency is AWS's sdk-core but that's fairly small (both by # of classes and size of 851k).

Additionally, you'd need an AWS SDK HTTP provider. Since this is for generating credentials you could use their Java HttpURLConnection provider which has very few dependencies. There's probably no real advantage to using Netty, but we are already pulling in all the Netty libs (for Vert.x) so it's probably the same "weight" for us.

Can you make this work by adding just those specific dependencies?

[yuriytkach]

@kdubb Thanks for the tip. I will work on that in the next week

[kdubb]

@yuriytkach I think much of this can/should be moved to the new client in #215. What do you think? We'll still need the Quarkus configuration parts but all the api can be generated via the new client.

If you can please build it locally and comment on #214 with ideas and/or #215 with specific suggestions for changes. There's no documentation on the yaml format for the generator yet but I'd like to get an outside perspective on implementing new apis (especially auth ones) with the generator and client structure; you should be good to look at the current yaml files to get a feel on it's usage.