AWS Bedrock - Programmatically fetch model-specific JSON schemas for InvokeModel requests

[cohml]

Describe the feature

I'd like a way to programmatically retrieve metadata on which inference parameters are available on a model-by-model basis. Bonus points for doing the same for response parameters as well.

Use Case

I have an app through which users can interact with multiple Bedrock foundation models. However, different models accept different inference parameters, and also not every model accepts a "prompt" key. This complicates the creation of a standardized interface for my app.

Unfortunately, asking model developers to standardized their schemas is a nonstarter. So instead, if the AWS CLI provided a way to fetch this information, that would go a long way towards simplifying things. Without that, I simply have to visit the model documentation page enumerating a request schema (example), then somehow hardcode that into my application, then endlessly maintain it as breaking changes are introduced.

Proposed Solution

aws bedrock list-foundation-models outputs an array of model summaries such as the following:

$ aws bedrock list-foundation-models | jq '.modelSummaries[0]'
{
  "modelArn": "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-tg1-large",
  "modelId": "amazon.titan-tg1-large",
  "modelName": "Titan Text Large",
  "providerName": "Amazon",
  "inputModalities": [
    "TEXT"
  ],
  "outputModalities": [
    "TEXT"
  ],
  "responseStreamingSupported": true,
  "customizationsSupported": [],
  "inferenceTypesSupported": [
    "ON_DEMAND"
  ],
  "modelLifecycle": {
    "status": "ACTIVE"
  }
}

Perhaps a requestSchema field could be added which maps to a JSON schema for the request, e.g.:

$ aws bedrock list-foundation-models | jq '.modelSummaries[0]'
{
  "modelArn": "arn:aws:bedrock:us-east-1::foundation-model/amazon.titan-tg1-large",
  "modelId": "amazon.titan-tg1-large",
  "modelName": "Titan Text Large",
  "providerName": "Amazon",
  "inputModalities": [
    "TEXT"
  ],
  "outputModalities": [
    "TEXT"
  ],
  "responseStreamingSupported": true,
  "customizationsSupported": [],
  "inferenceTypesSupported": [
    "ON_DEMAND"
  ],
  "modelLifecycle": {
    "status": "ACTIVE"
  },
  "requestSchema": {  # <-- ADD ME
    "$schema": "http://json-schema.org/draft-07/schema#",
    "type": "object",
    "properties": {
      "inputText": {
        "type": "string"
      },
      "textGenerationConfig": {
        "type": "object",
        "properties": {
          "temperature": {
            "type": "number"
          },
          "topP": {
            "type": "number"
          },
          "maxTokenCount": {
            "type": "integer"
          },
          "stopSequences": {
            "type": "array",
            "items": {
              "type": "string"
            }
          }
        },
        "additionalProperties": false
      }
    },
    "required": [
      "inputText"
    ],
    "additionalProperties": false
  }
}

If an equivalent responseSchema field could also be added, that would be equally useful for the same reason.

This proposal would satisfied the original need with only minimal perturbations to the current AWS CLI.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CLI version used

aws-cli/2.17.13 Python/3.11.9 Darwin/23.5.0 exe/x86_64

Environment details (OS name and version, etc.)

MacOS Sonoma 14.5

[cohml]

Or if there is already some way to programmatically fetch this information, even if not via the AWS CLI, I am all ears. My attempts to Google around have all fallen flat.

[tim-finnigan]

Thanks for the feature request. I don't think it is currently possible to fetch this information via an API, but I could see how that would be useful. The Bedrock service team would need to implement this, as they own and maintain the underlying APIs (https://docs.aws.amazon.com/bedrock/latest/APIReference/API_Operations.html). Maybe the inference parameters for models could be included in the GetInferenceProfile API response.

I'll create an internal feature request for the Bedrock team to consider this. In the meantime you can use the User Guide or console to find this information. Going forward you can refer to the blog and CHANGELOG for updates. I'm going to close the issue since this won't be tracked on the AWS CLI side. But any updates to service APIs like this would be released to the CLI/SDKs. And if there's any additional info you want us to pass along to the Bedrock team please let us know.

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.