Query for binary values

[fakraemer]

I struggle to understand how I am supposed to query for binary values with the aws-cli. My sort key, seq, is of type binary. When I use the AWS console I can just input the base64 encoded value:


I tried to write the same query with the AWS CLI (aws-cli/1.10.9 Python/2.7.9 Linux/3.16.0-30-generic botocore/1.4.0):

aws dynamodb query --table-name <mytable> --key-condition-expression '#mykey = :keyval AND seq = :seqval' --expression-attribute-values '{ ":keyval":{"S":"1"}, ":seqval": {"B":"AAAAAAAAbR4AAAAAAAAAAA=="}}' --expression-attribute-names '{ "#mykey":"key" }'
{
    "Count": 0, 
    "Items": [], 
    "ScannedCount": 0, 
    "ConsumedCapacity": null
}

When I look at the tcpdump, I can see that the binary value is base64 encoded again:

{
  "KeyConditionExpression": "#mykey =:keyval AND seq =:seqval",
  "TableName": "<mytable>",
  "ExpressionAttributeValues": {
    ":seqval": {
      "B": "QUFBQUFBQUFiUjRBQUFBQUFBQUFBQT09"
    },
    ":keyval": {
      "S": "1"
    }
  },
  "ExpressionAttributeNames": {
    "#mykey": "key"
  }
}

How am I supposed to communicate binary values on the CLI?

[kyleknap]

This is a duplicate of: #1097

In short, there is not really any workaround.

In general, we always base64 encode binary values. That is a standard practice for most of the SDKs across the various languages. Handling nested binary is probably one of the larger problems in the CLI for a couple reasons:

1. From the command line, you can not input literal binary. We also cannot make the assumption that the values are already base64 encoded because it is perfectly fine to input text as binary, and users should not have to base64 encode text.
2. Passing values through file:// and fileb:// can only be used as top level values: they cannot be nested values. They cannot be nested values because it would be a breaking change to go in and add them as users may actually be using file:// and fileb:// as literal values.

The ideal solution would allow users to pass values in files into nested structures, but we have yet to come up with a good solution that is backwards compatible and feels right. Another intermediate option we have thought of is add a --no-base64-encode flag that will make sure that all binary types are not base64 encoded. However, that would not allow for any granularity in which params to base64 encode and which ones not too. If you any suggestions, that would be great. Otherwise, let's follow this issue in the original issue I linked.

[fakraemer]

Apologies for raising this again, I had a short search for a similar issue but did stumble over the referenced issue. Thanks for the confirmation and explanation, personally I feel assuming base64 encoding for any binary types would have been the right call to make, but I understand that you would be reluctant to break current behaviour. In this context, the --no-base64-encode flag seems to me the easiest way to go about it.