output text and json generates different output

[leventyalcin]

Describe the bug

Whilst I was trying to get the latest tag of ecr images, I came across an interesting output. The text output type is generating output for filtered information.

Expected Behavior

aws --output json \
  ecr describe-images 
    --repository-name REPONAME 
    --registry-id REGID 
    --query 'sort_by(imageDetails,& imagePushedAt)[-1].imageTags[0]'

"0.0.134"

Current Behavior

aws --output text \
  ecr describe-images \
    --repository-name REPONAME \
    --registry-id REGID \
    --query 'sort_by(imageDetails,& imagePushedAt)[-1].imageTags[0]'

0.0.132
0.0.131
0.0.134

Reproduction Steps

Enough to run the commands above against a repo where there are multiple image tags

Possible Solution

No response

Additional Information/Context

No response

CLI version used

aws-cli/2.11.17 Python/3.11.3 Darwin/22.4.0 source/x86_64 prompt/off

Environment details (OS name and version, etc.)

macOS 13.3.1 (a)

[RyanFitzSimmonsAK]

Hi @leventyalcin, thanks for reaching out. This is actually intended behavior, and we've had a couple issues about it in the past. The relevant documentation mentions this behavior, saying the following :

"If you specify --output text, the output is paginated before the --query filter is applied, and the AWS CLI runs the query once on each page of the output. Due to this, the query includes the first matching element on each page which can result in unexpected extra output. To additionally filter the output, you can use other command line tools such as head or tail.

If you specify --output json, --output yaml, or --output yaml-stream the output is completely processed as a single, native structure before the --query filter is applied. The AWS CLI runs the query only once against the entire structure, producing a filtered result that is then output."

For updates about this behavior being better documented, please refer to the issue I linked above. Hope that helps, and let me know if you have any follow-up questions.

[leventyalcin]

I didn't notice that part in the documentation. Thanks for sharing it.

I do now understand how it works. Not necessarily but do yo mind sharing why it was desinged the way it is (if you could remember on top of you head)? There might be a use case for it but struggling to understand it conceptually.

[RyanFitzSimmonsAK]

Hi @leventyalcin, thanks for following up. I spoke with the team, and they explained that text is the only format that is capable of streaming output in the first place. If another output structure (json, for example) was chosen, then it'd just be an incomplete response. I hope that answers your question, please let me know if you have any other follow-up questions or if this issue can be closed.

[leventyalcin]

Hi @RyanFitzSimmonsAK ,

Thanks for taking time and looking into the details of it. I appreciate that.

My take from this is the one stands for a text output stream and the others stand for a structured data model output. So, text output is not considered as a data model. If my understanding is close enough, I totally understand it.

Cheers,
Levent.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.