Add performance benchmark summarization script

[aemous]

Description of changes:

• Added a new detailed_json output format to the performance summarization script. It outputs measurements from all iterations of the command instead of just the average / maxima.
• Added extra metadata that gets output with each metric which is useful for downstream processing of the metrics.
• Changed Summarizer class so that it returns the raw output data in summarize_as_json and summarize_as_detailed_json functions. Useful for external classes to utilize this data for processing.

Description of tests:

• Ran the following commands to benchmark a file upload with aws s3 cp and summarize the results in the new detailed JSON format:

% ./benchmark "aws s3 cp ~/file.dmg s3://test-bucket-aem278/file.dmg" --data-interval 0.1 --output-file perf1.csv
% ./benchmark "aws s3 cp ~/file.dmg s3://test-bucket-aem278/file.dmg" --data-interval 0.1 --output-file perf2.csv
% ./benchmark "aws s3 cp ~/file.dmg s3://test-bucket-aem278/file.dmg" --data-interval 0.1 --output-file perf3.csv
% python3 ./summarize.py perf1.csv perf2.csv perf3.csv

Output:

{
  "results": [
    {
      "name": "total_time",
      "description": "The total execution time of the command.",
      "unit": "Milliseconds",
      "date": 1735074973,
      "measurements": [
        354898.06270599365,
        355706.1140537262,
        348326.67684555054
      ]
    },
    {
      "name": "max_memory",
      "description": "The maximum memory utilization of the CLI throughout execution of the command.",
      "unit": "Megabytes",
      "date": 1735074973,
      "measurements": [
        123.338752,
        134.463488,
        130.023424
      ]
    },
    {
      "name": "average_memory",
      "description": "The average memory utilization of the CLI throughout execution of the command.",
      "unit": "Megabytes",
      "date": 1735074973,
      "measurements": [
        122.78925544327932,
        117.97557115870967,
        128.00244748641975
      ]
    },
    {
      "name": "max_cpu_utilization",
      "description": "The maximum CPU utilization of the CLI throughout execution of the command.",
      "unit": "Percentage",
      "date": 1735074973,
      "measurements": [
        3.1,
        3.0,
        4.2
      ]
    },
    {
      "name": "average_cpu_utilization",
      "description": "The average CPU utilization of the CLI throughout execution of the command.",
      "unit": "Percentage",
      "date": 1735074973,
      "measurements": [
        0.008484270734032429,
        0.026161290322580504,
        0.07978835978835712
      ]
    }
  ]
}

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Please upload report for BASE (v2-performance-testing@1ae0d1f). Learn more about missing BASE report.

Additional details and impacted files

@@                   Coverage Diff                    @@
##             v2-performance-testing   #9172   +/-   ##
========================================================
  Coverage                          ?       0           
========================================================
  Files                             ?       0           
  Lines                             ?       0           
  Branches                          ?       0           
========================================================
  Hits                              ?       0           
  Misses                            ?       0           
  Partials                          ?       0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.