Skip to content

Home

Jump to bottom
Alif Merchant edited this page Dec 4, 2024 · 6 revisions

Welcome to the Aurora Dsql Benchbase wiki!

This repository is a fork of the cmu-db/benchbase repository. All the available benchmarks can be found on their hosted wiki page.

Loading data and Running TPC-C against an Aurora DSQL Cluster

In this repository, there is a sample configuration file and ddl files to work with an Aurora DSQL cluster.

Before running any workloads:

Edit the config/auroradsql/sample_tpcc_config.xml file This file contains various settings that can be adjusted based on how users want to run the benchmarking test.

  1. Replace localhost in the <url></url> tag with an Aurora DSQL cluster endpoint.
  2. Set the username and the password token inside the <username></username> and <password></password> tags. To generate a password token, follow this guide.

This fork supports automatic password/token generation using IAM authentication for Aurora DSQL workloads. To use it, simply leave the <password></password> field empty. To understand where the credentials and region information are fetched from, checkout these libraries:

  1. Set the <scalefactor></scalefactor> to the number of TPC-C warehouses you would like to create and run the benchmark on.
  2. Set the workload parameters. These settings will be used by the workload in the next step. Change the number of <terminals></terminals> to determine how many concurrent connections to make. These connections are evenly spread over the number of warehouses. Note that more terminals mean more memory usage.

Running the workloads and interpreting results:

Run the workload using the following command:

java -jar benchbase.jar -b tpcc -c config/auroradsql/sample_tpcc_config.xml --create=false --load=false --execute=true

Once the workload has finished running, Benchbasa will output a histogram in the terminal, several .csv files in the results folder and a summary.json file in the same folder. For a 1-warehouse run with 10 terminals, the histogram would look like this:

[INFO ] 2024-11-21 19:10:23,042 [main]  com.oltpbenchmark.DBWorkload runWorkload - ======================================================================
[INFO ] 2024-11-21 19:10:23,043 [main]  com.oltpbenchmark.DBWorkload runWorkload - Rate limited reqs/s: Results(state=EXIT, nanoSeconds=60999908412, measuredRequests=3168) = 51.93450420618642 requests/sec (throughput), 52.24598008368564 requests/sec (goodput)
[INFO ] 2024-11-21 19:10:23,049 [main]  com.oltpbenchmark.DBWorkload writeOutputs - Output Raw data into file: tpcc_2024-11-21_19-10-23.raw.csv
[INFO ] 2024-11-21 19:10:23,081 [main]  com.oltpbenchmark.DBWorkload writeOutputs - Output samples into file: tpcc_2024-11-21_19-10-23.samples.csv
[INFO ] 2024-11-21 19:10:23,088 [main]  com.oltpbenchmark.DBWorkload writeOutputs - Output summary data into file: tpcc_2024-11-21_19-10-23.summary.json
[INFO ] 2024-11-21 19:10:23,096 [main]  com.oltpbenchmark.DBWorkload writeOutputs - Output DBMS parameters into file: tpcc_2024-11-21_19-10-23.params.json
[INFO ] 2024-11-21 19:10:23,096 [main]  com.oltpbenchmark.DBWorkload writeOutputs - Output benchmark config into file: tpcc_2024-11-21_19-10-23.config.xml
[INFO ] 2024-11-21 19:10:23,131 [main]  com.oltpbenchmark.DBWorkload writeOutputs - Output results into file: tpcc_2024-11-21_19-10-23.results.csv with window size 5
[INFO ] 2024-11-21 19:10:23,149 [main]  com.oltpbenchmark.DBWorkload writeHistograms - ======================================================================
[INFO ] 2024-11-21 19:10:23,149 [main]  com.oltpbenchmark.DBWorkload writeHistograms - Workload Histograms:

Completed Transactions:
com.oltpbenchmark.benchmarks.tpcc.procedures.NewOrder/01                         [1473]
com.oltpbenchmark.benchmarks.tpcc.procedures.Payment/02                          [1330] *
com.oltpbenchmark.benchmarks.tpcc.procedures.OrderStatus/03                      [ 136] *
com.oltpbenchmark.benchmarks.tpcc.procedures.Delivery/04                         [ 130] *
com.oltpbenchmark.benchmarks.tpcc.procedures.StockLevel/05                       [ 118] *

Aborted Transactions:
com.oltpbenchmark.benchmarks.tpcc.procedures.NewOrder/01                         [  15]

Rejected Transactions (Server Retry):
com.oltpbenchmark.benchmarks.tpcc.procedures.NewOrder/01                         [  91] *
com.oltpbenchmark.benchmarks.tpcc.procedures.Payment/02                          [3468]
com.oltpbenchmark.benchmarks.tpcc.procedures.Delivery/04                         [ 173] **

Rejected Transactions (Retry Different):
<EMPTY>

Unexpected SQL Errors:
<EMPTY>

Unknown Status Transactions:
<EMPTY>


[INFO ] 2024-11-21 19:10:23,149 [main]  com.oltpbenchmark.DBWorkload writeHistograms - ======================================================================

This result indicates that the test ran for 60 seconds and processed 3168 transactions, of which 1473 were New Order transactions. The number of New Order transactions is your tpmC. If the test runs for longer than 60 seconds, you can calculate the tpmC by dividing the number of completed New Order Transactions / the test duration in minutes.

(Checkout the TPC-C documentation to understand how the Tpmc is calculated: https://www.tpc.org/tpc_documents_current_versions/pdf/tpc-c_v5.11.0.pdf)

Clone this wiki locally