Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Load isn't getting generated in case transaction size is higher than 1. #66

Open
AndreyKoltsov1997 opened this issue Jun 22, 2021 · 1 comment · May be fixed by #67
Open

Load isn't getting generated in case transaction size is higher than 1. #66

AndreyKoltsov1997 opened this issue Jun 22, 2021 · 1 comment · May be fixed by #67

Comments

@AndreyKoltsov1997
Copy link

In case transactions are enabled and events per transaction is set to value higher than 1, load isn't getting generated.

Example
Pravega Driver manifest - standalone, enabled transactions, 3 events per transaction.

name: Pravega
driverClass: io.openmessaging.benchmark.driver.pravega.PravegaBenchmarkDriver
...
client:
  controllerURI: "tcp://localhost:9090"
....
enableTransaction: True
eventsPerTransaction: 3 # any value higher than 1
createScope: True
...

Load manifest - 1 topic, 1 producer (writer), 1 consumer (reader):

name: 1 topic / 1 partition / 100b
...
topics: 1
partitionsPerTopic: 1
subscriptionsPerTopic: 1
consumerPerSubscription: 1
producersPerTopic: 1
testDurationMinutes: 1
...

In case OpenMessaging Benchmark would be executed with parameters provided above, the actual parallel read and write wouldn't be executed. Please, note that it doesn't apply to the case when eventsPerTransaction is set to 1.

...
20:01:45.434 [clientInternal-3-10] INFO io.pravega.client.stream.impl.Pinger - Start sending transaction pings.
20:01:54.921 [clientInternal-3-5] INFO io.pravega.client.stream.impl.Pinger - Start sending transaction pings.
20:02:04.408 [clientInternal-3-7] INFO io.pravega.client.stream.impl.Pinger - Start sending transaction pings.
20:02:06.430 [pool-3-thread-1] INFO io.pravega.client.stream.impl.EventStreamReaderImpl - EventStreamReaderImpl( id=ee2096f6-e635-4a32-aee3-fdf62f207d63) at checkpoint b37e3c89-02b2-4e93-987f-ebe12183a35b
20:02:06.431 [pool-3-thread-1] INFO io.pravega.client.stream.impl.EventStreamReaderImpl - Reader ee2096f6-e635-4a32-aee3-fdf62f207d63 completed checkpoint b37e3c89-02b2-4e93-987f-ebe12183a
@AndreyKoltsov1997
Copy link
Author

The issue is related to probeProducers() method that is executed prior to the actual load generation.
It expects to write 1 event from producer and read it back from the consumer. Since we commit transaction only once it's populated with the requested amount of events, the data is never sent to Pravega in case its size is higher than 1. Thus, reader (consumer) couldn't get the data from the system since it hadn't been written.

@AndreyKoltsov1997 AndreyKoltsov1997 linked a pull request Jun 22, 2021 that will close this issue
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
1 participant
@AndreyKoltsov1997