parameter error - block size too large

[IbrahimBabiker87]

Hello,

Why Im seeing this error: the buffer size is over 32M in system

ad@intel-bm:~$ sudo iperf3 -c 10.45.0.1  -p 5210  --bind 10.45.0.10 --bind-dev wwan0 -t 86000  -R -w 128M  -l 32M
iperf3: parameter error - block size too large (maximum = 1048576 bytes)

Usage: iperf3 [-s|-c host] [options]
Try `iperf3 --help' for more information.

net.core.default_qdisc=fq
net.ipv4.tcp_congestion_control=bbr
net.ipv4.tcp_window_scaling=1
net.ipv4.tcp_rmem=87380 67108864 268435456
net.ipv4.tcp_wmem=87380 67108864 268435456
net.core.rmem_default = 268435456
net.core.rmem_max = 268435456
net.core.wmem_default = 268435456
net.core.wmem_max = 268435456
net.ipv4.tcp_mem = 87380 67108864 268435456

[bmah888]

This is totally an internal iperf3 limit. iperf3 has a sanity check limit of 1MB for the send block size.

This number has nothing to do with the socket buffer sizes, it's about the amount of data that iperf3 will try to send in a single send() socket call.

I'm not sure how valuable this limit is, to be honest. It's in the source code as MAX_BLOCKSIZE in src/iperf.h if you feel like changing it in your local sources and recompiling.

[IbrahimBabiker87]

Hi @bmah888 ,
Appreciate your response,

So basically this value will not affect the send block size as you suggested.

The issue is that I'm testing with a 5G modem connected to server with iperf3, I'm getting the maximum steady UDP throughput of 1.4Gbps, however the TCP throughput is very poor ~80 - 100 Mbps compared to UDP, noting the RRT is ~25 - 35 ms. I'm expecting more than that,

appreciate any ideas to solve the issue or some clues to define the root cause, or any special settings in iperf3 to increase the TCP throughput.
I used congestion control of BBR as it gave the best TCP throughput so far as well as above mentioned Kernel parameters.

[davidBar-On]

Hi, I assume that for the UDP test you also used -R and that the 1,4Gbps where measured on the receiving side ("ad@intel-bm"). What buffer sizes did you use for UDP and TCP? For UDP the default is usually 1460 bytes and for TCP 128KB. If you didn't try that yet, can you try setting the TCP buffer size to the one used by UDP (e.g. -l 1460)? In general, it can help testing with different values of TCP buffer sizes (starting from e.g. 512) and see how it effects the throughput.

Also, does TCP SACK (Selective Ack) enabled on your system, especially on the sending device (because of the -R I assume it is the cellular device)? To make sure, you can check the TCP traffic using tools like WireShark. The point is that since in cellular networks a packet loss is quite common, if SACK is not used there is relatively high chance that the whole TCP packet will be lost and not just a segment.