Comparative Analysis of ISP Link Performance: iperf3 Versions 3.1.1 vs. 3.17.1

[doublebaung]

Hi All

I conducted tests on my ISP link using iperf3 version 3.17.1 and 3.1.1 to evaluate the upload bandwidth required for a streaming application. This application needs a minimum upload bandwidth of 50 Mbps to send 10 parallel streams, each requiring 5 Mbps, to a server in Singapore.

During the test with iperf3 version 3.1.1, I experienced significant packet loss. However, when I tested with version 3.17.1, there was no packet loss. Below are the test results from both versions:

Version 3.1.1 Result

iperf3 -c 89.187.162.1 -u -b 5M -P 10 -t 15
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 4] 0.00-15.01 sec 8.89 MBytes 4.97 Mbits/sec 4.486 ms 223/1134 (20%)
[ 6] 0.00-15.01 sec 8.89 MBytes 4.97 Mbits/sec 4.418 ms 204/978 (21%)
[ 8] 0.00-15.01 sec 8.89 MBytes 4.97 Mbits/sec 6.191 ms 274/1135 (24%)
[ 10] 0.00-15.01 sec 8.89 MBytes 4.97 Mbits/sec 4.573 ms 254/1134 (22%)
[ 12] 0.00-15.01 sec 8.89 MBytes 4.97 Mbits/sec 6.870 ms 273/1135 (24%)
[ 14] 0.00-15.01 sec 8.89 MBytes 4.97 Mbits/sec 6.025 ms 383/1135 (34%)
[ 16] 0.00-15.01 sec 8.89 MBytes 4.97 Mbits/sec 5.236 ms 320/1135 (28%)
[ 18] 0.00-15.01 sec 8.89 MBytes 4.97 Mbits/sec 6.310 ms 330/1135 (29%)
[ 20] 0.00-15.01 sec 8.89 MBytes 4.97 Mbits/sec 6.264 ms 436/1135 (38%)
[ 22] 0.00-15.01 sec 8.89 MBytes 4.97 Mbits/sec 4.667 ms 363/1135 (32%)
[SUM] 0.00-15.01 sec 88.9 MBytes 49.7 Mbits/sec 5.504 ms 3060/11191 (27%)

Version 3.17.1 Result

iperf3 -c 89.187.162.1 -u -b 5M -P 10 -t 15
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-15.01 sec 8.94 MBytes 5.00 Mbits/sec 0.000 ms 0/6418 (0%) sender
[ 5] 0.00-15.05 sec 7.24 MBytes 4.03 Mbits/sec 0.997 ms 0/6418 (0%) receiver
[ 7] 0.00-15.01 sec 8.94 MBytes 5.00 Mbits/sec 0.000 ms 0/6418 (0%) sender
[ 7] 0.00-15.05 sec 7.06 MBytes 3.93 Mbits/sec 0.793 ms 0/6416 (0%) receiver
[ 9] 0.00-15.01 sec 8.94 MBytes 5.00 Mbits/sec 0.000 ms 0/6418 (0%) sender
[ 9] 0.00-15.05 sec 7.20 MBytes 4.01 Mbits/sec 0.728 ms 0/6417 (0%) receiver
[ 11] 0.00-15.01 sec 8.94 MBytes 5.00 Mbits/sec 0.000 ms 0/6418 (0%) sender
[ 11] 0.00-15.05 sec 7.22 MBytes 4.02 Mbits/sec 1.023 ms 0/6418 (0%) receiver
[ 13] 0.00-15.01 sec 8.94 MBytes 5.00 Mbits/sec 0.000 ms 0/6418 (0%) sender
[ 13] 0.00-15.05 sec 7.34 MBytes 4.09 Mbits/sec 1.094 ms 0/6418 (0%) receiver
[ 15] 0.00-15.01 sec 8.94 MBytes 5.00 Mbits/sec 0.000 ms 0/6418 (0%) sender
[ 15] 0.00-15.05 sec 7.36 MBytes 4.10 Mbits/sec 0.885 ms 0/6414 (0%) receiver
[ 17] 0.00-15.01 sec 8.94 MBytes 5.00 Mbits/sec 0.000 ms 0/6418 (0%) sender
[ 17] 0.00-15.05 sec 7.44 MBytes 4.14 Mbits/sec 1.050 ms 0/6418 (0%) receiver
[ 19] 0.00-15.01 sec 8.94 MBytes 5.00 Mbits/sec 0.000 ms 0/6418 (0%) sender
[ 19] 0.00-15.05 sec 7.18 MBytes 4.00 Mbits/sec 1.019 ms 0/6418 (0%) receiver
[ 21] 0.00-15.01 sec 8.94 MBytes 5.00 Mbits/sec 0.000 ms 0/6418 (0%) sender
[ 21] 0.00-15.05 sec 7.31 MBytes 4.07 Mbits/sec 1.003 ms 0/6414 (0%) receiver
[ 23] 0.00-15.01 sec 8.94 MBytes 5.00 Mbits/sec 0.000 ms 0/6418 (0%) sender
[ 23] 0.00-15.05 sec 7.43 MBytes 4.14 Mbits/sec 1.036 ms 0/6416 (0%) receiver
[SUM] 0.00-15.01 sec 89.4 MBytes 50.0 Mbits/sec 0.000 ms 0/64180 (0%) sender
[SUM] 0.00-15.05 sec 72.8 MBytes 40.5 Mbits/sec 0.963 ms 0/64167 (0%) receiver

I want to understand whether the results indicate that my internet bandwidth is sufficient for the streaming application requiring 50 Mbps upload bandwidth, with 5 Mbps allocated for each stream.

The 3.1.1 version shows significant packet loss (27% total loss), suggesting that the network may not be capable of handling the required upload bandwidth effectively during the test.
In contrast, the 3.17.1 version demonstrates no packet loss and maintains the required throughput, indicating that the network can support the upload demands of the application.
This discrepancy raises questions about the reliability of iperf3 version 3.1.1 in accurately measuring network performance compared to version 3.17.1.

[davidBar-On]

Hi @doublebaung,
Note that you opened this discussion twice - this and #1748, so one of them can be closed.

Version 3.1.1 is very old, so it is not recommended to use it.

Regarding the results you see with 3.17.1, the results from the server ("receiver") are strange. While it reports receiving almost all of the packets (usually the last packets are not received because of the delay), the transfer rate is lower than the client. Since this is a public server, I assume you don't know what iperf3 version it is using. Therefore, I suggest to try another public server and to see whether its statistics are reasonable there.

Regarding what you are trying to measure. I suggest to also try similar test using TCP, without the -b. It should give you an idea about the maximum rate possible. Note that by default, iperf3 TCP packets are much larger than UDP, so you should set -l for the TCP test using the UDP packet length (you can get by running the UDP test with -V and using the "byte blocks" from "Setting UDP block size to ...").

[bmah888]

I'd also advise against using iperf 3.1.1. Since it was released in 2015, a number of enhancements and bug fixes have been added to the code. As pointed out by @davidBar-On, version 3.17.1 is the current version of iperf3. In particular there were a few changes around the time of iperf 3.2 that decreased the default UDP send size to a value that avoids fragmenting the UDP packets (that fragmentation can lead to an increased chance of packet loss).

I suggest trying to use (as closely as possible) the same transport protocol (TCP vs UDP) and other parameters as your real application. In particular if your application uses TCP, then you should probably use TCP iperf3 tests (not UDP) in order to see the effects of TCP's congestion control and retransmission. Also note that TCP is a bytestream based protocol, and automatically handles breaking up large application-layer sends into packets that the network can handle.

Also, as @davidBar-On pointed out, if you are using a public server, you don't have any control, or might not know very much, about the capabilities and performance of the server.