Ostinato for Linux Maximum Transmit Rate

Update (2021): Ostinato now supports 10G, 25G and 40Gbps line-rate rate traffic using the Turbo Transmit add-on.

I’ve been intermittently posting results for the maximum transmit rate performance test with Ostinato 0.9. See results for Live ISO, MacOS.

Now we do the same test on Linux - specifically Ubuntu 16.04.3 LTS. I expected the results to be same as the Live ISO since it is also Linux based, but I was in for a surprise!.

I configured the same single stream as earlier -

• Protocols: Mac, Ethernet II, IPv4, UDP, Pattern
• Source and Destination Mac addresses populated with the actual Mac addresses of the source and sink
• Source and Destination IP addresses populated with the actual IP addresses of the source and sink
• Packets/sec: 0
• Number of packets: 10 (default)
• After this stream: Goto first

Performance figures

Here are the numbers -

Packet Size	Max Rate (Kpps)	Max Rate (Mbps)
64	920	618
128	807	955
256	442	976
512	231	983
1024	118	986
1518	80	984

Mbps rate was calculated using the below formula to take into account the ethernet line overhead of 20 bytes (1 byte SFD + 7 byte preamble + 12 byte inter-packet gap) -

Rate (in Mbps) = (PacketSize + 20) * kppsRate * 8 / 1000

Specs

Software

• Ostinato 0.9 - for Linux (Ubuntu 16.04, 64-bit)
• Ubuntu 16.04.3 LTS

Hardware

• Processor: Xeon E5-2620v4 @ 2.10GHz (8-core)
• RAM: 32GB
• NIC: Intel I218-V 1Gbps

Observations

• Although it’s a 8-core CPU, only one core reaches 100% CPU during the test (same as Live ISO) - as expected since for a single port, Ostinato doesn’t use more than one core
• RAM usage - 26MB (value of RES from top); this is lower than the 98MB used in the LiveISO
• Max line rate (well - quite close to it) is reached for 256 byte packets and larger, same as the Live ISO and MacOS.
• For smaller packet sizes, Ubuntu performs better (64 - 618Mbps, 128 - 955Mbps) than the Linux ISO (64 - 403Mbps, 128 - 702Mbps) - in fact, the 128 byte rate is quite close to line rate
• Speculating on what could cause this performance difference on the two different Linux distros -
  • The application code is exactly the same for both Ubuntu and LiveISO
  • In the packet transmit code path, there are no Qt APIs
  • The non application packet transmit code path is essentially composed of pcap_sendpacket() and the underlying system call(s) - confirmed by perf
  • Version comparison of these components -

Component	Ubuntu 16.04.3	Live ISO (TinyCore 8.2)
Kernel	4.13.0-32-generic (64-bit)	4.8.17-tinycore (32-bit)
glibc	2.23	2.50.3
libpcap	1.7.4-2	1.4.0

For more Ostinato posts, subscribe for email updates.