Spirent 100Gbps Capture Benchmarking
Many times there is disbelief about our sustained 100Gbps write to disk performance, but its real. We really do full sustained 100Gbps capture to disk. To prove this we used the gold standard, a Spirent 100Gbps tester to benchmark the capture performance of our system. Here are the results.
A bit of background first, Spirent is considered the market leader for network load testing. Its an outstanding product, very flexible and can generate simple to wide and very complex network loads. We`re using it in the most basic form, 100Gbps line rate Layer 2 (Ethernet/IPv4) packet generator thats fed into our 100Gbps packet capture system.
100Gbps IMIX Capture Benchmark
The go-to test for network load testing is the IMIX test, this provides a random mix of 64B, 598B, and 1518B packets. Where the idea is to simulate a mixed packet sized network workload, such that its similar to a generic packet rate and bandwidth load on a network. Its setup as follows below.
Full Packet Generator settings
Zooming on the packet generation areas:
The above settings explained below:
Black:
An IMIX packet generation algo is selected
Orange:
Default IMIX distribution, this generates a distribution of 64B, 598B, 1518B packets
Yellow:
Generating 1,000,000 Packets with an IFG gap of 12B
Green:
Generating 1,000 Bursts x 1,000,000 packets = 1 billion packets in total
This setting generates 1,000,000,000 packets at almost full line rate 95Gbits @ 32M packets per second. Its not full line rate in the sense of maximum number of bits sent down the wire. Maximum data rate is achieved using only 9200B jumbo packets.
Heres the Spirent 100G Packet Generator in action
And zooming in
Above shows its generating at a data rate of 94.7Gbps with a packet rate of 32.7M, all things considered this is a seriously heavy work load. Alot of 100G switches will fail to forward packets at this rate.
...and onto the FMADIO 100G Packet Capture device
Above shows FMAD 100G Packet Capture, this is on the capture tab. It shows we captured 1,000,000,000 packets and a total of 361GB of data.
Next we verify the data on the capture device by reading it back and running it though our "capinfos2" utility. Its similar to Wiresharks capinfos except it runs fast and can do data verification specific to our system. The exact command line is
sudo stream_cat test1234_20190509_1251 | capinfos -v
From the above output, we see 1Bn packets, 377,788,348,500 B (as it includes the PCAP header), 32.7M packets per second and 94.762Gbps. Also it shows the PCAP time length as 30sec, thus the Spirent 100G IMIX workload ran for 30sec @ max rate.
100Gbps Spirent 64B Packet Benchmark
IMIX is the base benchmark but it doesn't really push devices to their limit. At 100G the network devices limitation is usually the packets per second processing rate. For example many 100G switchs can only forward 256B packets @ 100G line rate, that translates to 48Mpps, which... is pretty low considering 64B @ 100Gbps is 148Mpps. At FMAD our capture devices must never drop packets, period. As our customers depend on us to capture everything on the wire. Everything here means up to 148M packets per second.... and yes we do full 148Mpps!
Below is the Spirent tester running in action
Showing full line rate 64B packet generation. Its showing less than 148Mpps as it reached 1,000,000,000 generated packets and stopped.
And below is the FMAD Capture page, we see 1 billion packets with a total byte size of 64,000,000,000 bytes... and a better named capture this time.
... using our capinfos2 tool to get the stats
Above result shows, 148M packets per second full line rate 64B packets @ 100Gbps... generated by Spirent... captured by FMADIO!
We test using our own internal tools which is great as we can modify them to meet our requirements. However testing against 3rd party verification tools such as Spirent is re-assuring. As it increases the robustness of our 100G packet capture platform and provides validation on our capture rate claims.
Any questions about sustained 100G capture, feel free to drop us an email!
