The Packet Sniffer Blog

Our FMADIO10 and FMADIO20 products are unique by operating in a hybrid SSD / HDD mode. However the lower HDD sustained throughput reduces the sustained capture rate when used in continuous high bandwidth 24H / 7 capture mode. We dig deeper and understand whats happening under the hood for long run capture and what the system limitations are.

Hybrid SSD/HDD architecture

We use a hybrid SSD/HDD caching storage architecture for our FMADIO10 and FMADIO20 10G line rate packet capture devices. This enables a cost effective way to get full 10G line rate capture with massive storage but there is a compromise, and that is reduced sustained 24/7 capture performance.



We beleive very few systems require sustained 10Gbps running 24/7 and if that is your requirement, there are plenty of other vendors who sell these kinds of crazy expensive very large and heavy capture devices. We focus on applications where the total capture size within a 24 hour period is less than 16TB. The calculation for that is 16TB / 24Hours =~ 1.5Gbps. That being said, we do capture sustained full 10Gbit line rate for bursts of up to 1TB at a time (yes capitial T for Terabyte).

Remember that 16TB / 10Gbps == 12800sec, 12800sec == 213min and 213 min == 3.5 hours. Meaning if your really doing 10Gbps line rate, all 16TB of the storage is consumed in just under 3.5 hours... that's a hell of alot of data!

RAID0 Sustained Performance

For applications that demand high throughput 24/7 sustained capture we recommend configuring the HDD in a RAID0 mode. We have rated the system`s 24/7 sustained capture rate at just under 4 Gbps in a RAID0 HDD configuration. As there is additional overhead for each packet that includes timestamp, port and other information, it increases the total required HDD write bandwidth. Thus the capture rate is slightly lower than the real disk throughput.

However there is an interesting behavior, as seen in the screenshot below. You`ll notice the cache usage rockets way up to over 400GB at 7am in the morning, during a 24H sustained long run test. Its a rather peculiar observation, but thanks to our outstanding user interface it is simple to observe and monitor.

Is your HDD slowing down?

Its a some what dirty little secret HDD vendors all know about but its rarely discussed, especially in a market thats dominated by ultra fast SSD`s. The secret is HDD`s throughput degrades the closer the drive head moves towards the center of the disk. For example, data on the outer edge of the platter has a higher read/write bandwidth than a location near the center.

Its a problem because if your building a home grown packet capture system and benchmark a single HDD at 1.2Gbps throughput and extrapolate to target 10Gbps capture to disk. e.g. 10Gbps / 1.2Gbps = 8.3 thus a 9 disk RAID0 array will capture your full 10Gbps line rate. Its actually a completely incorrect calculation, as you need to use the minimum throughput observed across the entire disk - not the average.



The above screenshot shows our RAID0 write bandwidth over the entire span of the 16TB array. From the graph its very clear the outer edge of the disk is around 6Gbps write throughput, while near the end of the disk it drops by almost half to just above 3Gbps. Thus if you budgeted for a 9 disk array expecting full 10Gbps capture to disk, you`re real write bandwidth at the end of the disk is half that and likely around the 5Gbps mark!

Summary

Cache is a beautiful thing. The HDD performance variability is fixed by our 1TB of high speed cache! The system clearly caches any HDD write bandwidth delta e.g. the last 2TB of the HDD array. After the disk head moves to the outer rings the disk, write speed increases and cache utilization drops to 0. Just one more advantage of our hybrid SSD/HDD capture systems.

The sweet spot for our 10G packet sniffer is short sustained 10Gbps line rate bursts lasting up to 1TB in length, with sustained long run 24/7 capture rates at the 2.5-4.0 Gbps mark depending on RAID5/RAID0 setup.

If you require further clarification, or have any questions please contact us at any time!