Did you know your budget MikroTik Switches can route 10 Gb/s?

Recently I had to merge two sites of my lab to one site and therefore from then on I had two /24 networks on that one site, which I had to route traffic between them, because obviously they where located on two sites before and even now they are two separate Layer 2 networks.

No problem, I said to myself, being equipped with a RB5009UG+S+IN router and a CRS326-24G-2S+RM. Let’s do the routing on the router (duh), since it even has one 10 GbE port, which also serves as uplink to one of the CRS326-24G-2S+RM 10 GbE ports of that switch. My server with both /24 VLANs was connected to the other 10 GbE port of the CRS326.

CRS326-24G-2S+RM

RB5009UG+S+IN

As I had everything wired up and configured, I did a simple file transfer between the two networks which the RB5009 now had to route and I also did an iperf3 test with 1-8 parallel streams. Results were rather underwhelming. Maximum throughput I was able to achieve was around 4-5 Gb/s and the poor CPU of my RB5009 hit the wall at nearly 100% load. So I tried to find and enable L3 hardware offloading and had to learn, that the RB5009 currently (and likely never will be) capable of that.

“What kind of router is that?” I asked myself and went on the MikroTik forums. There I was told that the purpose of the RB5009 is more LAN/WAN routing, NATting and firewalling, than intra-VLAN routing. and that I’d be better off with a proper Layer 3 switch for that purpose. “I’ve got something like that”, I thought to myself and immediately made plans to move the routing of the two /24 subnets from the RB5009 to my CRS326. When I was done with that, I ran some iperf3 tests again and the results were even more underwhelming. I only got around 1-2 Gb/s and the CPU of the CRS326 switch was almost at 100%, just like the RB5009 CPU was.

But then, L3 hardware offloading came for the rescue. I enabled that feature, restarted the switch, just to be sure and clear all the sessions, caches and tables, and started new iperf3 tests. Finally, I was overwhelmed, what a 200$ switch can do, and how incredible it is, that MikroTik added so much value after many years after the hardware release by implementing L3 hardware offloading to this budget switch with RouterOS 7.1, but see for yourself:

one connection

four connections

eight connections

And that wasn’t even with any optimizations like jumbo frames.

Some limitations and caveats apply, see https://help.mikrotik.com/docs/display/ROS/L3+Hardware+Offloading for details and pay attention to your specific switch and the used switch chip series.

It seems, that the CRS326-24G-2S+RM utilizes the same switch chip as the favourite lab 10 GbE switch of all times, the CRS305-1G-4S+IN, whis is the 98DX3236. Most likely it will be capable of the same performance and possibilities. EDIT: Yes, confirmed. I was able to achieve the same kind of performance with the CRS305-1G-4S+IN.

Being that my newest acquisition is a CRS309-1G-8S+IN, which utilizes a more powerful CPU, the dual core 98DX8208, I’ll probably move routing of the two subnets there and do another set of iperf3 tests. But that’s for another day.