How to install sqm in openwrt?

Note: As of early 2017, cake is an improvement both in speed and capabilities over the original fq_codel algorithm first deployed in May 2012. cake is in every way better than fq_codel. The following description is preserved for historical information.

The right queue setup script (simple, hfsc_lite, ...) for you depends on the combination of several factors like the ISP connection's speed and latency, router's CPU power, wifi/wired connection from LAN clients etc.. You will need likely to experiment with several scripts to see which performs best for you. Below is a summary of real-life testing with three different setup scripts.

This was tested with WNDR3700 running trunk with kernel 4.1.16 and SQM 1.0.7 with simple, hfsc_lite and hfsc_litest scripts with SQM speed setting 85000/10000 (intentionally lower than ISP connection speed), 110000/15000 (that should exceed the ISP connection speed and also totally burden the router's CPU), as well as 110000/15000 using Wifi.

(“flent” network measurement tool reports the overview as average of the 4 different traffic classes, so the total bandwidth was 4x the figures shown in the above table that shows “per-class” speed. The maximum observed combined download+upload speed was ~110 Mbit/s.)

With wired 85/10 the experience was almost identical with all four qdisc strategies in SQM. Approx. 20 Mbit/s download / 2.1 Mbit/s upload and 19 ms latency shown in the flent summary graph.

With wired 110/15 there was more difference. Interestingly “simple” kept latency at 20 ms, while with the other 3 strategies latency jumped to 50 ms after ~20 seconds. (Might be a flent peculiarity, but still mentioning it.) “simple” kept low latency at 19 ms and 21 Mbit/s download, while the other 3 strategies had 50 ms latency while having 24-25 Mbit/s download per class.

But when the LAN client connected with Wifi to the router with 110/15 limits, “simple” lost its download speed. Latency was still low, but download speed was really low, just half of the normal. Likely the CPU power required for wifi limited the CPU available for other processing and the router choked.

The Smart Queue Management (SQM) system on OpenWrt makes it easy to configure a rate limiter on your router. The goal is to set the software shaper to a bandwidth that is slightly lower than the actual (bloated) bottleneck in the hardware, so we can control the queue using FQ-CoDel or CAKE. In many cases, there is no real ground truth about the right setting, but we can find one by trial and error (tuning and doing repeated measurements until things improve).

This is a report of Dave Täht’s experience tuning the Smart Queue Management (SQM) system on CeroWrt (predecessor to OpenWrt) for a cable modem at Jim Reisert’s home. The SQM system (which works on any Linux-derived system) uses HTB + fq_codel underneath to give low latency to competing streams, and the codel AQM system to keep overall queue lengths short.

Note: You can use the DSLReports or Waveform speed tests as a good alternative to the RRUL test.

It took 4 tries (and 5 minutes) to get a setting that worked well! When we were done, we watched a videoconference and ran screen sharing session over skype while saturating the network with a RRUL test for 5 minutes.

Download and upload speeds remained high, latency remained low, and there was no observable effect on the video conference. It was perfect.

How to read the RRUL plots below

The RRUL test normally runs for 70 seconds: 5 seconds of idle (to give a baseline), 60 seconds of full-rate data transfer, and 5 more seconds of idle. There are three graphs:

The (four) colored lines indicate separate, simultaneous TCP sessions. The black line represents the average of those four upload/downloads; multiply that value by four to get the actual data rates.

For example, the top chart shows (1.2 mbps * 4 = about 4.8mbps); middle chart shows (7mbps * 4 = about 28 mbps), and latency starting low, but ramping to over 1000 msec (over one second) during the test.

Dave picks up the narrative…

“After installing the latest OpenWrt and the luci-app-sqm package, we left SQM turned off. I ran the RRUL test (or DSL Reports or Waveform) remotely. This was how his cable connection behaved without any latency control. (See charts above.) We see the usual 1-2 seconds worth of induced latency common to (and bedeviling!) current cable deployments. (Note: the up and download figures for all these charts are reversed since I was running RRUL from a remote server, not from within Jim’s network, as is normally done.)

“While awesome to be able to run this test over native IPv6, 1.2 seconds of latency left something to be desired. (The latency problem has nothing to do with IPv6, or IPv4, but bufferbloat in the modem and CMTS).

“The early spike here of extra bandwidth is due to speedboost kicking in for 10 seconds and providing some extra bandwidth, but even as it begins to kick in latencies are already skyrocketing.

“So taking a guess at the bandwidth from the averages (the black line * 4) on the up/down graphs, we tried setting setting the Smart Queue Management system (SQM) to 38mbits down and 8 up. (Well, actually I goofed when I looked at the graphs: 7*4 = 28, not 38). Note also that the black lines do not correctly add in the bandwidth used up by the tcp acks in the opposite direction. On some systems you need to factor in ~1/40th the bandwidth used in the opposite direction for a more correct estimate.

“A little better, but it still looks as if the data was taking a side jaunt to the moon!

“Taking another guess, we tried, 24mbps down and 6mbps up.

“Much better! But given the increase in latency and the average where it was, it was apparent that 6 mbit up was still too much, so we knocked that down to 4400 (kbps, or 4.4 mbps), and got this:

“A total increase of observable latency over the baseline of 65ms of 10 milliseconds (vs 1.2 seconds! A 110x improvement… ) and good sharing between streams and good throughput. And thus, we declared victory, and then talked for an hour doing various other tests while the videoconference continued to rock.

Final Notes:

These tests were made against a Comcast connection with IPv6 enabled, taken with a Motorola SB6141 cablemodem running firmware SB_KOMODO-1.0.6.10-SCM00-NOSH. OpenWrt’s Qos-scripts use similar techniques to OpenWrt’s SQM system, but are not IPv6 compatible, neither are most versions of wondershaper. It is unknown to what extent other smart queue management systems (gentoo, ipfire, streamboost, gargoyle) handle IPv6 at present. (And OpenWrt gets the same good results with any combination of IPv4 and IPv6.)

Update: May 2014

I reran the plots to clean up the plotting bug that we’d had in an earlier version. Since this test series was first run the netperf-wrapper tool (now called “Flent”) has gained the ability to compare multiple test runs. Previously, we were well aware that disabling powerboost (as we currently do) gives consistent latency, but leaves some bandwidth on the floor. How much was kind of an unknown.

Now we know. The speedboost algorithm is fairly well documented, and we do think that with some tweaks and fixes to the htb rate limiter to allow for more burstyness we can keep latencies reliably low and get closer to the full bandwidth available from a cable modem, all the time. (But we have no funding, and we’re focused on fixing wireless next.)

Losing that initial bit of bandwidth, in light of always getting good latency, seems like the bigger win, for the time being.

Update: December 2021

The generic Linux scripts can be downloaded from the link above.

On OpenWrt, SQM can be installed from the LuCi interface or by the following CLI commands on your router:

opkg update

opkg install sqm-scripts

Note: The standard and default SQM installation expects monitoring of the interface connecting to the WAN. What we need is for SQM to monitor the interface NDS is bound to. This of course will be a LAN interface. The default configuration will limit bandwidth from the WAN connection to services on the Internet. Our configuration will limit client bandwidth TO NDS, thus enabling a true fair usage policy.

To prevent confusion it is important to understand that SQM defines “Upload” as traffic “Out” of the interface SQM is monitoring and “Download” as traffic “In” to the SQM interface.

In the default SQM configuration, Upload will mean what is normally accepted, ie traffic to the Internet and Download will mean traffic from the Internet.

In our case however the terms will be reversed!

The default SQM configuration file on OpenWrt is:

For simple rate limiting, we are interested in setting the desired interface and the download/upload rates.

We may also want to optimize for the type of Internet feed and change the qdisc.

A typical Internet feed could range from a high speed fiber optic connection through fast VDSL to a fairly poor ADSL connection and configured rates should be carefully chosen when setting up your Captive Portal.

A typical Captive Portal however will be providing free Internet access to customers and guests at a business or venue, using their mobile devices.

A good compromise for a business or venue might be a download rate from the Internet of ~3000 Kb/s and an upload rate to the Internet of ~1000 Kb/s will be adequate, allowing for example, a client to stream a YouTube video, yet have minimal effect on other clients browsing the Internet or downloading their emails. Obviously the values for upload and download rates for best overall performance depend on many factors and are best determined by trial and error.

If we assume we have NDS bound to interface br-lan and we have a VDSL connection, a good working setup for SQM will be as follows:

We will configure this by issuing the following commands:

Note the reversed “upload” and “download” values.

Replace the linklayer and overhead values to match your Internet feed.

The following table lists LinkLayer types and Overhead for common feed types:

Some broadband providers use variations on the values shown here, contacting them for details sometimes helps but often the request will be “off script” for a typical helpdesk. These table values should give good results regardless. Trial and error and the use of a good speed tester is often the only way forward. A good speed tester web site is http://dslreports.com/speedtest

Further details about SQM can be found at the following links:

https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm

“Smart Queue Management”, or “SQM” is shorthand for an integrated network system that performs better per-packet/per flow network scheduling, active queue length management (AQM), traffic shaping/rate limiting, and QoS (prioritization).

“Classic” QoS does prioritization only.

“Classic” AQM manages queue lengths only.

“Classic” packet scheduling does some form of fair queuing only.

“Classic” traffic shaping and policing sets hard limits on queue lengths and transfer rates

“Classic” rate limiting sets hard limits on network speeds.

It has become apparent that in order to ensure a good internet experience all of these techniques need to be combined and used as an integrated whole, and also represented as such to end-users. After years of debate on the bloat and AQM mailing lists, no name for the idea has been agreed on. A lot of people object to “smart queue management” as being too general a phrase (“you can do anything and call it smart”), but we hope that by defining it as we have above to limit the mis-appropriations. A trademarked name has been suggested as well…

Probably the first widely deployed fully integrated “smart queue management” system was the venerated wondershaper, which emerged in the early 2000s as the linux based shaper of choice. It was widely deployed in internet cafes around the world, and in Linux users’ homes and workplaces. Although for the time it was a breakthrough, it has since been obsoleted by events and bugs in its design. See Wondershaper Must Die.

Much work on AQM (active queue length management) technologies like RED and BLUE took place in the period 1990-2002. Many variants of RED appeared between 2002 and 2013 - FRED, ARED, LRED, etc, but work mostly stagnated under variants of the same set of ideas, until the creation of the bufferbloat effort in 2011, which begat Codel and PIE. Feedback from the effects of these algorithms have led to improvements in various TCP implementations as well.

Packet scheduling has a longer and more successful history, starting with the first research into “fair queueing” in the mid-1980s and continuing to the present day with ideas like DRR, SFQ, QFQ and SQF. Along the way it was realized that strictly “fair” queueing was not desirable, which led to optimizations like WFQ (weighted Fair Queueing), SQF (shortest queue first), and sparse stream optimizations like those in fq_codel. Lacking a better phrase we try to distinguish between old-style “fair” queuing and new-style “flow aware queueing”, but the common understanding of the abbreviation “FQ” = Fair Queueing is the source of much confusion.

These techniques ( shaping, prioritization, packet scheduling, and AQM) are often used in serial, rather than parallel.

It is possible to make things worse by applying only a few of these techniques, a classic example of this is in NetGear’s current QoS system which allows you to rate limit but holds the fifo queue lengths constant, and does not apply either packet scheduling or AQM, leading to exorbitant delays. SFQ, used alone at higher bandwidths gets the bursty tail drop problem an AQM can solve. An AQM, used alone, has trouble managing bursts. QoS, used alone, only works on what packet types can be classified easily. And, policing, set incorrectly, can seriously damage downloads.

WRED was probably the most successful of the packet scheduling/QoS/AQM hybrids. fq_codel (the principal candidate for a successor to WRED) combines smarter packet scheduling with a innovative AQM design into a single algorithm also, but does not have native support for QoS packet markings. It is usually combined with something else to do that.

Most QoS systems as shipped today are terrifically elaborate and let you prioritize certain packet types to your heart’s content, but this is of no help in a world that consists mostly of bursty web traffic on ports 80 and 443.

Rate limiting is presently stuck with token bucket designs descended from the original CBQ, like HTB. Software rate limiting is far more abusive of CPU than any of the packet scheduling or AQM algorithms discussed above, or their hybrids.

Traffic shaping (“policing”) was a not very good idea hard limiting ingress speeds that become common because of the ease of implementation. It is far saner to apply Smart Queue Management to “police” traffic at today’s higher speeds.

Examples of deployed Smart Queue Management systems include CeroWrt’s SQM implementation, OpenWrt’s qos-scripts, IPFire, the Gargoyle router project, and Streamboost . WRED is deployed in many locations. France Telecom deploys SFQ. Free.fr has the first known large-scale fq_codel deployment, using three bands of fq_codel for tiers of QoS. (it is also the largest ECN enabled end-user deployment). The Streamboost product (now coming available in multiple 802.11ac routers) combines a bandwidth sensor, with a packet classification engine, with a multi-band fq_codel implementation.

We are beginning to characterise these in an upcoming internet draft on comprehensive queue management in home routers

• Check the Enable box.
• Set the Interface name to your wide area network ( WAN ) link. Interfaces are listed in the dropdown, or check Network → Interfaces to find the name for the WAN port.
• Set the Download and Upload speeds to 85-95% of the speed you measured in Preparation.