Blog

The Digital Agency for International Development

Simulating low bandwidths: how to make sure your apps work in the field

By tariq on 23 January 2010

I'm going to write about four ways to simulate a slow internet connection and a bit of background about why you'd want to do it. Simulation is great but I'll say this now: there's no substitute for testing stuff in the field. However, before you release to the team on the ground or grab your bag and hop on a plane, read this.

Why simulate low bandwidths?

Aptivate builds online software for people in the international development sector. Our users are in places where internet connections are slow and unreliable - we need to make sure our stuff works for them.

More people are accessing the web using mobile handsets or mobile internet connections (3G dongles and tethered phones). The bandwidth, latency and stability characteristics of these links are very different to the "always on broadband" that most developers target.

Finally, we're involved with CrisisCamp London over the next few weeks - part of an international effort by dedicated volunteers to provide remote technical support and build tools for individuals and organisations working in Haiti. What they build needs to work in that environment - hopefully this post is going to be useful.

So, if you think your technology is going to be used in the scenarios above, read on. If you're just awesome and like to know about this stuff, read on too...

Four ways to simulate a slow connection

  1. Use a profiler, your brain and some common sense (easiest)
  2. Use Aptivate's online low bandwidth simulator (easy)
  3. Use Sloppy a desktop Java app that simulates slow web links (pretty easy)
  4. Get a machine (with maybe two network interfaces) and do some IP traffic shaping (best results, not easy)
There are probably a few more but I'm on a plane from Rome to London to go the the first CrisisCamp and can't think properly!

1. A profiler + simulate it in your brain.

This is particularly useful if you're developing for the web.

There are a few online and in-browser tools that give you a breakdown of the resources your website is using.

  • Combination of Yahoo's YslowFirebug for Firefox
  • Safari or Google Chrome's built-in (webkit based?) inspectors
  • Pingdom tools online
Three important things to get from these tools is:
  1. What's the bandwidth usage of each of my pages and typical interactions?
  2. How many individual requests are required for each page?
  3. How much content is cached?
The overall bandwidth is a feasibility test. We say aim for 25k per page, but use your own judgement - how fast is your user's connection, how long will it take for them to get to something useful (hint - if it's longer than 5-10 seconds: #FAIL)

The number of requests also gives you an indication about performance over high latency or intermittent connections - in short, use fewer objects and cache them when you can.

Finally, if you've got a network usage meter (I have a noddy one running that comes with iStatMenus on the Mac ) you can get a rough idea of how much bandwidth an app is consuming (should work fine even if you're developing an app in a mobile simulator). I've seen stuff for Windows I can't remember, on Linux you could use BWM or get fancy with logging modes in IPTables - Google for more.

That's it.

2. Use Aptivate's online Low Bandwidth Simulator

This technique is only useful if your site is accessible from a public URL. It only simulates bandwidth, not latency or packet loss.

We make Loband, and online service that strips the junk out of webpages and gives you a compressed, simplified version that works better on slow links.

As part of the Loband code, there's a simulator which you can access here.

You plug in the URL of your site, select the bandwidth you want to simulate and hit go. I haven't tested it recently with any serious AJAX/HTML5/Flex/Flash stuff so your mileage may vary if you make heavy use of these tools.

Do what a user would do with your app and see if it's usable.

That's it.

3. Use the Sloppy desktop Java app

This technique is great if your site is running on a local dev box or even if it's online. It only simulates bandwidth, not latency or packet loss.

Make sure Java lives on your machine. Download Sloppy. Run it, start it, point it at your app.

Do what you would have done with 2.

That's it.

4. Get a machine, (maybe two network cards) do IP traffic shaping.

This technique is the best of the bunch: you can simulate bandwidth, latency and packet loss and do so for anything running on your machine or LAN. That's anything: browser apps, mail clients, Skype, mobile simulators etc. It's not hard but is a little fiddly. There are two broad ways you could do this: for yourself on a single machine or, for a bunch of people on a LAN.

Terminal Showing iperf measuring different bandwidths throttled by dummynet

Quickly,  to do it for yourself, on your own machine to do app testing: if you're running FreeBSD / MacOSX,, follow Bjørn Hansen's tutorial.

It gets a bit trickier if you want to do it for several machines at once.

What we're trying to do is turn a machine with two network interfaces (NICs) into a "router". Traffic goes in/out of the first interface at normal speeds, but the traffic goes in/out of the second interface at user-selected levels of crapness (bandwidth, latency, packet loss)

Relatively speaking: this is easy on a Mac / BSD box, trickier on Linux and hard on Windows. While most laptops actually have 2 network interfaces (wifi + ethernet) - I normally do this with a desktop that's got 2 NICs  or a laptop + a USB / CardBus/PCMCIA NIC.

On a Mac/BSD you're going to be using ipfw to control the dummnynet traffic shaper. Man up to find out more. In short: ipfw's a firewall that classifies packets (e.g. by which port or IP they're going to) into "flows". Dummynet takes a flow and sticks it in a "pipe". A pipe emulates a link with given bandwidth, propagation delay, queue size and packet loss rate.

....how on earth do we get this working?

There are better tutorials than I can write quickly here and  here. But in brief:
  1. Get a BSD machine with dummynet (OSX 10.4+ is enabled by default, might need a kernel rebuild for FreeBSD) running with 2 NICs. Fire up a terminal, type in ifconfig and make sure you can see the two interfaces (en0 and en2 for me)
  2. Make sure you can route packets between interfaces.
  3. Make a pipe for the traffic between interfaces
  4. Configure your pipe, stick your traffic in there and smoke it.
  5. Tweak the pipe and simulate to you heart's content.
In reality, this always takes me half an hour to get right - I've never had this go smoothly first time.

First things I check if it's not working:

  • Is OSX / BSD doing some daft routing / automatic internet connection sharing that messing with your ipfw settings?
  • Are you routing using the right interfaces? I've actually got 7 network interfaces that show up in ifconfig to choose from (firewire, bt, vm, wifi, ethernet etc.)
  • bit/s and Byte/s are quite different…
  • Don't despair, it will work, there's pictures of me doing it here. :-)

Typical bandwidth / latency / loss scenarios

The key commands you'll be running to set parameters will look like:
ipfw pipe 1 config bw 50Kbit ipfw pipe 1 config delay 200ms ipfw pipe 1 config plr 0.2
The three variables you have to play with are bw (bandwidth) plr (random packet loss rate) and delay (latency). Here's a super-rough guestimate for some typical scenarios, please advise if I'm way out or there are other common scenarios:
Scenario Bw (Kbit) delay (ms) pr (ratio)
2.5G mobile (GPRS) 50 200 0.2
3G mobile 1000 200 0.2
VSAT 5000 500 0.2
Busy LAN on VSAT 300 500 0.4

What about windows and linux?!

I promise to update this bit with more info when I've got Linux, Windows boxes and Chris to hand.

In short though: with linux it's the same idea, machine with 2 NICS, get them routing, use IPTables and the linux traffic shaper, tc. It's not as good as dummynet (no packet loss IIRC) but gets the job done. For Windows, I'd honestly have to do some more research, last time I tried it, I just pulled out my Macbook.

Please add any tips and corrections of  below!

Tariq

Jan. 23, 2010, 11:06 p.m. - Tweets that mention Simulating low bandwidths: ...

[...] This post was mentioned on Twitter by Heather LaGarde, Theresa Sondjo, Pablarribas Radar, Pablarribas Radar, Thad K and others. Thad K said: RT @theresac: Very cool howto RT @tkb @aptivateuk Simulating Low Bandwidths - get your apps working in the field http://bit.ly/918W2Z #ICT4D [...]

Jan. 29, 2010, 11:32 p.m. - Peter

One other suggestion: find a 14.4 modem. Seriously. If you're in the west and have an ISP, their rack of 56k dialins will support failing back to Hayes 14.4 if that's all the caller supports. Line noise, drops, etc? Probably not as bad as they will be in the field. But since you've probably forgotten how to turn call waiting off, do your testing when your mom often calls : )

Jan. 30, 2010, 3:33 p.m. - Lopad: high speed collaborative text editing ov...

[...] Simulating low bandwidths, how to make sure your apps work in the field. [...]

March 31, 2010, 11:54 p.m. - Vikram

> What about windows and linux?! >I promise to update this bit with more info when I’ve got Linux, Windows boxes and Chris to hand. I have been pulling my hair from last couple of days for how to do it on Windows using Windows version of dummynet. So far, I have not been successful. Any idea ..

April 1, 2010, 8:53 a.m. - chrisw

Hi Vikram, how far did you get on Windows? Did you create a bridge device? Did you install the NDIS driver? Can you run the testme.bat? Can you run the ipfw command? It's difficult to help you without knowing where you are.

April 15, 2010, 11:25 p.m. - David F

Hi Chris et all, I have been trying this (WindowsXP+Dummynet w/ 2 NICs). Using Windows network bridging to bridge the two NICs - the dummynet service installs to both NICs in the bridge automatically. Everything seems to install fine - then when I run the test.bat file I blue screen. If I undo the bridge and separate the NICs again and put dummynet on just one card, testme.bat works fine - no bluescreen. Any suggestions?

April 22, 2010, 4:34 p.m. - chrisw

David, could you post info about the bluescreen please? Maybe to http://pastebin.ca and just send us the link.

June 8, 2010, 4:10 p.m. - Simulating low bandwidth: in practice « A...

[...] few months ago, Tariq wrote a great article about how to simulate low bandwidth in your office, to see what it’s like for users with slow Internet connections to access your [...]

June 9, 2010, 11:28 a.m. - serge-parfait

hi chris i'm reall interrest on your tutorial about bandwidth management were can i get it ? and i want to know how can i really measure my bandwidth to be sure that i'm paying what i have buy (the bandwidth ).

June 11, 2010, 12:22 p.m. - chrisw

Hi Parfait, hope you're well. You can find the bandwidth management tutorial on the AfNOG Workshop Website. I have videos too, but they need editing before I can publish them. You can follow the instructions in the tutorial, or in the free online Bandwidth Management Book, to measure how much bandwidth you're actually getting, and compare it to what your provider is supposed to be giving you. Cheers, Chris.

June 11, 2010, 2:21 p.m. - serge-parfait

the provider (isp) is sopposed to give us 1Mega down and 512 up shared bandwidth. some time early in the morning i can even open my mail box even i'm alone.

June 18, 2010, 12:42 p.m. - Andymed

Hi Chris, just like David I'm trying to get dummynet running on a Windows machine (WinXP 32 bit) in bridging mode (to avoid necessity of address and/or routing changes to the network the emulation is going take place in. I'm not experiencing any bluescreens etc, but with the network bridge enabled no traffic is passing the system at all. BTW: with ipfw enabled on the separate NICs (with no bridge in place) everything works fine. Are there any commands to be issued in ipwf to get the bridge working? Any suggestions are highly appreciated. Best regards Andreas

June 1, 2011, 12:36 p.m. - hm

about "using WindowsXP+Dummynet w/ 2 NICs with bridge enabled" do not use Win XP bridge feature. This tool can do that instead http://www.ntkernel.com/downloads/ebridge_x86.zip I have tested it and it works great. HM

May 19, 2012, 6:12 p.m. - Pedro Lucas

Hey hm The ebridge_x86 does not BSOD like the XP bridge, true, and Dummynet works fine with it, but I cannot make the ebridge_x86 to work as it should, ie, bridge packets incoming from a LAN connection from another machine to an outgoing second (WLAN) connection, or vice-versa. I added both network adapters as bridged, I have IP connection OK via the WLAN connection on the machine where I installed Dummynet (to induce a delay) but the other machine that is supposed to be connected to this one via LAN cannot get an IP via DHCP, or even static, does not receive any packets. Any idea how did you fix this ? Thank you

May 21, 2012, 2:03 p.m. - Chris Wilson

Hi Pedro, Bridging anything to a WLAN interface is difficult, because a bridge must be able to send packets with any source MAC address. This is not allowed by 802.11, where an association is only good for one MAC address. Handling this requires either the use of WDS mode by both sides, instead of plain 802.11, or special magic to perform layer two (MAC address) NAT on the wireless interface of the bridge device. The latter is done by the Linux kernel. I don't know for sure whether the Windows kernel or ebridge does this, but if bridging LAN-LAN works but LAN-WLAN doesn't work, this is most likely the reason. Cheers, Chris.