Showing posts with label Bandwidth. Show all posts
Showing posts with label Bandwidth. Show all posts

Friday, 10 February 2012

How to Make Almost Any Wired Device Wireless

Running my Xbox 360 wirelessly without using the official Microsoft wireless adapter is something I've been doing for a number of years now, and it's not that difficult.  I wanted to write this guide for a long time as I believe it's such a better alternative to the grossly overpriced Xbox 360 wireless USB adapter (seriously, it's around 4x the price of the PC equivalent), and more to the point has the potential to be much faster.  Fortunately, I decided to upgrade my wireless bridge which means I'll be revisiting the setup.

Basically it involves taking a standard internet router and putting custom firmware on it that gives it additional features such as wireless bridge mode.  This is what will let your secondary router connect to your internet wireless router, wirelessly which effectively turns it into a wireless adapter or a client just like a laptop.  The clever part is that since your devices connect to it with ethernet cable, they act just like a wired device, so once the initial setup is completed, adding "wireless" devices to your network is as easy as plugging the cable in.  No messing with router passwords or anything ever again.

Top: Xbox 360 with original adapter connected to a wireless router limited to around 20mbps.
Bottom: Xbox 360 connected to a wireless router acting as a bridge, which is connected wirelessly to another router transferring at least 105mbps.




There are numerous advantages to this, which is what stood out to me more than the fact I was getting one over on Microsoft, or saving a good amount of money.  For instance, depending on the router model you go for, it can be cheaper than the Xbox 360 wireless adapter.  My old router was £35 compared to £60 for the USB adapter and since it allowed me to connect multiple devices it meant I didn't have to buy an adapter for each device.


The other big advantage with this method is that you are not bound to the speeds of internal or official wireless adapters.  My PS3 has an internal 54G wireless card which has a real world performance of about 20mbps max which is the same as the original Xbox 360 wireless USB adapter.  Even with the new Xbox 360 built in wireless or the Wireless N adapter, those are dual stream which means they top out at around 60mbps at any realistic range, but since the routers I am using are triple stream I am able to get around 120mbps from them at opposite sides of the house.  Remember that even if you decide to replace your existing router with an E4200 triple stream router, the client must support it which means having 3 antennas, and the Xbox 360 only has two.  It will still work, but it will be limited to around 60-80mbps at best, whereas a triple antenna device can transfer as fast as 230mbps at very close range

Wifi speed ratings are very misleading.  One of the most common standards we will have come across at some point is Wireless G which is rated as 54mbps, but since wireless communications are effectively half duplex in the way they work, the bandwidth available is at best half of the rated link speed or less.  In other words when they rate a device as 54mbps, that's 54mbps shared in both directions, operating in the best theoretical conditions.

Wireless G is fine for a lot of people, but is not so good for streaming high definition movies that can have bitrates peak in excess of 20mbps, or people with fast connections.  With the wireless bridge method, the wireless adapter is in effect external now, it means I can upgrade the routers as and when faster products and new specifications come out, so you are never stuck to a certain wifi speed or spec.  In fact right now, it's the Xbox 360's ethernet port that is the bottleneck in my setup.  If it had gigabit like the PS3, I would be able to download up to 120mbps in theory once Virgin carry out the upgrade in the summer.


Take a look at the above picture. In this typical situation, all devices are competing for wireless bandwidth. If someone is streaming movies from the PC to the PS3, it will take away from the total wireless bandwidth that could be otherwise used for the Xbox 360 causing slower downloads or the real possibility of lag.  The PS3 wireless is limited to around 20mbps, meaning that high bitrate movies could stutter and lag and the Xbox 360's original wireless adapter is also limited to around 20mbps (since it is also Wireless G), which means if you have a connection over 20mbps, you simply won't make the most of it.  What's worst is that if your main router is only Wireless G, then it will only have about 20mbps to share out between all the devices, so if when your brother or sister is streaming films on the PS3 from the laptop and it hits a part of the video that comes close to or exceeds 20mbps, the router will run out of bandwidth meaning there is none left for the Xbox and then your download will stall or game will lag.  You should at the very least ensure you have a high quality wireless router, even if you don't go to the trouble of setting up a second router as a wireless bridge.


Now let's revisit that situation but put a wireless bridge in there.  Straight away you eliminate a lot of potential for wireless bottlenecking, in particular with things like streaming.  Now instead of the laptop having to send the movie data through it's wireless to the router, and from the router through the PS3's wireless, the transfer now just takes place over ethernet, which in this situation is way better since the PS3 has gigabit lan, so do most laptops and the routers used in this example (Linksys E4200) have gigabit ports too.  Not only are transfers faster to the PS3, but lag in movies is eliminated and it has no impact on local users as the wireless link is now only used for internet data, and not communication between devices.  Communication between devices is now done on a switch/ethernet level, unless of course you have another laptop connected directly via wireless to the main router/modem.  Since the PS3 and PC are communicating through the switch, the Xbox 360 is free to request data from the internet using the wireless link as it likes.

One of the other cool things about this setup is that I can share my ethernet laser printer on the network, so if you imagine the laser printer is connected at the same point where the Xbox and PS3 are, and that the laptops in my house connect to the first router on the left, it means I can print wirelessly.

Port forwarding remains unchanged too, and rather than having to set port forwards on both devices, you just set them on the main router.  There is no need to set port forwards on your wireless bridge.

The only real downsides to this method are that it can be expensive if you want a good router (£130 for a Linksys E4200 for example) and that the choices of routers are limited by what are supported by custom firmwares, and at the end of the day, you still require some amount of cable (although if you are like me, all your entertainment stuff tends to be in a corner of the room).  One other limitation that was present the first time I tried this was that when your router is set up as a wireless bridge, other wireless devices cannot connect to it directly as the link is used solely for communications between the wireless routers.  Wireless clients will have to connect to the original wireless router/modem as usual. That said, I'm sure you will agree that the advantages far outweigh the disadvantages.


Flashing the Custom Firmware
For this particular project I will be using DD-WRT again.  Although stability is as much a hardware thing as it is software, my old faithful WRT-54GL has had in excess of 100 days uptime with no signs of getting bogged down with an 8% overclock.  There are two ways to attack this, either check the website to see if an existing/spare router you have is supported (in general, hardware with built in ADSL modems are not supported), or if you are looking to buy some hardware specifically to use as a wireless bridge, check the router database.  I will be using a Linksys E4200 V1 with DD-WRT.  It is currently a work in progress as the hardware isn't really that old in comparison with some of the supported routers.  Normally I would say to stick with tried and tested stable builds and hardware, but my hand has been forced a little into this because the WAN/Internet port on my E4200 was acting funny, so I bought a new one and decided to retire the old one to replace my decrepit WRT54GL.  Although the E4200 is a very good piece of hardware there are certain reasons why it is unlikely that DD-WRT will ever be able to extract the full speed out of it, but people on the forums are saying it should be good for 100mbps over wireless, which is good enough for my use.

1) 30-30-30 Reset
The very first thing you must do is a 30-30-30 reset.  This should be done before and after the firmware upgrade as this clears contents from NVRAM.  If you don't do this, remnants of information may be left in the NVRAM and cause conflicts with the new firmware.

A 30-30-30 reset is as follows.  First with the router switched on and everything else unplugged, hold the reset button in for 30 seconds.  While still holding the button in, pull the power cable out and wait another 30 seconds.  Finally while still keeping the reset button pressed in, connect the power and wait for another 30 seconds.  In all, you should have held the reset button in for 90 seconds continuously.

2) Inital Flash
The next step is an initial flash, which flashes a base version of DD-WRT onto your router, which you may then upgrade later to a version with more features.  The process for flashing can be a little different for each router and may also require different files, so it's strongly advised that you visit the DD-WRT Wiki and look for instructions relating to your specific model.  I'll be using this page for my E4200.  Download the initial flash file, then disconnect all your network connections, turn off wireless and connect the router to be flashed by ethernet cable.  The file I am using for the initial flash is:
ftp://ftp.dd-wrt.com/others/eko/V24-K26/svn16773/dd-wrt.v24-16773_NEWD-2_K2.6_mini-e4200.bin

Load up the web interface for the router you are flashing.  The exact address for these vary depending on model and manufacturer, but in general Linksys stuff can be accessed at http://192.168.1.1  Once you have logged in, find the administration tab and click to browse for and upload the firmware.


Click start and leave the web page open.  Flashing only took about 30 seconds for my router, but the DD-WRT guys say to wait for at least 5 minutes just to make sure.  When that has passed, click Continue and you should be greeted with a screen similar to this:



When prompted for a login, the default username is root and the password is admin.  These can both be changed later.

3) 30-30-30 Reset
As a matter of course, disconnect all network cables and clients then perform another 30-30-30 reset as outlined in step 1.  This will ensure that the NVRAM is cleared of any settings left by the stock Linksys firmware.

4) Upgrading to a Bigger/Newer Build
This step is optional since the initial build you just flashed should be pretty much fully functional, but you will probably want to flash one of the later builds for increased performance, stability and features.  Routers have a limited amount of flash memory, so this will dictate which build you can use.  Check the wiki to see how much flash and RAM your device has.  Recommended builds are sometimes listed in the Wiki for your device, but may not be up to date, so check this thread for recommended builds.  Pay special attention to details in the Wiki about any special requirements your device may have, such as the E4200 must have a build that uses no more than 60KB NVRAM, as denoted by -nv60k in the filename.  Do not use micro builds as they are reported to brick Wireless N devices.

The version I am upgrading to is:
ftp://dd-wrt.com/others/eko/BrainSlayer-V24-preSP2/2011/12-12-11-r18000/broadcom_K26/dd-wrt.v24-18000_NEWD-2_K2.6_big-nv60k.bin

Now that you have your device with the initial file flashed and have performed a 30-30-30 reset, you can proceed to upgrade the firmware.  Enter your router's address in your web browser to reach the interface.  If prompted for a username, it is root and password is admin, then find your way to the administration tab and then firmware upgrade from within that.  Again, making sure all other network devices are disconnected and wifi is shut off, select the firmware to be flashed and press upgrade.  Keep the browser window open all the time while the upgrade is in effect and wait for about 5 minutes.

5) 30-30-30 Reset
After 5 minutes has passed and your router has come back online, perform one last 30-30-30 reset.  You are now up and running with DD-WRT and can move on to configuration.


Configuring DD-WRT as a Wireless Bridge
Setting up DD-WRT to connect to your other wireless routers is very easy but we need to change some settings while the router is connected to only your computer as if you try to add it to the network with it's default settings it will likely cause an IP address conflict.

1) Ensuring your wireless is turned off and the only device connected to your computer is the one you just flashed, enter http://192.168.1.1 in your browser to reach the DD-WRT setup page.  At this point it will probably prompt you to enter a new username and password from the last 30-30-30 reset.  Do so and then click the setup tab.

2) Set the WAN connection type to disabled.  Next under the Network Setup header, change the local IP address to an unused one.  Since most Linksys routers default at 192.168.1.1, if we leave the DD-WRT flashed router with the same address it would cause a conflict.  You only need to change the number in the last segment of the address.  I have mine set at 192.168.1.3.  The gateway address should be that of your router that provides the internet connection.  Net mask should be left at 255.255.255.0 unless you have specific needs.  Local DNS is fine if it is left empty.  Check the option, Assign WAN port to switch.  Disable the DHCP server.  Click save settings and then apply.

Click for larger image.
Left page shows defaults and right page shows example settings for wireless bridging (your default gateway and IP settings may vary)


3) Once those settings have been saved and applied, you will have to enter the new address of the router into your browser to continue working in the GUI.  Once you are back into the setup, click the Wireless tab.  Depending on the hardware you have, you may see one or two sections.  Since I am using the 5GHz link, I disabled the 2.4GHz radio.  To set up the 5GHz radio to connect to my other router, set the wireless mode to client bridge, select your network mode from the drop down (5GHz was disabled by default) and under wireless network name (SSID), enter the name of the router you wish to connect to exactly.  Next go to the wireless security tab and enter the wireless key that you use to connect to the main router.  For Wireless N connections the encryption should be set as AES as TKIP will revert to Wireless G speeds.

Click for larger image.
Left page shows defaults and right page shows example settings for wireless settings (page may differ depending on DD-WRT version and how mant radios your device has)


4) Save and apply all settings.  It may also help if you reboot or power cycle your router.


Conclusion
It's pretty much as simple as that.  Of course I have a little experience with networking now so I am familar with most of the terms mentioned in the firmware, but as long as you know the IP address of your modem/router (which is referred to as the gateway) and you know of an open IP address to assign your router, you are good to go.

There are alternatives to this method.  You can actually buy wireless ethernet bridges from the likes of Linksys and Netgear, but in my past experience I found that they were often much lower spec than the routers and the performance just isn't that good.  All they do is flash existing hardware with their own wireless bridge firmware on it and that's pretty much all that device can do.  Flashing your own router with DD-WRT is a much better choice simply because of the amount of options and flexibility DD-WRT gives you.  For example if you ever decide to change how your network is set up or go wired instead of a wireless bridge, you can change some settings in the router and have it do something else ie working as another router or a switch, unlike off the shelf wireless bridges which tend to have such restrictive firmwares that you can't do anything else with them.

There are of course some alternatives too.  Possibly the easiest method of networking between floors without wires is with power line adapters, but I can't comment on how secure they are, much less the performance of them.  Last thing I heard was that the performance was pretty awful.  Basically they use your house wiring and transfer data over the mains.

Of course nothing beats the speed or security of Cat 6 cable and a couple of gigabit switches, but if like me you only need a means of getting internet data from one place to another this works quite well, and the local transfers are very fast over ethernet too.

Thursday, 7 April 2011

uTorrent Tuning

For most people, the default settings in uTorrent will work fine.  You will generally max out your connection before you hit the maximum number of connected peers for example.  However I have recently upgraded to 50Mb/s (6MB/s) fibre optic broadband, and found that I wasn't getting the most from my connection.

One thing I found is that I was connecting to the maximum default number of peers and still not maxing out my connection.  Consider this:  If you can download at 6000KB/s, but your software is set to connect to a maximum number of 100 peers, what happens if the fastest they can upload is 30KB/s each?  That's right, you connect to 100x30KB/s peers, and you only achieve 3000KB/s download (half your maximum throughput).  It's even more of a problem on connections such as 100Mb/s where the downstream is approximately 12MB/s.

One real life scenario was when I was downloading an episode of Nichijou.  I had already set my max connected peers to 150 from previous.  I left it to download in the background while I chatted in IRC for a little bit and then thought to myself, "Why hasn't it finished yet?".  It hadn't finished because I had connected to something like 135 seeds and 15 peers and was only managing to pull 800KB/s (new torrent).  I upped the limit to 200 just to see if it would help and although it didn't connect to 200 (maybe 180 in total), I got 3.8MB/s.  It's definitely worth adjusting your maximum connected peers if you have a good connection but rarely get your maximum seeds on well seeded torrents.

Then I ran into another problem.  When you start downloading a torrent, the default action is to write that file to disk and fill it with zeros.  Kind of a placeholder for the file if you like.  For MP3 albums and small downloads, it's not a big deal because your HDD will have written the file before you've had time to connect to all your peers and reach a high download speed.  The problem arose when I was downloading an OpenSUSE DVD ISO.  uTorrent wanted to write 4.3GB as soon as I had started the download.  It took a little while to write this, during which time I had connected to all my peers and hit 6MB/s download.  Because the HDD was busy, I got a disk overloaded 100% warning in uTorrent and my download speed dropped drastically (a few hundred KB/s) since the HDD couldn't keep up with writing the file at maximum speed, plus the stress of an extra 6MB/s.

Writing the file and zeroing it out when the download starts is annoying.  Not only will it cause your download speed to drop, but it also has the potential to cripple your system for a good few minutes.  What happens if you download a 50GB Bluray ISO?  It would take a good 15 minutes at least to write that at full speed, during which time I doubt you could load much else.  Fortunately I found two methods within the uTorrent settings that help alleviate the problem.

Another annoyance that can hinder you from reaching your full speed is traffic management.  An automated system that checks the packet headers to see what type of data it is.  If it detects torrent traffic, it throttles the speed.  You can work around this by using encryption in uTorrent.

So now let's move on to the actual tuning.  I use uTorrent 1.6, but these options are pretty much universal.  Click Options > Preferences to get started.

Downloads
Click the downloads tab, and ensure "Pre-allocate all files" is unchecked.

Connection
Bandwith Limiting
Did you know that uploading at your maximum speed can cripple your download speed?  The latest versions of uTorrent set your maximum upload speed to 80% of what you can actually achieve to prevent that happening.  If you don't know what your upload speed is, go to Speedtest, pick a local server and make a note.  Multiply your rating in Mb/s by 128 to get your upload rate in KB/s.  Then multiply that by 0.8.
Eg.  4.7Mb/s * 128 = 602KB/s * 0.8 = 481KB/s
Your result is what you should set your Global maximum upload rate to.  Check the box for Alternate upload rate when not downloading and set the KB/s to 0 (unlimited).  With these settings your downloads will not be crippled, and when they are done uTorrent will seed at maximum speed.

BitTorrent
Number of Connections
This will control how many peers you can connect to.  Take a look at the image below
Here you can see that under the Seeds heading, it says 119 (845).  What that means is that I am connected to 119 seeds out of a potential 845.  The same applies for the Peers heading (connected to 6 peers out of 126).  You will see that the setting Maximum number of connected peers per torrent controls how many seeds and peers your are able to connect to.  In this case my limit is set to 125, and uTorrent has decided to connect me to 119 seeds and 6 peers.  Fortunately for this torrent I had reached my maximum download speed, but there are occasions where you will connect to your maximum number of peers and still not reach your maximum speed.  In this case, increasing the maximum number of connected peers should help, providing the number of seeders in the brackets is larger than the number you are already connected to.

Be aware that setting this number too high could have adverse effects, so it's a case of trial and error.

Protocol Encryption
Setting encryption to Enabled and Allow incoming legacy connections will encrypt some of your traffic.  This is particularly useful if you suspect your ISP is throttling your torrent traffic as it scambles the data and makes it unreadable by bandwith management systems.  Again this is another setting that you should experiement with.  You could set encryption to Forced and uncheck Allow incoming legacy connections for full encryption, but it will also limit the amount of peers you can connect to.  It's probably better to have a little encryrption and a little throttling than full encryption and less connected peers.

Advanced
net.max_halfopen
This option specifies how many connections uTorrent should try to establish at any one time.  The default is 8 which is very low.  I have mine set to 100.  The theory behind this is that it will take less time for you to connect to all your peers and reach your highest speed.  Before tweaking this, I found that for files in the region of 200MB, that it would have completed before I had connected to all the peers and reached my full speed.

Users of XP SP2 or later should patch their tcpip.sys file to make the most of this increase, as the max number of halfopen connections was reduced.  More information can be found here.  This limitation was again removed in Windows Vista and Windows 7.

gui.update_rate
This won't affect your download speed but simply changes how often the data reported by the GUI is updated.  The default is 1000ms (1 second) and the lowest you can select is 500ms.  Any lower figures will be igored and 500ms will be used.  I prefer having it update at half second intervals, but if you find CPU usage is high, you can set it to update less frequently if you wish.

bt.connect_speed
This option specifies the number of connections uTorrent should make each second up to the limit set in net.max_halfopen.  The default is 20, and I set mine to 40.

diskio.sparse_files
This is an important option as it controls how the data is allocated on your hard drive.  If False is selected (which is the default), then uTorrent will write a dummy file to the hard drive as soon as you add the torrent, but fill it with zeroes (the place holder file as explained earlier).  This is fine for small files that can be written in a few seconds, but for large files like Bluray rips it can cause your drive to be busy for a long time, and the download speed will likely drop during this process as the HDD can't keep up with the incoming data.  It may also cause programs to become unresponsive.

Setting this option to True (as I have done) will inform the filesystem of the size of the file, but will not physically zero out the data.  Instead the only write operation uTorrent performs are writes for data actually downloaded.  This not only saves your computer becoming unresponsive when adding large torrents, but also means that the hashing process for incomplete files is much faster as it only hashes what has been downloaded, rather than a full size pre-allocated file.

In my opinion True should be set by default, however there are some limitations regarding the support of this mode.  It apparently only works on partitions formatted as NTFS (as opposed to FAT32 I suppose), however I can't speak of other filesystems such as EXT3.  Also if you are using a non-administrator account with a disk quota, sparse files won't work and it will still be allocated.

I encourage you to at least try this option if you've noticed high amounts of disk activity or unresponsiveness when adding large torrents.  This completely cured it for me.

Advanced - Disk Cache
Providing you set diskio.sparse_files to true, you don't really need to edit any settings here, but in case you couldn't (due to filesystem limitations for example), then these settings will help ease the pain of the file allocation stage.

These settings are fairly self explanatory, but most people rarely think about changing them.  I have done a little experimentation and find that the settings above give me the most preferable disk access.  Rather than writing data constantly, it stores it in RAM and writes it in chunks of about 80MB, or whatever your HDD can write in the space of a second.  If I'm downloading at my maximum speed, this means time between writes is 13 seconds plus, rather than the HDD writing 6MB every second.

You can check read and write patterns yourself in uTorrent.  Simply click on the Speed tab in the lower half of the window (the tab that gives you a upload and download speed graph) and change it to show disk statistics.  You can now see a bunch of cool stuff like how the cache is being used, how regular uTorrent is reading and writing to the HDD, and the throughput.

One setting I suggest you do experiment with is the cache size.  While the settings above should be good for everybody, the amount of RAM you have to spare for a disk cache will vary from user to user, and the amount of RAM you will need to allocate to make a decent buffer length depends directly on your download speed, ie a 1MB connection will require a lot less RAM buffer than a 6MB connection if the HDD is busy/unwritable for 2 minutes.

If you have a fast connection and download large files, I strongly suggest setting Override automatic cache size and specifiy the size manually to whatever figure you are happy to use.  This acts like a buffer in case the disk is busy and data cannot be written (a good example being the file allocation at the start of a large torrent, or if you do video editing etc).  The downloaded data is stored in RAM, instead of the download speed being dropped to a rate the HDD can handle.  Once the HDD becomes free again, the data stored in the RAM is then written to the file.  It's a lot more efficient than dropping the speed and/or disconnecting from peers and then reconnecting once the HDD becomes usable again.

You can calculate an effective cache size using the typical size of files you download and your maximum download speed.  Let's say that my system is set up to allocate disk space to new torrents and that I want to download a 4GB Linux ISO.  For the purpose of this example we'll say that my drive can write this file at 35MB/s - that means it would take 115 seconds to complete.  That's 115 seconds of the HDD thrashing and not being able to download at a decent speed (I'd usually get figures under 100KB/s when the disk was overloaded).

I know that I can download at a maximum of 6MB/s, and that I need to store this in RAM until the allocation has been written (115 seconds).  So 115s * 6MB/s would give me a value of 690MB.  In other words during the time it takes to write the pre-allocation, I could have downloaded 690MB, but without the buffer this wouldn't have happened.

And that concludes our uTorrent tuning.  If you find any better settings, please post them below