Showing posts with label Windows 7. Show all posts
Showing posts with label Windows 7. Show all posts

Wednesday, 23 May 2012

Formatting FAT32 Volumes Larger Than 32GB & SD Card Alignment

For a long time, flash memory for mobile phones and portable audio players seems to have been stuck at around 32GB.  Many devices report their maximum supported card size as 32GB, but this didn't stop me buying a Sandisk 64GB Micro SD for my Galaxy S2 and playing around with it.

Samsung claim the Galaxy S2 supports a maximum of 32GB via it's Micro SD card slot, but as the owner of AndroidNZ found out, this wasn't strictly true.  He had tried the card in the device, and once I saw that it was working, I decided to take a punt and buy one for myself.

64GB Micro SD card working flawlessly in my Galaxy S2, which claims to only support up to 32GB.

Initially you may run into some difficulty.  When I inserted the card into the phone, the phone said that the card is damaged or unreadable.  It appeared that my card shipped unformatted, which meant it had no filesystem.  The first challenge was to format the card in a filesystem the Galaxy S2 could read.  I tried the Linux filesystems first such as EXT2, EXT3 and EXT4, as the default filesystem FAT32 has a 4GB maximum filesize limitation, and I intended to use my phone as a portable HDD too (very useful for transporting or downloading Linux ISOs if you are caught out with a dead computer).

I found that with the Galaxy S2, EXT2, EXT3 and EXT4 were not supported, which was quite annoying and odd since that the internal memory of the Galaxy S2 uses EXT4.  So why can't it mount EXT4 SD cards?  I had also tried mounting external NTFS HDDs before with no luck, so I knew not to bother wasting my time with that.  I also tried exFAT but still no joy.  Hopefully since the Galaxy S3 supports exFAT the code will be retroported to the S2 (everything crossed).

The card even works on my 2008 Nokia N96!

So unfortunately that left me with FAT32.  For whatever reason, my S2 refused to format the card (but since I formatted the card, my phone will do it no problem now) which meant I had to do it on my computer, and immediately I ran into an old problem.  You see Microsoft impose a 32GB limit on partiton sizes for FAT32 in recent operating systems (XP onward I believe), despite the filesystem supporting partitions up to 2TB on paper.  My only guess is that Microsoft wanted to push NTFS as the dominant filesystem since it's a lot more robust.  I had tried formatting by right clicking the drive in "Computer", through launching diskmgmt.msc via the command prompt, and then even resorting to the CLI format utility, diskpart.  None of which would allow me to create a FAT32 partition of 64GB.

Fortunately, third party partitioning software will allow you to create FAT32 partitions in excess of 32GB, so it was time to find an alternative.

For simple partitioning, I found Easeus Partition Master which has a free version with all the functionality we need (for my final formatting, I eventually decided to use some tools supplied in Linux, but I'll come to that later as it concerns aligning the SD card too).  If you are a little more experienced with computers, or have used Linux in the past and are familiar with the filesystem structure, you could also use a GParted Live CD which boots into a basic version of Debian with the GParted program which is simple to use.

Additionally you could possibly use a Windows 98/ME boot disc or Windows setup disk to get into DOS and run fdisk.  Unfortunately even though I was able to find the boot disk files online, they wouldn't run within 64-bit Windows - they might possibly work within DOSbox though.

Partitioning With Easeus Partition Master
For the purpose of this tutorial I will be using an old 32GB Micro SD card as I have already formatted my 64GB and loaded it up with music and stuff.  The process is identical, but the reported volume sizes will be different.

Upon launching Easeus, you will be shown a screen like the one below.

Note the disk sizes, so that you don't accidentally delete partitions on your HDD!

Easeus is really easy to use, but here's a quick run down for those who might not have partitoned a disk before.

1) Upon opening Easeus, check to see if your SD card has any existing partitions.  The one in the screenshot is blank.  If you have existing partitions, select it by clicking on it, then right click and choose Delete Partition, then click Apply in the top left hand corner.

2) Right click the Unallocated space of the SD card you wish to format and click Create Partition.  Change the default settings to match the screenshot, so we are creating a Primary partition in FAT32.  You can adjust the space used if for some reason you want to create multiple partitions, but in general you just want to select the maximum amount.  The drive letter is not important, so leave that as the default.  Once you are done, press Ok to go back to the main screen.

The first part of creating a FAT32 partition
That's all there is to it, but if you would like to experiment with different cluster sizes which can improve write speeds, perform the final step below.

3) To change the cluster size, right click the partition and select Format Partition and choose a cluster size from the drop down menu.  Press Ok to go back to the main screen and press apply to write the changes.  In general, the larger the disk size, the larger the cluster size you would use.  Using larger clusters can help improve transfer speeds, but at the cost of disk space efficiency.  What this means is that files occupy space on the disk to the nearest cluster size.  So a 12KB file on a disk with a cluster size of 16KB, will use 16KB on that disk, or if the cluster size was 64KB, then that same 12KB file will occupy 64KB.

Aligning the SD Card
If you are comfortable with Linux and command line tools, then we can improve the speed of the SD card by aligning it.  The theory behind this is that SD cards erase/write 128KB blocks which means even if only 10KB in that block changes, the whole thing needs to be re-writtent.  Most formatting tools don't seem to be aware or care about how SD cards work and just format the FAT32 filesystem in the usual way, however better performance can be achieved if the File Allocation Tables end at the border of a block, with the data starting at a new block, rather than some of the FAT using part of a block and the data starting in that same part block.  This makes it easier for the SD card to write full blocks more often than it would if the partition was unaligned.

I found that with my SD card unaligned, I would get write speeds averaging around 6.5MB/s, but once aligned it would manage a consistant 9MB/s, and higher with large files.

Unfortunately I do not know of any SD card alignment tools for Windows when it comes to FAT32, so we will be using a Linux Live CD for this, but if your device supports exFAT, then I've found the official SD card formatter to give really good speeds.

We first need a Linux environment, you maybe already run some linux distro, but if not we can download a Live CD which runs the OS from optical media without installing anything.  I use Opensuse

Here is a link to the original thread.  The guide I will post will pretty much be a simplified version of this based on my own experience as I found the guide a little bit cryptic at first.

Partitioning The Card
1) First, insert the SD card so that Linux detects it and allow it to mount.  If it doesn't mount automatically, you can open the file browser and browse to the card as this will cause it to be mounted.  You may need to have a filesystem on the card beforehand, so you can just format it to NTFS or something in Windows as a temporary measure.

2) Open a terminal window from the progam menu and type:
df -h
df is the disk free command, which shows you the available space on your mounted drives, and the -h switch for "human" readable information, such as MB and GB rather than just bytes.
There are a few pieces of information we need here.  Locate your SD card by its capacity in the list, and make note of the Filesystem and Mounted on columns.



As you can see, my SD card is listed under filesystem as /dev/mmcblk0p1 and under mounted on as /media/15F4-1F1D

I should explain briefly about how Linux's filesystem works.  /dev/mmcblk0p1 is actually referring to a partition on the disk, and the disk itself is known as /dev/mmcblk0 - as you may have guessed, Linux simply suffixes p1, p2, p3 etc to the device name to denote partitions on a device.  /media/15F4-1F1D is the mount point and is the Linux equivalent to a drive letter, eg C:

3) Next we need to unmount the drive.  In the terminal window type (replacing the mount point with whatever your drive is mounted as when you typed df -h):
umount /media/15F4-1F1D

If you don't want to type the whole thing, you can start it off and press Tab to complete it automatically.  If the unmount was successful, you are returned to the prompt.  You can check it was unmounted by typing in df -h again and confirming that it is not in the list.

4) Now we need to change the drive geometry in order to make the 128k easy.  Flash memory works differently from harddrives and do not have physical, sectors tracks or heads, but it still uses that information to organise data.  The aim is to find a combination of sectors, tracks and heads that is evenly divisible by 128KB (131,072 bytes).  The suggested geometry of 224 heads and 56 sectors does that (224 heads x 56 sectors x 512 bytes per sector / 131,072 bytes = 49) whereas the default value (255 heads x 63 sectors x 512 bytes per sector /131,072 bytes = 62.75) does not.
In the terminal window, type the following:
su (then press enter, this enables super user mode, or admin)
fdisk -H 224 -S 56 /dev/mmcblk0

fdisk is the partitioning program, -H sets the number of heads and -S sets the number of sectors.

To confirm this worked, press p to get a list of the partitions and device properties



4) While still in fdisk, we will create the FAT32 partition.  To do this, type the following at the prompt:
d (then press enter to delete any existing partitions - if you only have 1 it is deleted automatically, otherwise you are prompted to enter the number of the partition)
n (then press enter to create a new partition, then select primary partition and then enter through the other options).
x (to enter expert mode)
b (to change the beginning of the data, then select partition 1 and set the start to 1)
r (to return from expert mode)
t (to set the partition type, enter c when prompted for FAT32)
x (to enter expert mode again)
b (to set the beginning of the data to 256 for partition 1)
r (to return from expert mode)
w (to write the changes to disk

You may press q at any time to quit without saving changes.

5) Finally we need to create the aligned FAT32 partition.  We first need to find out how big the FAT tables are going to be, so we can manipulate the size of the tables to finish at the end of an erase block (the 128KB blocks we keep talking about).
At the terminal window, type:
mkfs.vfat -F 32 -s 32 -v /dev/mmcblk0p1
Then press enter to format.  Once done, you will see something like below, giving a read out of the disk.  The line we need is the FAT size which tells us how big a single FAT is (there are two on this disk).


6) We then need to calculate the size of the FAT in bytes, to see if it will end on a whole erase block or not.  What we do is:
FAT size x No. of FATs x 512 bytes per sector / 131,072 = No of erase blocks
In this instance we would have:
14,976 x 2 x 512 =  15,335,424 / 131,072 = 117
Now if at this point the number of erase blocks is uneven, we need to pad that out to fill the erase block.  Simply round up your last result to the nearest whole number and reverse the calculation.

No of erase blocks x 131,072
Which gives us:
118 x 131,072 = 15,466,496

Now we subtrack the original FAT size in bytes, from the erase block aligned FAT size in bytes, then divide by 512 to give us the number of sectors difference to be used as reserve sectors.  This is essentially expanding the amount of space allocated to the FAT in order for it to finish on a complete erase block, so the data starts on a new block.

15,335,424  - 15,466,496 / 512 = 256 reserve sectors

7) Now we know how many reserve sectors are needed to ensure alignment to the erase blocks, we can continue to format with the correct information.
At the terminal window, type
mkfs.vfat -F 32 -s 32 -R 256 -v /dev/mmcblk0p1

The switches used are:
-F 32 (for FAT32)
-s 32 (cluster size expressed as sectors, so 32 x 512 bytes = 16KB clusters)
-R 256 (number of reserved sectors)
-v (device to be formatted)


Once that has completed, you can mount the drive to transfer items to it, or simply remove it and go back to Windows, or use it in your device.

Monday, 20 June 2011

Upgrading the Wireless LAN on a Vaio

Just recently, I upgraded my internet connection to 100mbps, but I found I wasn't getting the full speed over wireless.  I was downloading at around 80mbps max despite the fact both my wireless card in the laptop, and the modem/router itself are both wireless N devices and connected at a supposed 300mbps.

Surely if it's connecting at 300mbps, and is in the same room, it should reach 100mbps download no problem, right?

That's what I thought too.  I decided to look into it, and it turns out that this is just the way wireless is.  There are a number of reasons for this and I don't know them all, but I'll offer some suggestions as to why.  The first thing that I found is because wireless is effectively half duple - that means it cannot transmit and recieve at the same time, so that halves your bandwith to 150mbps right there.  The next big one is the CPU in the router itself.  Right now they are operating at around ~500MHz and it has to deal with everything from routing to encryption.  Another thing that will have an effect on quoted speeds are retransmissions due to transmission errors.

The theoretical quoted output of 300mbps is something you never see in real world situations, and in fact, you are lucky if you get 1/3 of that.  Just take a look at this chart.  Not even the fastest dual band wireless N device breaks 90mbps, so where does that leave me?

Well fortunately there are wireless N devices starting to come out that offer higher bandwidths up to 450mbps (which I'll take to mean a real world maximum of 150mbps if my previous experiences are anything to go by).  It is much the same as the current 300mbps wireless N, but where the 300mbps variant of wireless N uses 2 antennas (known as dual/two band/stream), 450mbps uses 3 antennas.  However as you might expect, it's not simply a case of buying a three stream router and expecting it to work.  You need a wireless card with three antennas too.

That presents me with a little problem.  Swapping a two stream wireless card like for like is easy.  Just take the back cover off, remove the old card, install the new card and connect the antennas.  However if I want to upgrade the wireless in my laptop to three stream, not only do I have to take the back off to install the card, I also have to remove the media keys, keyboard, LCD assembly and eventually the LCD panel itself to install the new antenna, since they are installed behind the display.  This was an ambitious upgrade, and essentially entailed a complete teardown of my laptop.

This guide details the process of upgrading a Vaio F12 (but should also work for F11 and F13 models) from the stock wireless card (in this case a dual stream 300mbps Intel Centrino Advanced-N 6200) to a three stream 450mbps Intel Centrino Ultimate-N 6300.  If you are simply swapping a dual stream (2 antenna) card for another dual stream card, you may still find this guide of use.

Please note that this upgrade may not be an option for some laptops.  Some manufacturers (eg. HP) lock the BIOS to only recognise certain brands and model numbers of wireless cards and will reject upgrades.  Even if this isn't the case, there is always the possibility that there will not be enough room in your case, cable path or behind the display to install the extra antenna.  Proceed at your own risk.


The Upgrade Process
Replacing the wireless card is easy, but the difficult part is installing the new antenna.  Do not proceed if you are uncomfortable with taking your LCD apart.

Make sure that you have earthed yourself and that you are not wearing any clothing that generates static electricity.

Things you will need:
  • A PZ0 Screwdriver.  Must have a thin shaft to fit inside the recessed holes.
  • Something thin and flat to push tabs in (eg a flat screwdriver).
  • A knife or something sharp to remove LCD screw covers.
  • Some sticky tape (not required on the Vaio if you use the original sticky tabs).
  • The service manual or a laptop with this website for more detailed information.
  • A three stream wireless card (eg Intel Ultimate-N 6300).
  • An extra antenna.
  • Drivers for your wireless card.
  • A backup of your drive just in case.
Notes:
The service manual is for the Vaio VPCF1 series, which appears to cover the F11, F12 and F13.  It contains exploded views and part numbers, so it's worth keeping just in case.
Drivers should be downloaded before hand unless you plan to connect via ethernet.

Please refer to this website for detailed images on opening the laptop.  It's fairly simple, so I shall just talk through the process.  Refer to this other website for details on removing the LCD later.

Installing the Wireless Card
1) Before you start doing anything to your laptop, create a system image and restore disk Windows backup and restore.  This can be found in Start > Control Panel > Backup and restore.

2) Next go into the device manager and uninstall the old wireless card.  You can get to this by right clicking the Computer icon and selecting Properties.  In Windows 7, there is a link to Device manager in the top left of the window.  Alternatively you can press the Windows key and type device manager in the search box.

Expand Network adapters, right click your wireless card and choose Uninstall.  When the next prompt is shown and it asks if you would like to delete the driver, select yes if you are installing a wireless card that is a different model or brand, or no if the wireless card uses a unified driver (for example, the Intel Advanced-N 6200 and Ultimate-N 6300 use the same driver).


3) Shut down, then remove the power cable and disconnect the battery.  I waited a few minutes with the battery out, and then held the power button in for about 10 seconds to drain any remaining power.

4) Now remove the DVD/Bluray drive.  This is secured by two screws, one in the top left near the power button, and one to the left of the RAM cover.  These have 3 small dots above the hole.

5) Proceed to remove the HDD.  Looking at the HDD cover, there are two screws at the bottom, remove these.  Again these have 2 small dots above the hole.  Once you have removed those screws, gently slide the door toward you to reveal the HDD.  Remove the two screws at the top securing the HDD and slide it left.  Once it is free from the connector, lift it out.

6) Remove the RAM cover.  The screw that secures the RAM cover does not actually come out as there is a retaining clip under it.  What you have to do here is unscrew the door as best you can and then lift it up from the screw side (you should be able to slide a nail under it).

Back in the lab.  Removing the HDD, BD and memory cover before removing the back cover

7) Remove the remaining screws from the back cover.  I placed them on a table in the same position that I removed them from the laptop, but it would be better to draw a quick diagram to save confusion.

8) Now to remove the back cover.  This was tricky at first as I wasn't sure what to expect.  The bottom 1/3 of the cover felt as though it was still secured by something.  I thought that maybe there were screws under the rubber pads but that wasn't the case.

Joe Bleau suggests working from the HDD side and carefully sliding the cover.  When I tried this it felt like it wasn't working (perhaps I was doing it wrong), so the method I used was to start from the opposite end and run my fingernail down the front of the case carefully forcing it apart, working from right to left.  I stopped just as I got to the HDD bay and lifted the case at the right hand end a little which also seemed to lift some of the PCB with it.  With the PCB clear of the lower half of the case, I was able to slide it right (as suggested) and get the back cover off.

I suggest going with the original instruction of Joe's of , "Starting from the hard drive bay side, slide the cover, shake & bake with care & patience & remove the cover."

Patience is definitely the key here.  Fortunately getting the case back on is rather easy.

9) You should be able to see the wireless card easily.  It's just to the left of the RAM and up a little.  Remove the screws securing it and slide it left to remove it.  Carefully turn it over and make a note of which antenna is connected to which port on the board.  The Intel boards are numbered.  It might not even matter which port you connect the antennas to, but if you can connect them to the same numbers, you might as well, right?

Wireless LAN card with the two antennas under it

10) Disconnect the antennas from the original wireless card and then connect them to the new one.  It's a very tight connection and you will find you have to apply a lot of pressure, so just be careful.  Once they have snapped on to the new card, slide it into the slot carefully.

11) Work back from step 8 to reassemble the laptop.  Put the back cover on and install all the drives and screws you took out so it is in original condition.

12) Power on and install the driver/software for your wireless card to ensure it works and is accepted by your laptop.  There is no point going to the trouble of installing a new antenna if the laptop BIOS is set to refuse other brands or models of wireless card after all.

If you are using a wireless card that uses a unified driver (like my upgrade from an Intel 6200 to 6300) and you chose not to delete the driver when you uninstalled it earlier, Windows will detect the card and install the driver automatically.  All that was left for me to do in my case was to enter my wireless password and  set my static IP settings.

If you were upgrading an old two stream card to a newer two stream card, this is all you need to do up to this point.  All further instructions will relate to installing a third antenna for three stream cards.

Installing Another Antenna
1) Power down the laptop and remove the battery and AC adapter.  Hold the power button in for around 5 seconds to drain any remaining power.

2) Flip the laptop over and remove the screws show below.  You do not need to remove any drives for this part of the install, just the battery (for safety, but there are some screws under that which we need to remove).  Removing these will allow us to remove the keyboard, media keys/speaker cover and eventually the LCD assembly.

There are two screws under the battery that you need to get.

3) Turn the laptop back over and open the LCD in the normal positon for use.  Starting with the media keys/speaker cover, start from the side nearest the power switch on the right.  Using a sharp object or your fingernails, push it in and gently lift upward.  Do the same to the left hand side, then run your fingernails across the front of the speaker cover to push in the rest of the tabs.  Carefully lift it upwards once it is free.  You may need to move it around a little or angle the LCD to get it free.

Speaker cover removed.  Excuse the dust, it gets everywhere once you've had the back cover off.

4) Now to remove the keyboard.  Carefully lift up one corner for example near the escape key, then insert your thumbnail and work toward the bottom of the keyboard.  You should feel the tabs pop out.  Do the same for the opposite side.  Carefully lift the keyboard up slightly, and pull it towards the LCD at the same time.  You should feel it pop free of the chassis, but be careful not to damage the connector.  Turn the keyboard over so it is resting on the palm rest.

Now would have been a great time to install a sexy backlit keyboard.  Why do I never think of these things ahead of time?

5) Next we have to remove the media keys as the wifi antennas are actually routed underneath them.  Just remove the three wide screws and unclip the power button connector on the right.  This will give us enough space to move the media keys out of the way so we can lay the new antenna cable.

6) Now remove the screws securing the speakers.  Do not worry if your right speaker only appears to have one screw, this is because it is secured from the other side with the screws that hold the back cover in place.  Fold the speakers inward so they are resting face down.  This will give you access to the LCD hinge screws, but we will come to that later.

7) Now that you have the speakers and media keys out of the way, you should be able to disconnect the wireless antennas and the LCD cables.  The LCD cables are on the left.  The leftmost cable should be pulled upwards gently, and the two smaller cables should slide towards the LCD itself.  It's a really fiddly job and you may find it easier if you pull the cable out of it's trench first.  Moving over to the wireless antennas, this is equally fiddly.  I found it easier to ease them out with a sharp object, or if you want to be careful, whip the back cover off and disconnect them there.  If in doubt, knife them out.

These are the only cables you really need to bother removing

8) With the LCD and antenna cables free, you can now seperate the LCD from the base of the laptop.  If you haven't already, move the speakers aside and that will reveal the hinge screws, 2 on the left and 2 on the right.  They are really long screws and tougher than the others to remove.  Be sure to support the weight of the LCD assembly with your other hand while unscrewing the hinges.  Get someone to help is possible, or use a magnetic screwdriver to make life a little easier.  Once that is done, you should be able to pull the display free from the base.

This is how the iPad was born, when Mr. Jobs thought, "Hey, let's take a laptop and remove everything good about it".  Probably.

9) Now it's time to work on removing the bezel (that's the surrounding of the LCD).  It is very flimsy, so be careful at all times.  First take a sharp object for example a knife and prise out the screw covers.  There are 4 in total, two square covers at the bottom left and bottom right, and 2 round covers in the top left and top right.  Once those are out, remove the screws and place them somewhere safe.

Now work on unsnapping the bezel.  It is held in place by a number of tabs.  I suggest working it from the top left or top right where the round screw covers were and push it apart with your fingernail.  Once you have got a bit of a gap started, go around the bezel unhooking the tabs by pressing inward with a little pressure.  This is a long and frustrating job and is largely trial and error.  Just take your time and be careful not to apply pressure to the LCD itself or scratch it.  I had the most trouble unsnapping the lower half of the panel, in part thanks to the hinges.  It's a similar story with the removal.  Once you have everything unsnapped, pull the bezel toward you (if the hinges are facing away from you) and gently wriggle it free.  This is not an exact science, but you can rest easy knowing that putting the bezel back on is about the easiest part of this whole upgrade process.

10) Now the bezel has been removed, it's almost time to install the new antenna, but first we have to unscrew the LCD panel so we can get behind it to install the new antenna.  Just remove the four screws from the bracket securing the panel to the back of the case (two on the bottom left and two on the bottom right).  When it comes to reinstalling these screws, just look for the arrows pointing to the holes in case you forget where they came from.  You may also use this chance to disconnect the LCD from the inverter so you can completely remove it from the case, which would make life nice and easy, but I chose to leave it connected as the cable was slack enough to give me a enough space to work with.

The hinges seemed more trouble than they are worth to remove, but the new cable just pushes in from the side.

11) With the panel removed (or propped up on a roll of tape in my case), we can go about laying the new antenna cable.  We will concentrate on the connector end because we don't want too much slack cable in the laptop itself - slack cable is easier to store in the LCD as there is much more space.  Align the ends of the connectors according to the mounting position on the wireless board, for example the third antenna connector on my Intel 6300, is between the first and second, so the connector for the third antenna needs to be somewhere between the existing cables in length.

Once you have the connectors aligned as you want, push the remaining cable into the side of the hinge and follow the existing cables to see how it's routed.  This sounds like a pretty vague instruction, but once you see the hinge for yourself, you will know exactly what to do.  This method will be pretty easy if you have removed the panel totally, but if like me you was lazy, you might find it easier to run the whole cable under the panel first, route it through the hinge and then pull it through to suit the required length.  This part is mostly trial and error.

You may notice something that look like tabs.  These are very sticky, and if you pull them up carefully, you can run the new antenna cable under them with the existing ones to keep it nice and tidy.  If you don't have such tabs, normal sticky tape will probably do the job.

Be careful when lifting these tabs.  They are so sticky that they might tear.

12) Once you have your cable routed, it's time to stick the antenna on.  The suggestion is to place antennas at least 12cm apart.  I placed mine somewhere in the middle as it allowed me to get a decent distance from the existing antennas as well as not getting too close to the LCD inverter.  Unfortunately the cable is quite stiff and very curly so the finished result looks kind of rushed.  I tried to route it on the inside of the other cables but it didn't quite work out.  It probably would have been better to go to the right of the cables instead.

Rest your LCD panel in it's mounting position to make sure it fits flush and isn't affected by the new antenna (eg it's not resting on it).

Avoiding a cable crossover was next to impossible thanks to how short the cable for the antenna on the right is.

13) Work back from step 10 to get your laptop back into normal working order.  Pay particular attention at step 7 when reconnecting the cables associated with the panel (LCD cables and wireless antennas).  You may find that it's a bit of a tight fit when putting the antenna cables back into their trench due to the fact you have an extra one.  I was able to get around this by pushing them into the clips with a screwdriver.  Once you have worked your way back through all the steps, power on and enjoy your new speed, but remember that you need a three stream router if you don't already have one.

Not a bad job in the end.

Special thanks once again to Joe (check his website here) for his help.  Without his guide and the excellent links he gave me, I doubt I would have had the confidence to attempt this upgrade.

Wednesday, 8 June 2011

Windows 7 Tips for the OCD

If like me, you are mildly OCD (or perhaps pedantic is more accurate), then you might appreciate these little bits of information that will help make your Windows 7 UI a little more pleasing to the eye.

If you have a large, high resolution screen, you might notice that Windows 7 doesn't look so hot in places, such as the login screen background being very low resolution and suffering from JPG artifacts.  Similarly, if you have decided to change the default user icon/avatar to something of your own, you might find that it looks a little blurry compared to the preinstalled icons.

Fortuantely in my fiddling and tweaking with Windows 7, I've come across some work arounds and information that will improve this situation.

Windows Login Icon
If it's one thing I like, it's a crisp image.  It has been a habit of mine to make sure I get the dimensions right when creating avatars and icons to avoid them being rescaled.  In fact, this is basic and you no doubt already take this into account when selecting or making desktop backgrounds.

I googled around at first to save myself the trouble of trial and error, but all information was pointing to the icon size being 128x128 pixels.  I created my icon with those dimensions and it didn't look as sharp as the supplied icons - I could tell it was being rescaled.  I managed to find a screenshot of the Windows 7 login screen and measured the dimensions of the user icon space in Photoshop.

It turns out the correct dimensions for the Windows 7 user icon at the login screen is 126x126 pixels.  I'm now using a 126x126 PNG as my login icon, and it's pixel perfect, no rescaling anymore.

You might want to note that Windows Live Messenger uses 96x96 pixels icons and also accepts PNG.

Desktop Background
It actually wasn't until last night that I noticed this.  I've been using a scan of some Ocarina of Time 3DS promotional art as my background for a while, and I found a high resolution PNG of it on the Nintendo E3 press site, so I decided to recreate it as the original had some weird artifacting around Ganondorf's hair despite the image being very good quality.  I didn't know why and presumed they were scanning artifacts.

After I had cropped and downscaled the new image, I saved it as PNG and applied it, but the weird thing was is that my recreated image didn't look as good as the image I was replacing.  If anything, Ganondorf's hair looked worse now than before, and I thought I saw some JPG artifacts around Link's ear.  That set some gears turning upstairs, so I pulled up the original PNG in window and compared them.

Left: Original PNG, Right: Original when set as wallpaper in Windows 7.  Notice the artifacts around his ear.

Sure enough the background appeared to have JPG artifacts and was evidently lower quality than the source file despite the fact that the source was a PNG.  It turns out that Windows 7 converts BMP and PNG wallpapers to JPG, and after a quick google, I found it stores them in "%appdata%\Microsoft\Windows\Themes" (paste that into an explorer window, or press the Windows key and R and paste it in there).

Saturated colours are a huge giveaway.  Who at Microsoft thought it would be a good idea to convert lossless wallpapers into JPG of this quality?
Fortunately there are two workarounds for this.

Method 1
Windows will recompress BMP and PNG images to a lower quality JPG, but it does not recompress images that are already in the JPG format, it simply copies it to the themes folder in %appdata%.  Open your BMP or PNG wallpaper in something like Adobe Photoshop or GIMP if you need a free alternative, and save it as JPG using the highest quality setting.  It's not lossless, but it's so good I doubt you will be able to tell the difference.  Apply the wallpaper in the usual way.

Method 2
This method will allow you to use a lossless background (eg BMP or PNG) without ever converting it to JPG.  Just get your image, drop it into an open Firefox window, then right click and choose Set As Desktop Background.  The only real down side to this is that Firefox converts it to a 32 bit BMP, which uses around 8MB for 1080p.  Ouch.



Microsoft, if people select BMP or PNG as a desktop background it's because they want a lossless image.  Don't go around compressing it to a shitty JPG.  My computer is supposed to do as I tell it to, not what someone else thinks is good.

Replacing the Login Screen Background
When I was testing the Windows 7 RC, I noticed that the login screen background was horribly stretched and full of JPG artifacts.  It irritated me, but I let it slide, and assumed they would put lossless backgrounds in for the retail version.  Well it didn't happen.  When I bought my Vaio at the beginning of 2011, it was the first time I had used a full retail version of Windows 7, and I was surprised to see the same low quality background.  It felt like a downgrade, because Vista certainly didn't have JPG artifacts, and I'm pretty sure the Windows XP login, as simple as it was, didn't suffer any quality issues either.

But one day my sister got a new laptop and to my surprise it had its own branded login screen.  The quality was really good too.  This gave me hope.  I had previously not even considered the possibility of changing the login background because I presumed it would be hidden deep within the system files and be tied up within a DLL, a locked file or something that wouldn't be worth the effort changing.

One of the workarounds for the low filesize limitation is to decrease the brightness of the image.

I googled it anyway and it turns out it's stupidly easy to change the login background, however there are a couple of limitations:
  • The image type must be JPG
  • The file size must not exceed 249KB (256,000 bytes)
  • Must be named backgroundDefault.jpg
Unfortunately a filesize that low can impact the quality for large resolutions, but there are a few tricks to help with that which will come later on.

How to use a Custom Login Background
1) The first thing you need to do is download this registry entry.  You may need to show the file extensions in Explorer and delete the .txt that mediafire seems to have added to it (so it ends in .reg).  Double click to add it to the registry (or right click and select merge).  This tells Windows 7 to prefer custom backgrounds over the OEM background.

2) Open an explorer window and navigate to C:\Windows\System32\oobe\  At this point you may need to create another couple of folders if  they don't already exist.  Right click and make a new folder called info within the oobe folder.  Open the info folder and make another folder in that called backgrounds.  Once you open the backgrounds folder, your path should read C:\Windows\System32\oobe\info\backgrounds

3) At this point we need to prepare/compress the image.  To prevent scaling of the image (which is what gives the default login background that pixelated appearance), you need to make sure the image is the same resolution as your display.  Right click your desktop and select screen resolution to find out the dimensions of your display, and match these dimensions if you are downscaling or cropping an image in your photo editing software.  If your image is smaller than your display, it is generally better to upscale it in your photo editing program than let Windows do it.

When you have the image at the correct resolution, it's time to compress it.  We need to get the filesize under 249KB, but at the same time squeeze every last bit of quality out that we can.

For Photoshop:
To save the file, click File > Save for Web & Devices.  This gives us much finer control over the JPG quality levels than the usual JPG export under File > Save As.  Select the Optimized tab to see a preview of your compressed image and then pick JPEG high from the presets in the top right corner.

The Photoshop Save for Web and Devices export screen

Taking note of the filesize in the lower left hand corner of the window, adjust the value in the quality setting (in the top right) until you get close to 245-250KB.  You can also experiment by checking and unchecking Optimized and Progressive as these will usually save you 5-10KB for free.  If embed colour profile is checked, uncheck it.  When you have reached your target filesize, hit save and put it on the desktop for now.

For GIMP:
To save the file in GIMP, click File > Save As.  In the save dialog that pops up, change the extension to .jpg and press enter (just save to the desktop for now).  This will take you to the JPEG compression options.  It's similar to Photoshop in that you have a quality slider with fine control over the amount of compression.

Check the box that says Show preview in image window.  This will enable a preview of the image quality after compression, but more importantly the filesize.  Move the slider left or right until you get to around 245-250KB.  You can expand Advanced Options to gain access to things like Optimise and Progressive which will save a few KB without affecting the image quality.  There are also advanced options such as subsampling which controls how the colour is encoded.  If you are saving a JPG to use as a desktop background like earlier, I'd suggest using the highest quality level, and setting the subsampling to 1x1,1x1,1x1, but for the purpose of the login background, you might want to experiment with the other subsampling options.

4) Once you have your compressed image, it's time to move it to the C:\Windows\System32\oobe\info\backgrounds directory.  The image should be named backgroundDefault.jpg.


Providing you applied the registry entry successfully and got the filesize small enough, your custom login background should now show when you Ctrl Alt Del or lock your system with Windows key and L.  If the background does not show, you might need to reboot.  If it still doesn't work after that, compress your image so it's a little smaller (I read the limit is 256KB, but it appears to be closer to 249KB or 256,000 bytes.

Once the registry entry is applied, you can change the backgrounds in real time without reboots by just changing out the image stored in the backgrounds folder as long as it has the same name.  If it doesn't show, it's likely to be a filesize issue or the JPG settings (eg Windows doesn't like the subsampling or optimisations the program uses).

Disabling the Boot Logo
I had used Windows 7 on a 4:3 laptop (1400x1050) for quite a long time, maybe over a year, so imagine how many times I have seen the boot logo.  Well now that I have a widescreen laptop, that boot logo simply annoys me.  Why?  Because it's the wrong aspect ratio.  Apparently Microsoft can cram their OS with 10GB of redundant crap, but they can't be bothered to put in a boot logo for widescreens.

It's not as if widescreens are anything new either, and they are only becoming more popular as time goes on.  Well there isn't really a fix for this, aside from unpacking a system file and resizing 105 or so images.  It's just not worth the time (even if I automate it) and not worth the risk of screwing something up.

So my solution is to not look at it at all, and here's how you can do the same.

1) Press the Windows key and R.
2) Type msconfig and hit enter.
3) Go to the boot tab and check No GUI boot
4) Click Ok to confirm

Now the next time your computer boots, it just shows you a black screen (with some weird little dotted lines in the top left corner as it loads).  I found that skipping the boot animation speeded up my boot by about 5 seconds, so it's a win-win.

The only downside to this is that I rebooted to run a surface scan on my HDD once, and because the GUI did not load, it just ran the scan with nothing on screen so I had no idea what point it was at.  That wasn't too bad since the program was automated, but it might cause some problems if you ever require user input.

Thursday, 7 April 2011

Upgrading to an "Advanced Format" HDD (Windows 7)

About a month or two back I bought a new laptop, since my old HP was literally falling to pieces (the LCD hinge broke and it the metal bracket was stuck.  Not even a hammer would get it to move).  So rather than run the risk of the LCD cable becoming severed one day and not having a laptop, I decided to buy a Vaio and went for the VPCF12M0E.  It has an i7 740QM CPU, 4GB of RAM and 500GB HDD.  I decided on this model becuase in my opinion, anything over 4GB of RAM right now is a waste.  I'm a pretty demanding user and I've yet to reach a point where I feel 4GB isn't enough.

It's pretty nice.  The CPU encodes video literally 5x as fast as my old dual core Athlon 64 2x2GHZ TL-60 in the HP, the memory scores high on the Windows Experience Index, and the GPU is decent too.  The only thing that let it down (as is usually the case with laptops or manufacturer assembled computers) was the HDD.  It shipped with a Western Digital 500GB Scorpio Blue (WD5000BEVT).  I could tell straight away that it wasn't great, and while I didn't perform any proper benchmarks, it scored 5.8 in the Windows Experience Index, compared to an old Western Digital 250GB Scorpio Black (WD2500BEKT) which scored 5.9.

At this point I should tell you that the in Windows 7, the Windows Experience Index caps the score for platter based HDDs (ie not solid state) at 5.9.  It's possible that it would have scored even higher.

The Blue didn't seem unusually slow as such, just I remember my 250GB Black feeling faster and more responsive.  The main differences between the Blue and Black series notebook HDDs are:
  • Blue - 8MB cache.  Black - 16MB cache
  • Blue - 5400 RPM.  Black - 7200 RPM
The Black drives also typically have a longer warranty (5 years compared to 3), but are generally a step behind the Blue when it comes to capacity (at the time of writing the largest Blue notebook drive is 1TB, and the Black is 750GB).

However data density also plays a part in speed, and the general rule of thumb is that the larger the HDD, the higher the write/read rates, so although the 500GB Blue lost out on cache and RPM to my old 250GB Black (which are pretty major points to lose out on), it does high a higher data density, so sequential throughput shouldn't have suffered quite as much - it's more seek operations and loading multiple files where the Black series outclasses the Blue.

So after looking at my options, I decided to go with a Western Digital 750GB Scorpio Black.  It was the right combination of capacity and speed for me (though I really wish they had made it available in 1TB.  I smell a marketing ploy...).

I had also looked at the Seagate Momentus XT 500GB.  It's an interesting drive as it's a hybrid.  It has platters like any other harddrive, but it also has 4GB of on-board SSD storage where most accessed files are stored (eg the OS files).  The performance is really impressive, but since it's only 500GB it would have been a sidegrade, so I passed on it.  Hopefully this type of drive will catch on.  Who would need SSD if we have 2TB platter HDDs with 16GB of SSD?

Advanced Format
My choice of the 750GB Black was a relatively easy one.  Right now it's probably the best performing HDD of that capacity.  The Momentus is awesome, but I wanted extra space as well as extra speed.  However I ran into a little problem.  Something called "Advanced Format".

What is this and why is it such a big deal?  Ever since HDDs were first created way back in 1956, they have used 512 byte sectors.  It was predicted that using current technology that HDD sizes were going to stagnate.  One solution to this was to use larger sector sizes which allows for more efficient usage of the recording surface which in turn allows the manufacturer to use more robust error correction which is needed at the ever increasing data densities.

The 750GB Black uses 4096 byte physical sectors, but the controller still communicates in 512 byte sectors (4k emulation).  Think of it as a kind of translation.  The decision was made to use this kind of emulation because of the many programs and operating systems that presume a HDD will have 512 byte sectors - think of it in a similar way to how your 64-bit OS and CPU still offers emulation for 32-bit programs.  Going full 64-bit would be a pain right now.

I had read about this prior to buying the 750GB Black and I got myself confused.  Due to the fact that the sectors are larger now (8 sectors combined into one if you think of it that way), it's possible for data to be unaligned.  That is instead of the 4k sectors sitting nicely like this:

Some operating systems and partition creation utilities can create paritions that are unaligned.  I do not understand fully, but I believe it's to do with the location and/or size of the Master Boot Record.  In this case, data can become unaligned like this:
That creates two specific situations.  One where data is aligned (as the first example) and one where it isn't.  To the best of my knowledge it doesn't cause any compatability problems, just decreased read and/or write performance.  If your data is unaligned, then you need to use a tool like WD Align to fix it.

Here is a chart Western Digital put together to help end users determine whether or not they need to run the WD Align tool on their disk.

As you can see, Windows XP (and presumably earlier) requires you to run the WD Align tool, since the OS is not 4k sector aware, and neither does the OS really "know" that the drive has sectors of these size since the drive's firmware still communicates in 512 byte sectors (4k emulation).  In this instance, XP creates a partition that is at a sub optimal offset for 4k drives, and the WD Align tool fixes that.

Windows Vista, Windows 7 and Mac OS use an offset that is optimal for 4k Advanced Format drives and does not require alignment under normal circumstances (for example a clean install of one of these operating systems).

However cloning utilities are a grey area.  Western Digital claim that you should align the partition after restoring the data to ensure full performance, however I believe that some disk cloning utilities (such as Macrium Reflect) support 4k drives and would not require alignment after (providing the OS supports the correct drive offset).

This is what raised questions with me.  I needed to migrate my data from a 512 byte sector HDD that I did not know the offset of to a 4096 byte sector HDD.  Luckily since the old HDD was formated with Windows 7, it already had it's data at the right offset, but I think that would probably end up to be irrelevant anyway.

I decided to use the internal Windows 7 backup/restore utility.  One of the reasons was because it's was free and readily available.  The other reason was that since this is a backup tool, rather than a dumb data cloning program, I expected that upgrading a HDD this way would be less hassle as this type of situation will have been considered when Microsoft were designing the software.  Think about what happens if you have been making regular drive images, and 5 years later the original disk dies?  You may not be able to source an identical drive, so I expect there is some amount of flexibility within how much, and the type of data is stored, to allow you to restore to other brands, sizes and types of drive.

The Migration Process for Windows 7 SP1
This process will most likely work for Windows 7 installations without the service pack, but the following guide will be based on SP1 since that's what I used and can confirm it was successful.

What you will need:
  • USB HDD with at least as much storage as the HDD you are migrating (prefereably 2x and not password protected)
  • Screwdrivers (this is obvious but you usually need a very small screwdriver also to attach your HDD to a cradle)
  • Latest chipset drivers (Intel Rapid Storage Technology must be 9.6 or higher)
  • "F6" chipset drivers if your vendor supplies them
  • Blank media (At least 1 CD/DVD/BD for the recovery disk, more if you plan to save your HDD image to disc instead of USB HDD)
1) Connect your USB HDD.  We are going to make an image of your current drive, prior to messing around installing updated drivers.  This is just a precautionary step, but is good practice if you intend to update your HDD controller driver.  You should ensure your HDD is not encrypted, hidden or requires unlocking of any sort.  You won't have access to Windows during the recovery stage, so it will not be possible to run the unlocking software for these types of drive.

2) Grab your blank media and run the program.  Once the image has been made, it will prompt you to burn a recovery disk.  This contains the bootable program that will transfer your disk image stored on the USB HDD to your new internal HDD.  To start the image creation process click Start > Control Panel > Backup and Restore.  In the left hand pane, click Create a system image.  Follow the prompts to create a system image.  How long it will take to make the system image depends entirely on how much is stored on your HDD and how fast your external is, so it may take a while.  Once the image has been stored, you will be prompted to burn a recovery disc, click yes.

3) Now that you have a system image of your current configuration and a restore disc, it's time to ensure your chipset drivers are up to date.  For Intel chipsets, the Rapid Storage Driver must be version 9.6 or higher for 4k sector support.  The Windows driver can be found here and the F6 driver (which you load during the recovery process) can be found here.  Download and install the Windows software.  In my case I had to boot into safe mode (keep pressing F8 as your system boots) to install, as it would fail under normal conditions.  Unzip and place the F6 driver on a USB stick or the HDD that you stored your drive image on.  At this point, it would also be good to download and install any Windows updates.

4) Once your chipset driver is up to date and any OS updates are installed, it's time to make another image.  Before we do this, we need to rename, or move the original image to another location (since the default action is to overwrite the old image).  Open your USB HDD in your file browser and look for a directory called WindowsImageBackup which should be in the root.  Within that directory will be another directory named the same as your computer (in my case it says RX-93).  Rename this directory so that you will be able to tell it's the previous version (eg. RX-93 Old).  Now repeat step 2 to make your new drive image.  This will appear in the same directory as the name of your computer and this will be the image that we restore onto the new HDD - that's why it's important that we mark the first image as old.

5) By now you should have 2 system images on your HDD (the original one before installing updated drivers, and the current one with the new drivers/updates), a recovery disc (CD/DVD/BD) and dependant on your chipset vendor, some F6/installation drivers on the external HDD to be loaded during the restore process.  If you have all that, power off your computer/laptop, remove any batteries if applicable and press and hold the power button for a few seconds to drain any extra power.  Before removing the HDD from it's anti-static packaging or from the laptop, you should ensure you are earthed and/or that any static electricity has gone.  You can do this by touching a radiator for example.  The enthusiastic lady at Western Digital tells us that clothing can also generate static electricity, so install it naked if you wish.

If you haven't replaced a laptop HDD before, check out this video.  The laptop used in the video is the same model HP laptop I have, and I can tell you that the process is very similar.

6) Once you have your new HDD installed, and everything is closed back up and screwed into place, it's time to attach the battery and the AC cord.  Do not try to run the restore process using the battery alone because it will take a long time, and will likely run flat during the process.  Turn on your laptop and insert the recovery disc if it isn't already inside (you may need to power off and power on again if you missed the point where the system reads from the optical drive).

7) Once the recovery program starts to load, attach your USB HDD.  Follow the prompts, but look out for the Load drivers option.  Click this if you have downloaded the F6/install drivers, browse to them and load the applicable chipset driver.  It is likely to give you a list of different drivers, so pick the AHCI driver for your chipset (which you found earlier in the device manager).  Proceed through the prompts, selecting to restore using an image.  The option to format the drive will be greyed out as this is the default action for a blank disk.  Click next and the restore process begins.

8) All being well, your new drive should boot up with all your configuration, programs and files intact.  If you used a different brand or model of HDD, you will be prompted to restart your computer.  This is normal as it detects the HDD as new hardware since it's ID was different to the original.

You may notice that once your computer boots that even if you installed a larger HDD, that the capacity appears to be the same.  The reason for this is that the restore program only formats the same amount of space that was available to the old drive.  However we can extend the partition to use the full capacity, or even format the remainder so it appears like a seperate HDD.  Click Start > Control Panel > Administrative Tools.  Double click Computer Management and then select Disk Management in the left hand pane.  This will bring up a list of drives attached to the computer.  You will notice Disk 0 will have some space unallocated.  Right click on this and choose Extend Partition.  Follow the prompts all the way through (this will extend the C:\ partition to use the whole of the unformatted space).  The process takes a matter of seconds.

Once that's done, enjoy your new HDD.



    For a more information on Advanced Format, check these links:
    http://en.wikipedia.org/wiki/Advanced_Format
    http://www.wdc.com/advformat
    http://consumer.media.seagate.com/2010/03/the-digital-den/4k-sector-hard-drive-primer/