The Hard Drive Prepping Primer
Robert Spotswood
Partitions Part II
The Command Line Tools
Introduction
Welcome back! Partitions, Part I (http://www.hal-pc.org/journal/2006/06_july/feature.html), covered what they are and some considerations on how to use them. Part II of this series will cover some of the command line tools (CLT) and nearly command line tools (i.e. ones that don't require a full graphical interface and don't support a mouse) useful in creating and manipulating partitions. All the tools covered are free or come with your Operating System (OS).
Now, some of you might be thinking, "Command Line...ewwww! Do I really need to know this? Aren't there graphical tools I could use instead?" Yes to both questions. You need to learn the command line as the command line tools are actually some of the most readily available tools out there. In some cases, they may be all you have to work with. They can also be faster to work with than some of the graphical tools as well and are free. There is a learning curve, but with that curve comes understanding. For those that insist on graphical tools, those will be covered in Part III.
One thing all the Linux tools have in common is they query the hard drive itself for its parameters, rather that relying on the BIOS like the Windows tools. In Windows, if your BIOS only detects your new 300 Gig hard drive as 137 Gig (due to the BIOS being too old), then that is all you get from Windows without using some special software overlays usually provided by the drive manufacturer. While these overlays do work, many to all of the standard tools will now either not work, or totally hose your hard drive, or both. In general, avoid the overlays if you can.
Linux, on the other hand, will always see the hard drive as it actually is, 300 Gigs in this particular example. If you partition the hard drive with Linux, you can then format it at its full capacity under Windows (or Linux) even if the BIOS doesn't recognize its full capacity. The only limitation to this trick is you can't boot off the drive. The BIOS is what starts the computer booting, then hands off the process to the next boot device, normally a hard drive which runs a boot loader program to continue the booting process. If the BIOS can't see the hard drive correctly, it can't do the hand-off correctly. The drive can be used as a secondary (non-bootable) drive just fine though. This should be considered experimental though as it does violate one of the warnings below.
Warnings!
Rule number 1 of partitioning: Unless it's a brand new drive, always be sure you have a proper backup first. It's too easy to trash your system accidentally. Do not attempt to modify any partitions when rushed or not thinking clearly, especially if the partitions have data on them and you don't have good backups. Take your time and do it right.
Rule number 2: It is best to use the partitioning tool that's native to the operating system (OS) you are planning to use that partition with. Various partitioning tools have different assumptions built into them, and so do the OSs - especially Windows.
Windows tends to be fussy about partitions ending on cylinder boundaries, and seems to come up with its own ideas about what CHS geometry to simulate. This doesn't always correspond with what Linux or another OS would come up with. If you define the Windows partitions first, with Microsoft's tools, then the Linux tools are usually able to figure out what CHS geometry Windows has used and can work with that.
Primary partitions should always be created by the partitioning program of the OS that will use them. If you must mix Linux and Windows partitions, always create the Windows partitions first. Linux partitioning tools understand the partition tables produced by Windows, provided no "dynamic disks" are involved, but the reverse is not always true. In any case, Windows will only acknowledge the existence of one primary partition per hard drive, so the Windows partition drive has to be first.
Another good rule of thumb is to never mix Windows logical partition(s) with another OS's (i.e. Linux) logical partition(s) in the extended partition. Too many people have been burned too many times by Microsoft tools or third party Windows tools to have any faith that they won't eventually trash their "neighboring" logical partitions when performing some seemingly simple disk maintenance or alteration. In fact, MS FDISK has a known bug where it will trash NTFS logical partitions. There are reports that if the last logical partition in the extended is not a Windows type, Windows can sometimes see "phantom" partitions.
XP can be very picky about partitions being moved about, so get it the way you want it before you install XP. Finally, there are reports that Windows can have problems with partitions that are not sized with whole numbers of MB (Megabytes). The Windows tools only allow for whole MB sizes and won't allow you to create a partition with a fractional MB as part of the size (i.e. 1000.25 MB).
Win 98/ME tools
Windows 98/ME (and DOS and Win 95) come with only one partitioning tool, fdisk (hereafter referred to as MS FDISK). This tool supports only FAT (aka FAT16) and FAT32 file systems (for Windows 95 SR2 and later). It has been around for ages, and is really starting to show its age. It does not support partition resizing. It is also very destructive. All changes are immediate, and it randomly writes over sections of the partition that it claims it is only checking, effectively trashing your partition(s). This makes it dangerous to use on a hard drive with data you care about. The version of MS FDISK supplied with Windows 95 and 98 will not correctly report hard drives larger than 64 GB. An updated version, which will recognize up to 137 GB hard drives, is available from Microsoft (http://support.microsoft.com/kb/q263044/). No version of MS FDISK, even the one supplied with Windows ME, will correctly report the size of hard drives over 137 GB. MS FDISK has problems deleting non FAT and FAT32 partitions, forcing you to use other means to change your partition table. Sometimes the partitions you create with this tool don't stick (something your author has personally witnessed), and you have to create them a second time. This tool will automatically overwrite your master boot record when a partition has the bootable toggle set. This is because the DOS format command expects MS FDISK to clear the first 512 bytes of the data area of a partition whenever a size change occurs. This behavior can not be overridden, and is a bug in the program which will never be fixed.
Overall, MS FDISK isn't a good tool to use anymore. With Windows 98 and ME fading into the history books, there is little need to use it with the alternatives available today (see the Linux tools below for example). For those who still want to use it, the RADIFIED FDISK Guide (http://fdisk.radified.com/) provides an excellent tutorial.
Windows 2000/XP tools
There are two partitioning tools that come with Windows 2000 and XP, a semi-text mode partitioning tool in the installer, and the disk administrator tool. Only the semi-text mode tool is discussed (see Part III for the disk administrator tool). This tool is one of the nearly command line tools mentioned earlier. It is only accessible from the setup program. Like MS FDISK, there are only a very limited number of file systems supported: NTFS, FAT, and FAT32. Also like the MS FDISK program, it will overwrite your boot loader program without asking. The Windows 2000 and XP partitioning tool is a semi-graphic one
(see Figure 1; this was actually taken after Figure 10 and shows how Windows interprets Linux partitions). You can't use a mouse and must use the keyboard. To select a partition to work on, including the unpartitioned space, use the up/down arrow keys. For an existing partition, you choose to install or delete. If unpartitioned space is highlighted, you can create a new partition (see Figure 2).
If you attempt to install on an unknown partition (it is a choice given), Windows throws up an error screen (see Figure 3) and refuses to install.
If you attempt to install on unpartitioned space, Windows will automatically use all the space it can and create a new partition and proceed with the install. If you delete a partition, you must confirm the delete (see Figure 4) by pressing the letter L.
If the partition is a primary partition, you will get an additional warning screen before you get the confirmation screen. One thing that is missing from all these choices is whether you want a logical partition or primary partition. This is because Windows will only recognize one primary partition and one extended partition on any particular hard drive. It automatically creates a primary partition for the first partition you create and all the rest are logical partitions. It will also only install to the primary partition, totally ignoring whatever logical partition, if any, you select as the target of the install.
Linux tools
In the Linux category there are several tools to consider: the Linux fdisk, cfdisk, sfdisk, parted, and Ntfsresize, although only Linux fdisk and cfdisk will be covered in Part II. The various Linux install disks come with their own partitioning tools, some text based, but these won't be discussed further. Even if you have no intention of using Linux, you can still find some of the Linux tools can do things that can't be done with the free Windows tools. You don't have to install Linux to use them either. There are several live CDs that allow you access to the tools without ever installing Linux. Three such CDs are (there are others!):* Knoppix (http://www.knoppix.org/; click on the British/American flag for English), * System Rescue CD (http://www.sysresccd.org/Main_Page), * gparted-livecd-0.3.3 (http://gparted.sourceforge.net/). Note that all the Linux tools require root privileges to actually do anything. This is a safety feature. Finally, none of the Linux tools demand a reboot after using them, but a reboot is recommended.
Linux fdisk
Probably the oldest of the Linux partitioning tools (the first version came out around 1992), it shares the same name as the Microsoft tool (hence the prefix MS earlier), but is much, much, safer to use, more versatile, and up-to-date. While MS FDISK supports only three partition types (extended, FAT, and FAT32), and the Windows 2000 tools only support four partition types (extended, NTFS, FAT, and FAT32), fdisk supports more than 74 different types (type 83, Linux, several different file systems including EXT2, EXT3, and ReiserFS versions 3 and 4). Unlike MS FDISK, the Linux fdisk program (hereafter referred to as fdisk (Lin), although the actual command to invoke it is just "fdisk") won't actually change anything until you issue the write command. Only then are your partition table edits actually written. Even then, it does not trash your partitions as MS FDISK does , but merely changes the partition table. Your author has written changes to a partition table, then gone back in and immediately put the partition table back to its previous form without any data loss. One thing MS FDISK does do that fdisk (Lin) doesn't is overwrite your boot loader program. In fact, fdisk (Lin) has no way of touching your boot loader. Any changes to your boot loader program, including creating it in the first place, must be done by another program. To start fdisk (Lin), you must tell it what hard drive (and here this includes USB) you want to work on. In Linux, hard drives are always named /dev/hd* for IDE disks, /dev/sd* for SCSI disks, where * is a-h (normally, but it can go as high as a-t) for IDE, and a-p for SCSI). It also has support for ESDI and XT disks. For IDE drives, the primary master is /dev/hda (if in doubt, this is probably the correct hard drive for most computers), the primary slave is /dev/hdb, the secondary master is /dev/hdc, etc. For SCSI, the first device is a, the second b, etc. If the drive doesn't exist , fdisk (Lin) will immediately exit with the error "Unable to open ____" where the blank is the device name you gave. You also must be root (the user, not the partition) to use fdisk (Lin) or you will also get this error. You can use fdisk (Lin) to make changes on a hard drive that is in use, but the changes will not take affect until you reboot. If the hard drive is not in use, a reboot is not required. When you start fdisk (Lin) (for example: fdisk /dev/hda; see Figure 5),
you are presented with a very sparse screen menu. To get a help screen at almost any time (see Figure 6), you can type m (think m as in menu) and press enter.
All fdisk (Lin) commands require you press enter and, unlike most things in Linux, are not case-sensitive. Generally, one of the first things you want to do is type p and press enter to see the current partition table. Does it look like you think it should? For example, if you think you selected a brand new hard drive, and you see something like Figure 7, you know you have the wrong drive. Fdisk (Lin) also gives you the approximate size of the hard drive (Figure 7, second text line). Is this what you are expecting? If not, quit now (by typing q and enter) and no damage is done. Remember, unless you issue the write command, no changes actually occur. Figures 8 and 9 show the messages you would get on a brand new, totally blank hard drive. Looking at Figure 7 again, you will notice that all the partitions have a number after the drive letter, such as /dev/hda5.
Numbers 1-4 are reserved for the primary and extended partitions, while numbers 5 and up are logical partitions, even if some of the numbers 2-4 aren't taken by primary (or extended) partitions. A DOS partition table (the default for Windows and all Linux tools) has a limit of 60 logical partitions. Some fdisk (Lin) commands will ask for a partition number. This is where you get the number. Just type the number, and not the drive name part. If the Boot column has a star (*) in it, that means that is a bootable or active partition. This doesn't actually tell the BIOS where the boot loader program is. The BIOS loads the program in the master boot record. The master boot record program then scans for an active partition and loads the boot sector from the partition marked as active which in turn loads the operating system. Some boot loaders; such as GNU GRUB does not require any active partitions, while all versions of Windows up to at least XP do require an active partition. You should not have more than one partition active. Some systems will boot, but most will not. For DOS and Windows, the active partition is automatically assigned as the C drive, although the letter can be changed in Windows 2000 and up. Many disk partitioning programs will refuse to make an extended partition bootable, although fdisk (Lin) has no such problem. Some programs will also refuse to make a logical partition active too. Note that a DOS MBR (which includes Windows up to ME) only checks the bootable flag for primary partitions.The Start and End columns show which cylinders the partition is located, and the blocks column shows, albeit in a somewhat cryptic fashion, the size of the partition. In the case of Figure 7, these are 1KiB (i.e. 1,024 byte) units. The ID and System columns tell you what type of partition you have. The ID tells you the hex code, and the System column gives the name of that code. For most people, there are only a few commands in fdisk (Lin) that will be used regularly. The print the partition table command, the quit command, and the help/menu command have already been covered. To toggle the bootable flag, type the letter a and press enter. You are then asked for a partition number. If the flag is present, it will be removed. If it is not present, it will be added. Deleting a partition is as simple as typing the letter d and hitting enter. You are then asked for the partition number. There is no confirmation, but remember that there are no changes to the hard disk until you tell fdisk (Lin) to commit the changes. If you make a mistake, or think you made a mistake, just exit (via q and enter). If in doubt about being in doubt, quit and try again. To see a list of partition types supported, type l (that's a lower case L as in Linux) and press enter. This lists the hex code (aka ID) and the English name (aka System) for all the partitions supported by fdisk (Lin). This command doesn't allow you to actually change anything. To actually change the partition type, use the t command. You then enter the number of the partition you want to change, then the new hex code. You can see the list again by typing l instead of a hex code. The codes most users will be interested in are:c – FAT32 (LBA)7 – NTFS83 – Any type of Linux partition except swap82 – Linux swapIncluded in the listing of codes are two extended partition codes, 5 (Extended) and f (W95 Ext'd (LBA)). If you have used an MS tool to create the extended partition, you will likely see f rather than 5. Linux doesn't care which one is used, but.Windows9x/DOS does. If you use type 5, and have a logical partition you are using from Windows 9x/ME or DOS (this doesn't affect Linux), data on other partitions may (will) get corrupted because the logical partitions are addressed in CHS mode for type 5 by Windows, while for cylinder 1024 crossings, LBA addressing is required under Windows 9x/ME and DOS. To create a new partition, type the letter n and press enter. You are then asked if you want to create a new primary (p), or logical (l as in Linux) if you already have an extended partition, or extended (e) if one doesn't already exist. Then you are asked for a partition number (1-4) if the type is primary or extended. Fdisk (Lin) will not let you pick a number already in use. Logical partition numbers are automatically assigned and the way this happens confuses MS FDISK and some other DOS partitioning programs (i.e. MS FDISK will trash other partitions). Next you can choose the starting cylinder. Unless you really, really have a good reason for not doing so, take the default and just press enter. Finally, you are asked for the size. You can specify cylinder, or a size in megabytes or kilobytes. To enter a size in megabytes, type a +, then the number of megabytes followed by an M. If the value is too big, it will tell you "Value out of range" and let you try again. Finally, typing w (as in Write) and pressing enter will write your changes and exit fdisk (Lin). While it is far easier to recover from accidentally writing unintended changes with fdisk (Lin) than from most Microsoft and some third party disk partitioning software, it is far, far better to check twice first. Remember, the print command is your friend. For all its strengths, fdisk (Lin) does have some limitations. The biggest is you can't resize an existing partition and keep your data, although no other tool discussed in this article can do that either. The rest are pretty much covered under the Warnings section above. It is interesting to note that the documentation (aka man page) included with fdisk (Lin) has the following to say: “...There are several *fdisk programs around. Each has its problems and strengths. Try them in the order cfdisk, fdisk, sfdisk. Indeed, cfdisk is a beautiful program that has strict requirements on the partition tables it accepts, and produces high quality partition tables. Use it if you can. fdisk is a buggy program that does fuzzy things - usually it happens to produce reasonable results...”
Cfdisk
While cfdisk may look like a semi-graphical front-end to fdisk (Lin), it is not. It is actually a separate program and a separate code base. However, it does the same thing as fdisk (Lin), so everything said above about fdisk (Lin) applies equally to cfdisk unless contradicted here. Cfdisk is in many ways a much more user friendly program than fdisk (Lin) is. The display (see Figure 10, for example) gives more information and keeps it right there for you rather than constantly having to use the print command like fdisk (Lin).
It tells you the partition in meaningful units, and it limits the commands to those that are reasonable for the situation. While you can override this (see fdisk (Lin) above), you shouldn't. If you think you are smarter than cfdisk, you're probably not. Cfdisk is started the same way fdisk (Lin) is started. Type cfdisk and the drive you wish to modify. See fdisk (Lin) section above for help in specifying the drive. If you fail to specify a drive, it will use /dev/hda (primary master) by default or /dev/sda if no IDE drives are available. You must be the user root, of course, to use the program. Like fdisk (Lin), cfdisk will not actually make any changes until you issue the write command. All the commands may be entered by using the left and right arrow keys to highlight the desired command and pressing enter (see Figure 10; Bootable is the highlighted command), or using the first letter of command. The arrow keys wrap around, so if you are on the right-most command and press the right arrow again, the left-most command will be selected. The first letter shortcuts are not case-sensitive, except for the write command, which must be a capital W. In addition, the write command has a further yes/no prompt safeguard where you must type yes (not case-sensitive) to actually write your changes. Unlike fdisk (Lin), the write command does not exit the program. You must use the quit command to actually exit. To change which partition, or free space, you wish to modify, use the up/down arrows to change item highlighted. There are no keyboard shortcuts for this. You must use the up/down arrows to navigate and the up/down arrows do not wrap around. As you change partitions (or free space), you will notice the available commands at the bottom will change. Compare Figure 10 to Figure 11.
Obviously, creating a new partition (the new command), isn't applicable to an existing partition, while the delete command isn't applicable to free space. Unlike fdisk (Lin), the new command only gives you two choices (aside from cancel), primary or logical. It will never show you an extended partition, nor do you need to create one if one does not exist. If you select logical for any new partition, the extended partition is created automatically. After the first logical partition, you may continue to make more logical partitions as long as you have free space, or you may make another primary partition. Once you follow a logical partition with a primary, you can only make additional primary partitions. The free space type column will change to reflect that. Furthermore, there can only be up to four (4) primary partitions in total, including any extended partition as one of the primaries -- which, in fact, it is. Beyond that, no additional partitions can be made, and the free space label changes to unusable (see Figure 12).
Another difference between cfdisk and fdisk (Lin) is the print command. In cfdisk, the command gives a rather raw output of the partition table (see Figure 13).
Most people will have little need for the print command. The units command is another command that is used very rarely in cfdisk. This command toggles the display for all partitions, not just the one selected, between MB (megabytes), sectors, and cylinders. Only experts and masochists will ever use the units command intentionally. If you do use it accidentally, just use it again one or two times to get back to the friendly MB display. The maximize command seems like a tempting command to use, and it does make more disk space available, but it comes at a price. The cost of maximizing is that the partition becomes incompatible with DOS, OS/2 and possibly other operating systems. Do not use it unless you really know what you are doing. Even if you have no intention of using other operating systems to access the disk today, you never know about tomorrow. The type command does the same thing that fdisk (Lin)'s type command does, except it shows you the list before you are asked for the new code. The bootable command works exactly the same way as the fdisk (Lin) command, except you can't make the extended partition bootable, as it is not displayed. One command that is not displayed, and is only accessible by shortcut key is Control-L. Should the screen display ever get messed up, this will re-draw the screen. Cfdisk has several other advanced commands that aren't covered here. They are advanced and should not be used unless you really know what you are doing. Like fdisk (Lin), cfdisk can not resize partitions non-destructively. The label column in the display doesn't always get the partition names correct, but it has no effect whatsoever on the actual partitioning regardless of whether it gets it right or not. The label column is just an attempt to help you identify your existing partitions.
Conclusion
Creating partitions isn't very hard regardless of which tool you use. Creating partitions without trashing any existing data or other existing partitions is a bit more difficult, but once you understand the tools available and their limitations, it's not quite so hard. Just don't forget that the partitions must be formatted before they can be used (in all but a few cases). Part III of this series will cover a few of the many free graphical tools you could use to play with your partitions. See you then!
Robert Spotswood, a HAL-PC member, is active in the Linux SIG and a freelance computer professional. He can be reached at robert@spotswood-computer.net.
|