I have backed up my system to an external ximeta drive using "dd" and the well-known linux live cd distribution, Knoppix to boot from. Below are the steps in brief:
-
Boot from the live cdrom distribution.
Switch to root.
Make sure NO partitions are mounted from the source hard drive.
Mount the external HD.
# mount -t vfat /dev/sda1 /mnt/sda1Backup the drive.
# dd if=/dev/hda conv=sync,noerror bs=64K | gzip -c > /mnt/sda1/hda.img.gz
"dd" is the command to make a bit-by-bit copy of "if=/dev/hda" as the "Input File" to "of=/mnt/sda1/hda.img.gz" as the "Output File". Everything from the partition will go into an "Output File" named "hda.img.gz". "conv=sync,noerror" tells dd that if it can't read a block due to a read error, then it should at least write something to its output of the correct length. Even if your hard disk exhibits no errors, remember that dd will read every single block, including any blocks which the OS avoids using because it has marked them as bad. "bs=64K" is the block size of 64x1024 Bytes. Using this large of block size speeds up the copying process. The output of dd is then piped through gzip to compress it.
To restore your system:# gunzip -c /mnt/sda1/hda.img.gz | dd of=/dev/hda conv=sync,noerror bs=64KStore extra information about the drive geometry necessary in order to interpret the partition table stored within the image. The most important of which is the cylinder size.
# fdisk -l /dev/hda > /mnt/sda1/hda_fdisk.info
Notes:
One of the disadvantages of the dd method over software specifically designed for the job such as Ghost or partimage is that dd will store the entire partition, including blocks not currently used to store files, whereas the likes of Ghost understand the filesystem and don't store these unallocated blocks. The overhead isn't too bad as long as you compress the image and the unallocated blocks have low entropy. In general this will not be the case because the emtpy blocks contain random junk from bygone files. To rectify this, it's best to blank all unused blocks before making the image. After doing that, the unallocated blocks will contain mostly zeros and will therefore compress down to almost nothing.
Mount the partition, then create a file of zeros which fills the entire disk, then delete it again.
# dd if=/dev/zero of=/tmp/delete.me bs=8M; rm delete.me
References:
A simpler, but more compressed way to back up your system
You can also use the cat command and pipe it through gzip with the "-9" option (for maximum compression) and then redirect the output to your image file:
cat /dev/sda | gzip -9 > hdbackup.img.gz
It works for me, but I have only been using Linux for less than a year (I like it better than windows now, witch I have been using all my life). So don't tell me if you have any problems with it...
-- A 13 year old anonymous Linux geek
Not working for me
I'm trying to restore an image back to the original disk, and unfortunately it's not working. It's a dual boot box, XP and Gentoo, and what I get after the restore is a bogus partition table according to my boot disk. I can see two partitions, the windows and extended like so:
*************************************************************************
fdisk -l /dev/sdc
Warning: invalid flag 0x0000 of partition table 5 will be corrected by w(rite)
Disk /dev/sdc: 160.0 GB, 160041885696 bytes
129 heads, 4 sectors/track, 605778 cylinders
Units = cylinders of 516 * 512 = 264192 bytes
Disk identifier: 0x8ff57ec4
Device Boot Start End Blocks Id System
/dev/sdc1 * 1 406425 104857648 7 HPFS/NTFS
/dev/sdc2 406426 605778 51433074 5 Extended
*********************************************************
Here's what I actually backed up:
rabbit scoutBackup # cat scoutHdFdisk.info
Disk /dev/sdc: 160.0 GB, 160041885696 bytes
129 heads, 4 sectors/track, 605778 cylinders
Units = cylinders of 516 * 512 = 264192 bytes
Disk identifier: 0x8ff57ec4
Device Boot Start End Blocks Id System
/dev/sdc1 * 1 406425 104857648 7 HPFS/NTFS
/dev/sdc2 406426 605778 51433074 5 Extended
/dev/sdc5 * 406426 406804 97780 83 Linux
/dev/sdc6 406805 414556 2000014 83 Linux
/dev/sdc7 414557 605778 49335274 83 Linux
rabbit scoutBackup #
Looks like it has something to do with restoring the extended partitions. Any tips on how to fix this?
I also tried manually
I also tried manually partitioning the hard drive just like my backed up fdisk -l info, and still no joy on booting up. I'm re-restoring it from the image, and I'll take a fresh stab at it.
/me sighs....
Got it
Restoring the image with this command worked for me:
gzip -cd /mnt/sda1/hda.img.gz | dd of=/dev/hda
good to know
Good to know you finally got it to work.
plz explain why giving bs=64K
hello , i am new to linux. i was woudering why we are using bs=64K not 4K (4096/1024) or somthing like that.
plz explain. thx for the article.
block size
Using large block size speeds up the copying process.
Great posts
It's rare to find great and useful posts. Thanks all!
Very informative post; also an alternative
Thanks for the reminder that you can use dd to do a disk backup / restore. Sometimes we forget that the simple low-level tools are still there.
One other alternative I wanted to mention is clonezilla (http://clonezilla.org/). The documentation is a bit lacking but it's a very powerful tool that duplicates and extends the norton ghost functionality. In the past I've set up a PXE-boot clonezilla config so you can remote-boot a machine and clone the OS directly onto it over the network without using any removable media.
Great Post
Just want to say this post was a great help, I also found another post at this address http://rob-ward.co.uk/blog/ddbackup.php that has some useful information that someone might find useful, it explains how to split images up to fit on a disk which is really nice and also gives examples of how to backup DVD's and stuff with it.
And again great post, saved me loads of time :-)
buffer problems?
the method for restoring the hdd in the article doesn't work for me.
# gunzip -c /mnt/sda1/hda.img.gz | dd of=/dev/hda conv=sync,noerror bs=64Kalthough my source image was correct, it wasn't transferred properly to the hdd. i inspected the two images, and realized that all the data was being transferred, but it was being regularly padded with a chunk of zeros every so often. Here's what I think is happening: either the pipe fills up, gzip runs too slow, or (since i'm transferring the image over the network) the transfer time is too slow. This means dd will sometimes get a chunk of data less than the 64K blocksize, and since sync is enabled, it will pad the rest of the 64K with zeros. bad!
i would recommend just leaving off sync and noerror (since they have to do with read errors, which you really shouldn't get because you're reading the image)
has anyone else run into this problem? what are your thoughts?
No space left on device
I just tried a backup and restore using your instructions. I was getting a "no space left on device" error on restore and found this to work instead:
# gzip -cd /mnt/sda1/hda.img.gz | dd of=/dev/hda
Thanks for the article, works great apart from that!
change made
That looked to be an error with the command option for gzip. Thanks, it has been corrected.
reconstructing partition images
If the sector size of partitions are known, you can extract partition images from the main image via:
# fdisk -l -u /dev/sdaOutputs as:
So to create an image of the second partition (sda2) from existing image of sda...
# dd if=/path/to/sda.img of=/tmp/sda2.img bs=512 skip=16771860 count=$[75489434-16771860]I just came across this
I just came across this tutorial today, and I noticed what I think is an error in the above post: shouldn't the "skip" parameter be set to Start-1? In other words, for the above example, it should be skip=$[16771860-1], because skip starts after the block you specify. Is there anyone who would confirm this?
sector count start at zero
sector count start at zero not at one, so the above should be fine.
Copying entire disk/partition to another drive
Copy /dev/sda to /dev/sdb:
# dd if=/dev/sda of=/dev/sdb conv=noerror,sync- /dev/sda: Source disk
- /dev/sdb: Target disk
- sync: Use synchronized I/O for data and metadata
- noerror: Continue copy operation after read errors
Notes linked from rignesnet.tzo.com
Just a trackback to Brain's Blog. I feel good that there are people referencing my notes and being of some help.
Backing up the MBR
This stores the first 512 bytes of the disk (contianing the MBR and the primary partition info - i.e. the first four primary entries) into the file "mbr.img".
To restore (be careful - this could destroy your existing partition table and with it access to all data on the disk):
If you only want to restore the actual MBR code and not the primary partition table entries, just restore the first 448 bytes of the MBR:
# dd if=/dev/hda
# dd if=/dev/hda conv=sync,noerror bs=64K | gzip -c > /mnt/sda1/hda.img.gzAlso I think that should be a lowercase 'k' in 'bs=64k', dd doesn't seem to like the capitalization.
> Also I think that should
> Also I think that should be a lowercase 'k' in 'bs=64k', dd doesn't seem to like the capitalization.
That's true for the BSD dd (at least the one that comes with Mac OS X 10.3). GNU dd, on the other hand, uses k=1000 and K=1024.
That's why it's a good idea to consult local man pages (or at least distribution-specific man pages).
- Chris
1k=1000bytes, 1K=1024bytes
1k=1000bytes
1K=1024bytes
Use factors of 512 to do your copying, since 512 bytes is the sector size used on your HDD.
1K != 1024 bytes
1k = 1000 unspecified units of measurement
1K = 1 Kelvin = 274.15°C
1kb = 1000 bits
1kB = 1024 bytes
In the last example,
In the last example, shouldn't "448" be "446" instead?
MBR sectors...
The 512byte MBR sector is actually two parts:
1. The first 446 bytes of the sector are the grub stage1 bootloader or the windows bootloader.
2. The last 64 bytes is where the partition table is stored.
So it does not really matter whether it is 448 or 446 since this area of 2 bytes contains a constant that doesn’t change.
MBR parts
The MBR consists of _three_ parts, not two:
1. The first 446 bytes contain the boot loader.
2. The next 64 bytes contain the partition table (4 entries, each 16 bytes).
3. The last 2 bytes contain the identifier 0xAA55 (or 0x55AA, I forgot).
To copy the boot loader only, one *must* use "bs=446". To copy the partition table only, one must use "bs=1 skip=446 count=64".
Post new comment