Dd

関連記事

• ディスクのクローン
• USB インストールメディア#dd を使う
• ベンチマーク#dd
• ディスクの完全消去#dd

dd はファイルの変換とコピーを主な目的とする コアユーティリティ です。

cp と同様にデフォルトでは dd はファイルのビットごとのコピーを作成しますが、低レベルの I/O フロー制御機能を備えています。

詳細は、dd(1) またはフルドキュメントを参照してください。

ヒント: デフォルトでは、dd はタスクが完了するまで何も出力しません。操作の進行状況を監視するには、コマンドに status=progress オプションを追加します。

警告: この種のコマンドはどれも、データを不可逆的に破壊する可能性があるため、dd の使用には細心の注意を払う必要があります。

インストール

dd は GNU coreutils の一部です。パッケージ内の他のユーティリティについては、Core utilities を参照してください。

ディスクのクローニングとリストア

dd コマンドはシンプルでありながら多機能で強力なツールです。ファイルシステムの種類や OS に関係なくコピー元からコピー先へブロック単位でコピーすることができます。ライブ CD のようなライブ環境から dd を使用するのが便利です。

警告: このタイプのコマンドと同様に使用時には十分な注意が必要です。データが破壊される可能性があります。入力ファイル (if=) と出力ファイル (of=) の順番を覚えておいて、逆にしないでください。出力先のドライブやパーティション (of=) のサイズが、入力元 (if=) と同じかそれ以上である事を常に確認してください。

パーティションの複製

物理ディスク /dev/sda のパーティション 1 から、物理ディスク /dev/sdb のパーティション 1 へ:

# dd if=/dev/sda1 of=/dev/sdb1 bs=64K conv=noerror,sync status=progress

ノート: 出力パーティション of= (例では sdb1) が存在しない場合、dd はこの名前のファイルを作成し、ルートファイルシステムをいっぱいにしてしまうので注意が必要です。

Cloning an entire hard disk

From physical disk /dev/sda to physical disk /dev/sdb:

# dd if=/dev/sda of=/dev/sdb bs=64K conv=noerror,sync status=progress

This will clone the entire drive, including the MBR (and therefore bootloader), all partitions, UUIDs, and data.

• bs= sets the block size. Defaults to 512 bytes, which is the "classic" block size for hard drives since the early 1980s, but is not the most convenient. Use a bigger value, 64K or 128K. Also, please read the warning below, because there is more to this than just "block sizes" -it also influences how read errors propagate. See [1] and [2] for details and to figure out the best bs value for your use case.
• noerror instructs dd to continue operation, ignoring all read errors. Default behavior for dd is to halt at any error.
• sync fills input blocks with zeroes if there were any read errors, so data offsets stay in sync.
• status=progress shows periodic transfer statistics which can be used to estimate when the operation may be complete.

ノート: The block size you specify influences how read errors are handled. Read below. For data recovery, use ddrescue.

The dd utility technically has an "input block size" (IBS) and an "output block size" (OBS). When you set bs, you effectively set both IBS and OBS. Normally, if your block size is, say, 1 MiB, dd will read 1024×1024 bytes and write as many bytes. But if a read error occurs, things will go wrong. Many people seem to think that dd will "fill up read errors with zeroes" if you use the noerror,sync options, but this is not what happens. dd will, according to documentation, fill up the OBS to IBS size after completing its read, which means adding zeroes at the end of the block. This means, for a disk, that effectively the whole 1 MiB would become messed up because of a single 512 byte read error in the beginning of the read: 12ERROR89 would become 128900000 instead of 120000089.

If you are positive that your disk does not contain any errors, you could proceed using a larger block size, which will increase the speed of your copying several fold. For example, changing bs from 512 to 64K changed copying speed from 35 MB/s to 120 MB/s on a simple Celeron 2.7 GHz system. But keep in mind that read errors on the source disk will end up as block errors on the destination disk, i.e. a single 512-byte read error will mess up the whole 64 KiB output block.

ヒント: If you would like to view dd progressing, use the status=progress option. See dd(1) for details.

ノート:

• To regain unique UUIDs of an ext2/3/4 filesystem, use tune2fs /dev/sdXY -U random on every partition. For swap partitions, use mkswap /dev/sdXY instead.
• Partition table changes from dd are not registered by the kernel. To notify of changes without rebooting, use a utility like partprobe (part of GNU Parted).

Backing up the partition table

See fdisk#Backup and restore partition table or gdisk#Backup and restore partition table.

Create disk image

Boot from a live medium and make sure no partitions are mounted from the source hard drive.

Then mount the external hard drive and backup the drive:

# dd if=/dev/sda conv=sync,noerror bs=64K | gzip -c  > /path/to/backup.img.gz

If necessary (e.g. when the resulting files will be stored on a FAT32 file system) split the disk image into multiple parts (see also split(1)):

# dd if=/dev/sda conv=sync,noerror bs=64K | gzip -c | split -a3 -b2G - /path/to/backup.img.gz

If there is not enough disk space locally, you may send the image through ssh:

# dd if=/dev/sda conv=sync,noerror bs=64K | gzip -c | ssh user@local dd of=backup.img.gz

Finally, save extra information about the drive geometry necessary in order to interpret the partition table stored within the image. The most important of which is the cylinder size.

# fdisk -l /dev/sda > /path/to/list_fdisk.info

ノート: You may wish to use a block size (bs=) that is equal to the amount of cache on the HD you are backing up. For example, bs=8192K works for an 8 MiB cache. The 64 KiB mentioned in this article is better than the default bs=512 bytes, but it will run faster with a larger bs=.

ヒント: gzip is only able to compress data using a single CPU core, which leads to a data throughput considerably lower than the write speeds on modern storage. In order to leverage multicore compression and create a disk image more quickly, one could for instance install the pigz package, and simply replace the gzip -c command above with pigz -c. For large disks, this can potentially save hours.

Restore system

To restore your system:

# gunzip -c /path/to/backup.img.gz | dd of=/dev/sda

When the image has been split, use the following instead:

# cat /path/to/backup.img.gz* | gunzip -c | dd of=/dev/sda

バイナリーファイルのパッチ適用

ファイルのオフセット 0x123AB を16進数列 FF C0 14 に置き換えたい場合は次のコマンドラインで実行できます:

# printf '\xff\xc0\x14' | dd seek=$((0x123AB)) conv=notrunc bs=1 of=/path/to/file

Backup and restore MBR

Before making changes to a disk, you may want to backup the partition table and partition scheme of the drive. You can also use a backup to copy the same partition layout to numerous drives.

The MBR is stored in the the first 512 bytes of the disk. It consists of 4 parts:

1. The first 440 bytes contain the bootstrap code (boot loader).
2. The next 6 bytes contain the disk signature.
3. The next 64 bytes contain the partition table (4 entries of 16 bytes each, one entry for each primary partition).
4. The last 2 bytes contain a boot signature.

To save the MBR as mbr_file.img:

# dd if=/dev/sdX of=/path/to/mbr_file.img bs=512 count=1

You can also extract the MBR from a full dd disk image:

# dd if=/path/to/disk.img of=/path/to/mbr_file.img bs=512 count=1

To restore (be careful, this destroys the existing partition table and with it access to all data on the disk):

# dd if=/path/to/mbr_file.img of=/dev/sdX bs=512 count=1

警告: Restoring the MBR with a mismatching partition table will make your data unreadable and nearly impossible to recover. If you simply need to reinstall the bootloader see their respective pages as they also employ the DOS compatibility region: GRUB or Syslinux.

If you only want to restore the boot loader, but not the primary partition table entries, just restore the first 440 bytes of the MBR:

# dd if=/path/to/mbr_file.img of=/dev/sdX bs=440 count=1

To restore only the partition table, one must use:

# dd if=/path/to/mbr_file.img of=/dev/sdX bs=1 skip=446 count=64

ブートローダーの削除

MBR ブートスラップコードを消去するには(別のオペレーティングシステムを完全に再インストールする必要がある場合に役立つ場合があります)、最初の440バイトのみをゼロにする必要があります。

# dd if=/dev/zero of=/dev/sdX bs=440 count=1

Troubleshooting

Partial read

Files created with dd can end up with a smaller size than requested if a full input block is not available and the read(2) system call returns early. This can happen when reading from a pipe(7) or when reading from /dev/random and there is not enough entropy[3], or from /dev/urandom when reading more than 32 MiB[4].

It is possible, but not guaranteed, that dd will warn you about the issue:

dd: warning: partial read (X bytes); suggest iflag=fullblock

The solution is to do as the warning says and add iflag=fullblock to the dd command. For example:

$ dd if=/dev/random of=bigsecret.img bs=1K count=1 iflag=fullblock
$ dd if=/dev/urandom of=bigsecret.img bs=40M count=1 iflag=fullblock

ノート: It is strongly recommended to always add the iflag=fullblock option to the dd command when the input file is /dev/random or /dev/urandom.

An alternative for /dev/urandom is to specify a block size smaller than 32 MiB, but a larger copy count. For example:

$ dd if=/dev/urandom of=bigsecret.img bs=1M count=40

When reading from a pipe, an alternative to iflag=fullblock is to limit bs to the PIPE_BUF constant value, defined in /usr/include/linux/limits.h [5]. For example:

$ cat input.img | dd of=output.img bs=4k count=100