Create a block-level backup with Linux for Data Recovery

When we get a data recovery job, the first thing that should be done before attempting any repair or recovery is to create a block-level backup of the disk using Linux and ddrescue.

This process is dangerous

The commands in this article have the possibility of completely destroying data on the customer's drive or on our backup server. There is no 100% safe way to perform this process - it cannot be "idiot-proofed". Therefore, be 1000% sure before you enter a command that you've typed it correctly.

Requirements

A Linux machine or a live boot Linux environment via DVD, USB, or network (PXE)
Some type of storage medium that needs recovery (external USB drive, internal HDD or SDD, etc)
Thoroughness and an attention to detail so you don't destroy the customer's data by accident

While this method will work on any storage medium that may come into the shop, you will have particular difficulty performing it on Macs with non-removable storage because booting them into Linux is either impossible or difficult enough to not make the process worth it. For those Macs, exhaust all other data recovery options but don't bother attempting this method as it will not work and will be a big waste of time.

Why do this?

Any attempts to recover from or repair the original drive can irreversibly damage the original drive. You maximize your chance of recovering data successfully if you create a block-level backup first, and then attempt to repair or recover from the image you've made.

What is a block-level backup

A "block" is a unit of storage. All traditional storage mediums (hard drive, SSD, flash drive, CDs, DVDs, etc) use blocks for storing data. You can think of them like storage bins for organizing toys - each storage bin can hold a fixed volume of toys, and the storage bins exist whether or not there are toys in them.

In this context, there are two types of backups:

Sparse backups
Block-level (non-sparse) backups

Sparse backups respect what the disk "knows" - if the disk knows that only 52GB of its 1TB capacity is used, then the file size of a sparse backup will be 52GB. The only information in a sparse backup is information that "exists" according to the disk. Most backup programs you are familiar with (Carbon Copy Cloner or CCC, Acronis, EaseUS) create sparse backups by default.

Block-level backups, on the other hand, don't care about what the disk "knows" or if it "knows" anything at all. When performing a block-level backup on a 1TB disk, the block-level backup file will be 1TB.

Additionally, sparse backups only work when the disk is in good enough condition that its partition(s) are available and not corrupted. Sparse backups cannot be done when there is significant corruption or if the partitions have been deleted.

Block-level backups, on the other hand, just care about backing up the blocks of a drive, regardless of what's there.

To return to the analogy - a sparse backup would be like taking inventory of all the toys in the bin by reading from a list of what toys are in the bins. A block-level backup would then be similar to looking inside each bin yourself and taking note of what you find.

Perform the backup

Get into a Linux environment

For removable internal drives, remove it and connect it to a shop Linux machine
For external drives, connect it to a shop Linux machine
For non-removable internal drives, boot the machine to a live CD version of Linux
For non-removable internal Mac drives, cry in despair and move on to other recovery options

When in the Linux environment, open a Terminal and run ddrescue --version to see if it's installed. You will either get some output about the version of ddrescue or you will get a message saying that the command ddrescue doesn't exist.

If it doesn't exist, install it via:

Ubuntu/Lubuntu/etc: sudo apt-get install gddrescue
CentOS/Debian: sudo yum install ddrescue

Run ddrescue --version again to verify it's installed. You may need to close and reopen the terminal window.

It is important to understand why and how ddrescue functions. By understanding this, you are able to accurately and efficiently troubleshoot issues that may arise during a recovery attempt. The ddrescue manual can be found here

Mount the backup NAS shares

Linux requires manually mounting the PC Backup or Mac Backup share for it to be accessed. On shop machines, these should already be mounted in one or more folders located at /media/upcc/. In a Live CD environment, these will not be mounted.

On shop Linux machines, assume that the folders are mounted already and skip to step #4 to verify. Otherwise:

Install prerequisites
- Ubuntu: sudo apt-get install cifs-utils ntfs-3g hfsplus hfsprogs testdisk gddrescue
- CentOS: sudo yum install cifs-utils ntfs-3g hfsplus hfsprogs testdisk ddrescue
Create a destination folder
- sudo mkdir /media/upcc
- sudo mkdir /media/upcc/pcbackup
- sudo mkdir /media/upcc/macbackup
Mount the respective shares
- sudo mount -t cifs -o rw,user=USERNAME,pass=PASSWORD //IP/SHARE /media/upcc/FOLDER
- Replace USERNAME and PASSWORD with the username and password that you normally use to access the NAS share elsewhere
- Replace IP with the IP Address of the NAS
- Replace SHARE with the share name on that NAS (i.e. the name of the folder such as "PC-Backups")
- Replace FOLDER wit the folder name you created in the previous step ("pcbackup" or "macbackup")
Verify that the shares are mounted correctly
- ls /media/upcc/FOLDER
- Replace FOLDER with the name of the folder you just mounted into ("pcbackup" or "macbackup")
- You should see a listing of all the current backups in these folders. If you know that the folder on the NAS is empty, then put a dummy file in the folder on the NAS before checking.
- If the folder is empty, then the share wasn't mounted correctly - try again.

Locate the problem disk

Run gnome-disks or use the GUI start menu to search for "Disks"
If it doesn't exist, run sudo apt-get install gnome-disk-utility on Ubuntu or sudo yum install gnome-disk-utility on CentOS

Use the GUI to find the problem disk. Match important details from the label on the disk to the listing in the Disks program:

Manufacturer
Drive size
Serial Number

If you're unsure:

Disconnect the problem disk from the Linux machine
Take note of the disks that are still present in the list
Reconnect the problem disk - the only new entry in the Disks program will be the problem disk

DO NOT PROCEED BEYOND THIS POINT IF YOU'RE NOT 1000% SURE YOU HAVE THE RIGHT DISK

Take note of the device identifier, in the form of /dev/sdX#. The X represents the disk as a whole, and the # represents partitions on the disk. We are only concerned with the disk as a whole, so the device identifier you care about is just /dev/sdX where X is a letter from a-z.

You will use this device identifier in later commands, and it must be exactly correct.

Verify the problem disk is recoverable

The final step before attempting recover is to verify that it is even recoverable at all. The only two criteria for this are:

The disk is visible in the Disks program on Linux
In the Disks program, the total disk capacity is reported for the disk

If either of these criteria are not met, the device is not recoverable by us and should be sent for professional data recovery.

Begin the recovery

The template for running ddrescue is as follows:

ddrescue /dev/sdX /media/upcc/FOLDER/SUBFOLDER/BACKUP.dd /media/upcc/FOLDER/SUBFOLDER/BACKUP.log

In the template, replace the following:

/dev/sdX with the device identifier you found earlier - be 1000% sure you have this right
FOLDER with the local folder name ("pcbackup" or "macbackup") where the share is mounted
SUBFOLDER with this customer's subfolder on the share (see below)
BACKUP with the name of the backup for this customer (see below)

Customer subfolder and backup name

When creating backups using any procedure, we always create a subfolder in the respective PC or Mac Backup share in the form of #####Name where ##### is the PCID and Name is the last name of the customer. This folder may already exist from an earlier backup attempt, or you may need to create it now. To create it on Linux after making sure the share is mounted properly:

sudo mkdir /media/upcc/FOLDER/SUBFOLDER

Replace FOLDER with the local folder name ("pcbackup" or "macbackup") and SUBFOLDER with the subfolder naming format as above. For a customer with PCID 13690, Windows computer, and last name of Strickland, this command would be:

sudo mkdir /media/upcc/pcbackup/13690strickland

The backup name follows the same scheme as the folder. In the example above:

The backup would be at media/upcc/pcbackup/13690strickland/13690strickland.dd and the log file for the backup would be at /media/upcc/pcbackup/13690strickland/13690strickland.log

Monitor progress

If all went well, the ddrescue program begins to run and report its progress in the Terminal. As an example:

Press Ctrl-C to interrupt
     ipos:   47775 kB, non-trimmed:        0 B,  current rate:  25690 kB/s
     opos:   47775 kB, non-scraped:        0 B,  average rate:  15925 kB/s
non-tried:    4000 GB,  bad-sector:        0 B,    error rate:       0 B/s
  rescued:   47775 kB,   bad areas:        0,        run time:          2s
pct rescued:    0.00%, read errors:        0,  remaining time:  2d 21h 46m
                              time since last successful read:         n/a
Copying non-tried blocks... Pass 1 (forwards)

Fields of note:

pct rescued is how much of the disk has been backed up
read errors are how many times the program failed to read from the disk
current rate and average rate represent how quickly the backup is proceeding
remaining time is the estimated time remaining

You can use those fields to get a rough indication of how long a backup will take and if it will be successful. A high number of read errors (1000+) indicate a high chance of failure. An astronomical number of failures (100k+) means almost guaranteed failure.

The remaining time can fluctuate wildly but is generally reliable, especially if you check it a few times over the backup to get a feel for how it's progressing. If 3 hours ago the remaining time said 4 hours, and now it says 1 hour, then you can be reasonably sure that the time is accurate. If, however, 3 hours again the remaining time said 4 hours and now it says 4 hours still, then this isn't a good indicator of time remaining and you should instead focus more on the average rate and pct rescued to get an approximate time.

This process is interruptable

ddrescue is designed to be paused and interrupted. If, for example, the computer crashes or the disk disconnects, you can resume the backup from where it left off by running the exact same command again.

Completion

When the backup is finished, the pct rescued should show 100% or very close to it. You can then attempt all recovery options on the image instead of the original drive.

Mounting the image

Linux

sudo mkdir /media/upcc/mnt
sudo losetup --partscan --find --show /media/upcc/FOLDER/SUBFOLDER/BACKUP.dd
- Replace FOLDER, SUBFOLDER and BACKUP as described in the "Customer subfolder" section above
sudo mount /dev/loopX /media/upcc/mnt
- Replace dev/loopX with the device identifer that was returned as a result of the losetup command you just ran
The folder will now be accessible at /media/upcc/mnt

Mac

Open Finder and navigate to where you saved the DD image on the network, then double click the DD image to mount it.
- Alternatively, connect to the backup share through Finder then run sudo hdiutil attach -imagekey diskimage-class=CRawDiskImage -nomount /path/to/nas/FOLDER/SUBFOLDER/BACKUP.dd
  - Replace /path/to/nas with the path to the network share, which is normally something like /Volumes/Mac\ Backups. Use the autocomplete feature of the terminal (hit Tab after typing a few letters of each folder) to assist.
  - Replace FOLDER, SUBFOLDER and BACKUP as described in the "Customer subfolder" section above
The disk will be mountable now as you would any other disk. Open Disk Utility and either mount it directly, or to unlock it (Filevault) and then mount it

Windows

Download OSFMount
Use the GUI to find and mount the DD image from the NAS share