Secure Deletion, File Shredding, on Windows

Perm url with updates: http://xahlee.org/mswin/file_shredding.html

Secure Deletion, File Shredding, on Windows

Xah Lee, 2010-03-28

This article tells you how to securely delete files on Windows Vista system, and current state of the art.

Recently i need to delete a folder, full of my online passwords, financial info, etc. As you know, removing to trash and empty trash won't work, because the data is still on disk.

Simple Method: Just Duplicate Files Till Full

In the past, when i don't have any disk management tools or don't want to use it, one way i make sure all my data on my disk are absolutely gone, is simply duplicating files until my disk is full.

Start by, copying the largest file you have (such as video file). You can just keep selecting all the files in the folder and keep duplicating. After a few times, your disk will fill up. Then, you start to duplicate a small file (say, 1 mega byte file). This is to make sure that all crannies in the disk sector are also filled. When the OS says disk full, repeat this again with even smaller file (say, 1k byte file).

This method, will at least erase all your files by over-writing them once. Your disk is probably still not clean if some FBI want to dig it, but good enough for selling your old computer.

The above method is simple, works ok, but will take you about 20 min manually. Right now, i simply wanted to remove a folder that's 40 megabytes, but i have some 200 giga bytes free space. Am not going to spend 20 min to manually copy files. Even so, it is quite questionable that tiny files (few k bytes) will be overwritten, because, modern OS, the file system, virtual file system, cache scheme, are quite complex. Simple methods as above just isn't reliable, time consuming, and have minor problems. (e.g. your OS might complain that disk is reaching full and impinging its virtual memory functionality and refuse to copy.)

So, i spend about 3 hours looking at file shredding today. Specifically, i need to find and choose a program that lets me shred files on Windows Vista.

Tools

After about 2 hours reading several articles, here's several trustworthy options i found:

At first, when i found the GNU's “shred” unix command line utility, and i thought i need look no further. Because i have cygwin installed on my Windows, and the program is there. I tried it, it works. However, reading its man page, quote:

CAUTION: Note that shred relies on a very important assumption: that the file system overwrites data in place. This is the traditional way to do things, but many modern file system designs do not satisfy this assumption. The following are examples of file systems on which shred is not effective, or is not guaranteed to be effective in all file system modes:

* log-structured or journaled file systems, such as those supplied with AIX and Solaris (and JFS, ReiserFS, XFS, Ext3, etc.)

These days, as far as i know, most file systems are journaled in one way or another. So i can't say what's the relevance of “shred” anymore. Out with “shred”.

Some other tools mentioned above also seems noteworthy, many are open source, but often years old. In the 10 min or so i spent reading about each, i can't make sure they still work in Microsoft Vista. (open source ones are especially bad like that)

File shredding tools are low level, so you better be sure, else they screw up your files or disk partition and it's gonna be a big headache. Also, of course, make sure you backup first.

In the end, i just settled on SDelete. It's from Mark Russinovich of Sysinternals fame, who also wrote Process Explorer (i highly recommend).

From the SDelete's home page, it start with this paragraph, which goes to show how complex file erasure are these days:

One feature of Windows NT/2000's (Win2K) C2-compliance is that it implements object reuse protection. This means that when an application allocates file space or virtual memory it is unable to view data that was previously stored in the resources Windows NT/2K allocates for it. Windows NT zero-fills memory and zeroes the sectors on disk where a file is placed before it presents either type of resource to an application. However, object reuse does not dictate that the space that a file occupies before it is deleted be zeroed. This is because Windows NT/2K is designed with the assumption that the operating system controls access to system resources. However, when the operating system is not active it is possible to use raw disk editors and recovery tools to view and recover data that the operating system has deallocated. Even when you encrypt files with Win2K's Encrypting File System (EFS), a file's original unencrypted file data is left on the disk after a new encrypted version of the file is created.

How To Use SDelete

Here's a example of how i used SDelete:

sdelete -p 1 -s "c:\Users\xah\Documents\my secrets\"
sdelete -p 1 -z -c "C:"

Note, that the second line too some 4 hours for a 200 giga bytes of free disk space. Though, the CPU usage is below 5%.

Here's its usage:

Usage: sdelete [-p passes] [-s] [-q] <file or directory>
sdelete [-p passes] [-z|-c] [drive letter]
-c  Zero free space (good for virtual disk optimization).
-p passes Specifies number of overwrite passes.
-s  Recurse subdirectories.
-q  Don't print errors (quiet).
-z  Cleanse free space.

Wipe Once Is Enough!

In file shredding articles i've read in the past decade, it is said, a million times, that to be absolutely safe, you must overwrite the file n times, n being like 3 or 10 or more. I can understand that the magnetic disk tech do leave traces of previous bits, but i never really understood how tracing previous bits are really possible at the physics level, or why writing that many times is necessary.

One surprising thing i learned in one of the article is that, one time is enough today! I find that convincing. (see article below)

Some quotes from Data remanence:

Daniel Feenberg, an economist at the private National Bureau of Economic Research, claims that the chances of overwritten data being recovered from a modern hard drive amount to "urban legend".[3]

As of November 2007, the United States Department of Defense considers overwriting acceptable for clearing magnetic media within the same security area/zone, but not as a sanitization method. Only degaussing or physical destruction is acceptable for the latter.[4]

On the other hand, according to the 2006 NIST Special Publication 800-88 (p. 7): "Studies have shown that most of today’s media can be effectively cleared by one overwrite" and "for ATA disk drives manufactured after 2001 (over 15 GB) the terms clearing and purging have converged."[1] An analysis by Wright et al. of recovery techniques, including magnetic force microscopy, also concludes that a single wipe is all that is required for modern drives. ...

  • Disk Wiping – One Pass is Enough (2009-03-17), By Max. Source
  • Can Intelligence Agencies Recover Overwritten Data? (2003-07-21), by Daniel Feenberg, National Bureau of Economic Research. Source
  • Schneier on Security: File Deletion (2009-09-10) by Bruce Schneier. Source schneier.com

WTF is “Data Sanitization”?

Many articles uses the term Sanitization instead of just secure shred, wipe, erasure. What the fuck is with that? In short, it's basically a term used in government orgs borrowed from paper documents. It doesn't mean much other than emphasizing the highest degree of data security.

No Mention of SDelete?

Another thing i find bugging is that in the many articles i read talking about file shredding tools, none mentioned SDelete.

In hindsight, i probably should've used BCWipe. They have a free trial version, and is highly recommended and used by security experts.

Popular posts from this blog

11 Years of Writing About Emacs

does md5 creates more randomness?

Google Code shutting down, future of ErgoEmacs