Can A Mechanical Hard Drive Really Be 'Securely Wiped'?

Page 1 of 2 [ 24 posts ]  Go to page 1, 2  Next

DeepHour
Veteran
Veteran

User avatar

Joined: 1 Jun 2014
Gender: Male
Posts: 78,192
Location: United Kingdom

07 Dec 2018, 3:34 am

As someone who's become a fairly hardcore computing geek over the past couple of years, I've bought a lot of old mechanical hdds to experiment with, installing operating systems on them, encrypting them, using them for storage, etc. They've become ridiculously cheap now that SSDs are taking over, and you can get a one or two year old 500GB laptop drive for £14 from CEX, or a 320GB desktop drive for £5.

These drives are supposed to have been wiped before sale, but it seems that it's often done pretty superficially. Scanning them with a utility like Piriform Recuva often reveals plenty of files from previous ownerships, inevitably including a fair bit of porn. Deleting with an overwriting program like that included with CCleaner usually seems to get rid of it, but I wonder....

As an experiment, I decided to delete the Windows operating system from my desktop machine (which has been in use for over four years), then overwrite the entire 1 Terabyte hard drive with zeros, using the Linux dd /dev/zero command. This took around 110 minutes to complete. I then used the dd /dev/urandom command to do a second overwrite with random ones and zeros, 'just to make sure'. That took an astonishing 28 hours.

Then it was scanned with Recuva, a formality one might have thought, but in fact it revealed the presence of seven large (from 1GB to 3.5GB) video files. I've no idea what these were (also don't recall having any 3.5GB video files), and couldn't get any video program to play them, but nonetheless they were stated to be intact and in good condition (indicated by a green marker). How could those survive such a heavy-duty overwrite?

Later performed another zeros overwrite, and this time there was apparently no sign of any leftover files, but who knows? Recuva is an entry-level free utility, and there are surely more powerful and sophisticated programs and processes available?

I've also wiped an msata solid state drive using Parted Magic, which uses a process called 'Secure Erase' to deliver some sort of electric shock to the drive rather than overwriting it. This appeared to work, or at least no files showed up in Recuva.

I'd very much appreciate any views on this. One would expect there to be some kind of consensus or definitive pronouncement about these matters after at least two or three decades of debate about it, but it seems there's nothing of the sort. Some people, including 'experts', insist that a single overwrite with zeros will render the data on any magnetic hard drive unrecoverable, some (eg Gutmann) say several (from 3 to 35 overwrites) are needed, and yet others maintain that there's no such thing as 'secure overwriting', and that nothing short of the physical destruction of the device will suffice.

:?


_________________
On a mountain range
I'm Doctor Strange


BTDT
Veteran
Veteran

User avatar

Joined: 26 Jul 2010
Age: 60
Gender: Female
Posts: 7,123

07 Dec 2018, 6:38 am

I'm not a disk expert but I know a lot about solving problems.

There can well be clues that an enable expert to recover data. Think of an old floor with grooves in the floor from use. Sanding will superficially get rid of them, except that floor is denser where the grooves used to be!

I'd expect similar wear issues, so with age, the locations of the 1s and 0s won't precisely align with were they used to be. Or maybe the size of each bit is slightly different due to drift in the electronic circuitry. Thus, you can't erase them completely!

But, that assumes that we can't go in and tweak the electronic circuitry. Or wire up new circuitry! Someone who could do that may be able to effectively erase a drive so that recovering the data may prove impossible for even the"experts."



Last edited by BTDT on 07 Dec 2018, 11:06 am, edited 1 time in total.

ImAnAspie
Veteran
Veteran

User avatar

Joined: 15 Oct 2013
Gender: Female
Posts: 7,686
Location: Erra (RA 03 45 12.5 Dec +24 28 02)

07 Dec 2018, 11:01 am

Yes. With a hammer!


_________________


Your Aspie score: 151 of 200
Your neurotypical (non-autistic) score: 60 of 200

Formally diagnosed in 2007.

Learn the simple joy of being satisfied with little, rather than always wanting more.



DeepHour
Veteran
Veteran

User avatar

Joined: 1 Jun 2014
Gender: Male
Posts: 78,192
Location: United Kingdom

07 Dec 2018, 11:27 am

^ I guess that places you in the third category of experts, then..... :wink:


_________________
On a mountain range
I'm Doctor Strange


jimmy m
Veteran
Veteran

User avatar

Joined: 30 Jun 2018
Age: 75
Gender: Male
Posts: 8,553
Location: Indiana

07 Dec 2018, 12:05 pm

I thought that maybe a strong magnet might erase the data. So I searched on the internet and sure enough someone tried that approach and it didn't work.

Hard Drive Destruction


_________________
Author of Practical Preparations for a Coronavirus Pandemic.
A very unique plan. As Dr. Paul Thompson wrote, "This is the very best paper on the virus I have ever seen."


Ichinin
Veteran
Veteran

User avatar

Joined: 3 Apr 2009
Gender: Male
Posts: 3,653
Location: A cold place with lots of blondes.

07 Dec 2018, 2:25 pm

When everything works as designed, yes it can be securely wiped.

Recuva is okay, there are better files for carving and recovering files that are "wiped", most of the better ones are in the data recovery, as supposed to forensics tools.

Secure erase is a built in feature in most modern drives that was created by the US Government because wiping drives were so insanely slow, it can be activated by various tools and is controlled by firmware on the controller card of the drive.

A Gutman wipe is not needed, it was "needed" earlier when disks were slow and used interleaving, if a disk had 1:7 interleave, you had to write it 7 times to make sure you got all sectors of the drive. This does not apply anymore since no modern drives use interleaving. Gutman had very little science backing his claims, and was more paranoia.

The case when data has been found involved faulty/failing wiper programs, forgotten partitions, user error and resized drives. It is never successful after one wipe. But it's your life and your time, if you want to wipe a drive 35 times, go ahead.

Wé did bring up the same subject on Forensicfocus:
https://www.forensicfocus.com/Forums/viewtopic/t=17056/

Scott Moulton (Data recovery/Forensics guy) did a series of data recovery where he showed how to use tools to recover data by basically telling the computer to ignore CRC errors and signals that said that the sector/cluster wasn't readable, in a court case he did recover an entire drive unlike law enforcement who used regular forensics tools that pad out sectors/clusters with 0x00 bytes when something goes wrong.

I cannot remember which video it was but it's somewhere on his channel:
https://www.youtube.com/user/SuperFlyFlippingA/videos

If it is someone who eats, breathes, lives and sleeps data recovery, it's Scott, and i dare you to find anyone better than him.


_________________
"It is far better to grasp the Universe as it really is than to persist in delusion, however satisfying and reassuring" (Carl Sagan)


kokopelli
Veteran
Veteran

User avatar

Joined: 27 Nov 2017
Gender: Male
Posts: 3,657
Location: amid the sunlight and the dust and the wind

11 Dec 2018, 4:35 pm

For a secure wipe:
1) erase it normally,
2) use a blow torch on it until it is red hot,
3) break it to pieces with a sledge hammer,
4) drop the pieces, one at a time, into the Mariana Trench as the boat moves along.

After all that, someone would really have to want it to try to recover it.



fiber bundle
Deinonychus
Deinonychus

User avatar

Joined: 20 Sep 2016
Age: 29
Gender: Male
Posts: 312
Location: USA

30 Dec 2018, 11:48 am

Even though subtle differences in the magnetization states of the "bits" in a digital magnetic storage medium compared to the ideal "0" and "1" states may encode much of the original data prior to a simple one-pass of zeroing, the technology to extract those data from the removed platters of a hard disk drive does not exist in practice.



Last edited by fiber bundle on 30 Dec 2018, 12:30 pm, edited 1 time in total.

fiber bundle
Deinonychus
Deinonychus

User avatar

Joined: 20 Sep 2016
Age: 29
Gender: Male
Posts: 312
Location: USA

30 Dec 2018, 12:29 pm

DeepHour wrote:
As an experiment, I decided to delete the Windows operating system from my desktop machine (which has been in use for over four years), then overwrite the entire 1 Terabyte hard drive with zeros, using the Linux dd /dev/zero command. This took around 110 minutes to complete. I then used the dd /dev/urandom command to do a second overwrite with random ones and zeros, 'just to make sure'. That took an astonishing 28 hours.

Then it was scanned with Recuva, a formality one might have thought, but in fact it revealed the presence of seven large (from 1GB to 3.5GB) video files. I've no idea what these were (also don't recall having any 3.5GB video files), and couldn't get any video program to play them, but nonetheless they were stated to be intact and in good condition (indicated by a green marker). How could those survive such a heavy-duty overwrite?


Your question is assuming that they were from the original filesystem whose partition was completely overwritten with zeros (then pseudorandom data), which is outright impossible.



kokopelli
Veteran
Veteran

User avatar

Joined: 27 Nov 2017
Gender: Male
Posts: 3,657
Location: amid the sunlight and the dust and the wind

30 Dec 2018, 8:13 pm

fiber bundle wrote:
Even though subtle differences in the magnetization states of the "bits" in a digital magnetic storage medium compared to the ideal "0" and "1" states may encode much of the original data prior to a simple one-pass of zeroing, the technology to extract those data from the removed platters of a hard disk drive does not exist in practice.


Supposedly, there is enough there to read some of the old data after it has been overwritten. There are reasons that securely overwriting the data requires several passes.

There are a couple of other problems in which one might think one has overwritten data when not having done so.

In many modern file systems, when one is "overwriting" the data you don't really know if the new data was written on top of the old data. It may be in a different location on a disk leaving the old data still there and discoverable.

Also, modern drives typically set aside some space to be mapped in for bad sectors. If a sector is found to be bad, the drive just maps in some of the reserved space and continues on leaving the old sector there. The sector might be bad, but that doesn't mean that it cannot be substantially read. If your hard drive is 5 years old, it could easily have data from any time in that five years that you thought you had deleted and have long forgotten about.



hale_bopp
Veteran
Veteran

User avatar

Joined: 2 Nov 2004
Gender: Female
Posts: 17,054
Location: None

30 Dec 2018, 10:49 pm

Yes, by grinding it to dust and throwing it into the ocean. :lol: :lol:



Ichinin
Veteran
Veteran

User avatar

Joined: 3 Apr 2009
Gender: Male
Posts: 3,653
Location: A cold place with lots of blondes.

31 Dec 2018, 2:56 am

kokopelli wrote:
There are reasons that securely overwriting the data requires several passes.


No there is not.

kokopelli wrote:
In many modern file systems, when one is "overwriting" the data you don't really know if the new data was written on top of the old data. It may be in a different location on a disk leaving the old data still there and discoverable.


Overwriting a drive does not work on a filesystem level. It is done on a track/cluster/sector or now on a flashram sized basis (128k and up).


_________________
"It is far better to grasp the Universe as it really is than to persist in delusion, however satisfying and reassuring" (Carl Sagan)


kokopelli
Veteran
Veteran

User avatar

Joined: 27 Nov 2017
Gender: Male
Posts: 3,657
Location: amid the sunlight and the dust and the wind

31 Dec 2018, 4:45 am

Ichinin wrote:
kokopelli wrote:
There are reasons that securely overwriting the data requires several passes.


No there is not.

kokopelli wrote:
In many modern file systems, when one is "overwriting" the data you don't really know if the new data was written on top of the old data. It may be in a different location on a disk leaving the old data still there and discoverable.


Overwriting a drive does not work on a filesystem level. It is done on a track/cluster/sector or now on a flashram sized basis (128k and up).


True, but most people I have heard of talk about overwriting their disk drives think all they have to do is delete and erase the files on it.

Some with a little more sophistication think that all it takes is to repartition the drive.

Very few are talking about actually writing over the entire hard drive, sector by sector. These are the rare exceptions, not the norm.

And even then it will still leave behind any sectors that were found to be bad and were remapped with other sectors.

And then there is Magnetic Force Scanning Tunneling Microscopy. Research has shown that it can read at least some previous data that has been written over. How much can be recovered is the real question. Someone would really have to have a serious need for the data and a lot of money and patience to even try this.

I've never tried to read someone else's used hard drive, but I have done it with an old 9-track magnetic tape. Years ago, a vendor sent me a magnetic tape with a single file of about a megabyte or two on it. There were three marks written to the 'end' of the data to tell the tape drive to stop reading. However, it was possible to read past it. So I tried it with the vendor's tape and found nearly their entire customer list for the state with business names, contact names, addresses, and telephone numbers. I didn't tell anyone about it and as far as I can remember, never even told anyone about it until now (this was 36 or 37 years ago).

By the way, on those 9-track magnetic tapes, there was a rather odd method that enabled one to read a magnetic tape with the naked eye! The density of data on the tapes were pretty low (a large reel wouldn't have more than about 180 or so megabytes of data) that you could apply "bit juice" directly to the tape and then read it with a magnifying glass.



Ichinin
Veteran
Veteran

User avatar

Joined: 3 Apr 2009
Gender: Male
Posts: 3,653
Location: A cold place with lots of blondes.

31 Dec 2018, 5:17 am

kokopelli wrote:
Very few are talking about actually writing over the entire hard drive, sector by sector. These are the rare exceptions, not the norm.


They are the norm if you work with security for real and isn't sitting at home talking about your home computer.

kokopelli wrote:
I've never tried to read someone else's used hard drive, but I have done it with an old 9-track magnetic tape.


Then you just disqualified yourself from this discussion. Your analogy to magnetic tape is not the same as HDD platters. Do some research into the subject before you utter your opinion. There are other factors like magnetic encoding, drive types, interleave (as i mentioned earlier), platter material, bios on the HDD controller that affect how data is written to the drive. This is more complex than you think.

I listed some of the situations when data has been found earlier:
Quote:
The case when data has been found involved faulty/failing wiper programs, forgotten partitions, user error and resized drives.


You do realize that i work in forensics and have been around much smarter people than me who do disk forensics and data recovery for years?


_________________
"It is far better to grasp the Universe as it really is than to persist in delusion, however satisfying and reassuring" (Carl Sagan)


kokopelli
Veteran
Veteran

User avatar

Joined: 27 Nov 2017
Gender: Male
Posts: 3,657
Location: amid the sunlight and the dust and the wind

31 Dec 2018, 5:38 am

Ichinin wrote:
kokopelli wrote:
I've never tried to read someone else's used hard drive, but I have done it with an old 9-track magnetic tape.


Then you just disqualified yourself from this discussion. Your analogy to magnetic tape is not the same as HDD platters. Do some research into the subject before you utter your opinion. There are other factors like magnetic encoding, drive types, interleave (as i mentioned earlier), platter material, bios on the HDD controller that affect how data is written to the drive. This is more complex than you think.


Actually, you just showed yourself to have very little reading comprehension. Or maybe you just like to misread things intentionally so as to make yourself seem like the smartest guy in the room. The mention of the old 9-track tapes was not intended as an analogy at all and it is incomprehensible that anyone could possibly think that it was an analogy. It was an aside that seemed to go along with this topic as a historical example of passing magnetic material around that had not been erased.

That said, the odds that someone other than a home user would bother to securely erase a hard drive, sector by sector, is probably not much different than that someone would rewrite an entire magnetic tape before using it to send data to another party. After all, hard drives are cheap enough that there is no reason to ever sell one if there is anything at all sensitive on it. Why go to all that effort to sell a hard drive that might be worth a few bucks wholesale and maybe $50 retail?



Ichinin
Veteran
Veteran

User avatar

Joined: 3 Apr 2009
Gender: Male
Posts: 3,653
Location: A cold place with lots of blondes.

31 Dec 2018, 5:52 am

Unlike you, i'm a professional with real experience in the field, not someone sitting at home examining tape, which you did use as an analogy.

Buh-bye.


_________________
"It is far better to grasp the Universe as it really is than to persist in delusion, however satisfying and reassuring" (Carl Sagan)