Disk conidence test while wiping

[Curious.George]

I've married an expert system to my disk wiping software. It looks at "data" from the current drive, coupled with historical data for previously encountered drives of that make/model (because we encounter drives in large, identical batches).

Initially, the AI gives me an assessment of whether or not the wipe is exhaustive or runs the risk of leaving data behind (which dictates physical destruction is required).

[I don't want to have a separate "disk test" activity -- esp as the disk will be hammered on more or less continuously during the wipe... problems SHOULD be pretty obvious!]

But, it is also intended to assess the drive's "future reliability prospects" -- i.e., should it be returned to service after it has been wiped or discarded.

I use lots of "macro" data as well as "micro" data in forming this decision. (e.g., macro: average wipe rate; micro: number of retries for each write operation).

Most operating systems hide all of the "micro" data from the application/user. E.g., an OS will repeatedly attempt to reissue a "write" until the drive processes it -- before resorting to complaining to the application that "write failed".

I'm looking to understand what I can reasonably expect from a "working" drive at this level of detail. E.g., why would a good drive ever "fail" a write operation... but succeed when it is retried? (perhaps if a sector remap operation was initiated within the drive as a result of the commanded write operation?)

I know I can't blindly rely on the "time" required for an operation to complete as I have to expect recals, remaps, autospin-ups, etc. to alter these figures...

[brethin]

Re: Disk conidence test while wiping

reinventing the wheel? plenty of good wiping software out there and unless you have DOD drives no need to do more than a full write 1 then write 0, takes 2 passes and done.

[Curious.George]

Re: Disk conidence test while wiping

reinventing the wheel? plenty of good wiping software out there and unless you have DOD drives no need to do more than a full write 1 then write 0, takes 2 passes and done.

Most software sits atop the OS -- so you don't actually see what is happening at the disk interface level (the OS tries to make operations work, not report potential "issues" with the device).

Depending on the OS/BIOS, you may not be able to remove the DCO & HPA (assuming the wiping software is even aware of them!)

Nor does the software have any smarts to determine WHICH drives are not worth the trouble to wipe and reuse (because the software doesn't track the past performance of drives).

[Hence the rationale for the questions in my post]

And, of course, that software is not likely to print QR/bar-coded labels for the drives and log them into "inventory" -- along with the data observed in the wiping process.

Finally, wiping MANY drives simultaneously requires a lot more "hardware" (MIPS) when you have the OS's bloat in the middle. E.g., try wiping 60 drives concurrently and let me know what your aggregate disk bandwidth is. Then, divide by number of spindles and discover just how much time was wasted waiting for the software (the DISKS should be the limiting factor).

[Curious.George]

Re: Disk conidence test while wiping

Finally assembling the hardware for this. Installed 7 disk shelfs, today (5 to wipe and 2 for RDBMS). Plus keyboard/monitor and a small switch.

Still to add: two servers (one to wipe disks, the other for the RDBMS), a pair of rack-mountable UPSs and firewall appliance. And, a "smart" PDU.

[ron350]

Re: Disk conidence test while wiping

CG when you get this working will the company or charity sell on ebay?
Thanks Ron

[Curious.George]

Re: Disk conidence test while wiping

CG when you get this working will the company or charity sell on ebay?
Thanks Ron

The initial provider (there are several other charities asking for copies of the machine) presently supplies computers to underprivileged clients. The computers are typically donated by businesses (periodic upgrade cycles where they replace every machine in their organization). One condition of the donation(s) is that the machines be purged of any data they may contain (even removing asset tags and other information stored in NVRAM). A client should never be able to tell where the machine came from. And, the donor should never have to worry that their "data" may be leaking out.

Wiping the disks also protects the CLIENT from any consequences of "residual data" that may be left on the machine from its previous owner/user.

Doing the disks individually is just too time consuming -- roughly an hour for a 250G-500G drive (determined largely by the write speed of the drive). My goal is to be able to wipe 120T per "shift" (i.e., 120 1T drives, or 240 500G drives, or 480 250G drives, or combinations thereof). This is a monotonous task that is easily prone to error ("Hmmm... did I wipe this drive, already? Or, was it THIS one? To know for sure, I'll have to READ the entire drive to verify its contents -- which takes just as long as WIPING it!")

This also avoids the problems inherent in naive/ad hoc wiping solutions (e.g., DCO and HPA). It can be configured to be as pedantic as we deem appropriate (e.g., discard drives with remapped sectors; follow-up verify pass, etc.). It also provides an indication of the "quality" of the drive by noting how well it performed during that process (how fast is it, does it throw any errors, etc.)

It incorporates a database that will, ultimately, let it also act to install disk images on selected drives (consulting the database for information on the image to apply for a particular "computer" as well as retrieving that image). So, the "wipe disk" step can be combined with the "install system" step to cut the time required to "build" a system (i.e., the system -- OS + apps -- is installed in the time it takes to wipe the disk!)

[The database then logs the disk and/or computer into "Inventory" so you can see what stuff you have on hand]

The goal of all this is to make it easy for an untrained user (volunteer) to wipe disks or build computers reliably and in very little time (if a volunteer only is present for 3 hours, you want to be able to get results from that time, not just "partial builds" -- which would be another opportunity for errors to creep into the process). This would make it easier to feed an eBay person with tested/completed systems for sale (to subsidize the "charity" aspect of the organization).

[brethin]

Re: Disk conidence test while wiping

so in that long ass post you still failed to answer his question, typical of your posts, you ramble on and on.☹️

[Curious.George]

Re: Disk conidence test while wiping

so in that long ass post you still failed to answer his question, typical of your posts, you ramble on and on.☹️

Dear illiterate moron,

Here is his question:

"...will the company or charity sell on ebay"

Now, if you got your head out of your ass and wiped the fecal matter out of your eyes, you'll note:

"The goal of all this is to make it easy for an untrained user (volunteer) to wipe disks or build computers reliably and in very little time (if a volunteer only is present for 3 hours, you want to be able to get results from that time, not just "partial builds" -- which would be another opportunity for errors to creep into the process). This would make it easier to feed an eBay person with tested/completed systems for sale (to subsidize the "charity" aspect of the organization)."

If my posts bother your tiny mind, why don't you add me to your IGNORE LIST. Perhaps one of your 8-year old neighbors could show you how to do that, eh?

Moron.

<plonk!>