Partitioning based on media transfer rate zones

[japlytic]

You may know this, but for over 25 years hard disk drive manufacturers made use of zoned recording (more sectors on the outer tracks and fewer on the inner tracks) to maximize media recording density, but the inner tracks are slower as a result.
For a piece of custom test equipment I am building (which would stream long patterns stored on the drive), a 500GB 2.5" drive was split into four partitions: the first 40% for 70-80 MB/sec, 20% for 60-70 MB/sec, 20% for 50-60 MB/sec and the rest for 40-50 MB/sec.

Have you considered such partitioning?

[stj]

Re: Partitioning based on media transfer rate zones

wont work,
modern drives re-map the data virtually.

if your after speed, get an "A/V" drive.
the firmware is written to keep the data on consecutive sectors to reduce seeks.

the downside to that aproach is you get heavily used areas on the platters that wear faster - hence the regular firmware scattering the data all over the platters.

[diif]

Re: Partitioning based on media transfer rate zones

I've setup a few short stroking hard rives back in the day but they were only two partitions.
Since SSDs are cheap and far faster, why bother ?

[eccerr0r]

Re: Partitioning based on media transfer rate zones

Yes this works but if you plan on using the whole disk equally, you're killing yourself back to square one. "Short stroking" is the name of the game but you waste a lot of disk space.

I did this decades ago with swap partitions to increase swap throughput. However if I needed to access the rest of the disk constantly, it would be a net wash or potential loss compared to sticking swap in the middle of the used area.

[RJARRRPCGP]

Re: Partitioning based on media transfer rate zones

That's what I do for RAID-0. (At least for the OS and games)

Need a bigger partition? Then just create another RAID array.

[Topcat]

Re: Partitioning based on media transfer rate zones

I've setup a few short stroking hard rives back in the day but they were only two partitions.
Since SSDs are cheap and far faster, why bother ?

For the amount of data on my NAS, SSD still isn't feasible.... to small and too expensive.

[RJARRRPCGP]

Re: Partitioning based on media transfer rate zones

For the amount of data on my NAS, SSD still isn't feasible.... to small and too expensive.

I got a Western Digital Black 1 TB spinner, just in case my Samsung TLC SSDs take a dump.

[Curious.George]

Re: Partitioning based on media transfer rate zones

For the amount of data on my NAS, SSD still isn't feasible.... to small and too expensive.

And, likely, offers more performance than you'd actually *need*!

At least in my case, my NAS is accessed much like my safe deposit box; occasionally fetch something and bring it home for normal use.

So, my NAS is "cold", most days/weeks. And, I have to pick which physical volumes to install in it prior to actually accessing any of that data (list of every volume's contents can be browsed in an on-line database).

[brethin]

Re: Partitioning based on media transfer rate zones

I run 3 NAS units, 2 are very active, 1 with movies 32 TB raid 6, 1 with music 8 TB raid 6 and 1 for backups 48TB raid 6.

[Topcat]

Re: Partitioning based on media transfer rate zones

I run 3 NAS units, 2 are very active, 1 with movies 32 TB raid 6, 1 with music 8 TB raid 6 and 1 for backups 48TB raid 6.

So is mine. This is my NAS setup. Each system pulls 280w off the grid. The second system in this NAS is a backup/mirror of the first one, and is off 99% of the time.....but I have considered ways to make it more efficient, such as ITX PC-on-a-chip setups, they don't require much raw horsepower. I run WD greens and WD reds, all 5400RPM spinners. THey run cool, efficient, and plenty fast enough for multiple HTPC's to stream from simultaneously....and well, of course store files. Each system has 2x 8TB RAID-10 arrays. They're close to capacity, but not crazy full....raw DVD ISO's are ~8gb each....then tunes....then work-related files.

[ratdude747]

Re: Partitioning based on media transfer rate zones

So is mine. This is my NAS setup. Each system pulls 280w off the grid. The second system in this NAS is a backup/mirror of the first one, and is off 99% of the time.....but I have considered ways to make it more efficient, such as ITX PC-on-a-chip setups, they don't require much raw horsepower. I run WD greens and WD reds, all 5400RPM spinners. THey run cool, efficient, and plenty fast enough for multiple HTPC's to stream from simultaneously....and well, of course store files. Each system has 2x 8TB RAID-10 arrays. They're close to capacity, but not crazy full....raw DVD ISO's are ~8gb each....then tunes....then work-related files.

I'm running a 14 drive 1.75gb RAID 10 with a 4tb weekly backup drive. Not terribly efficient but it fits in my rack, so I'm happy.

[Curious.George]

Re: Partitioning based on media transfer rate zones

I'm running a 14 drive 1.75gb RAID 10 with a 4tb weekly backup drive. Not terribly efficient but it fits in my rack, so I'm happy.

My NAS is "distributed" (many hosts on an internet) and operates at a much higher level of abstraction than a NAS "appliance". I.e., there are media that hold data, a VTOC that keeps track of "what's where" (and other metadata), interfaces that provide access (hardware and software) to the media (and, thus, the data) and rules that govern how data is replicated and "vouched for". Presently, about 100 spindles that can be powered up in various combinations on a score of hosts (so, HUGE processing and transport bandwidth).

Unlike an appliance, I place no constraints on the "availability" of data. The media holding it may be offline -- that doesn't make it corrupt, rather, just "having a longer access time"! Whenever a volume "appears", scanning (reverification) of its contents can resume -- as it is being otherwise accessed. If a particular volume is needed, the user is prompted to "mount it" (e.g., load a CD into a drive, somewhere that can be found).

I also don't care what type of media is used -- spinning rust, solid state (not just SSDs), tape (until recently), WORM (weening myself of this, presently), optical, etc. As long as there is an interface that can make that data available to the "system", it's OK. Interfaces can be IDE, SATA, SAS, SCSI, SCA, FC-AL, iSCSI, USB, 1394, CF, SD, SSD, etc. -- why should the NAS care?! It's just "storage"! (why do all of the physical volumes in the NAS have to share a common interface?)

I use an RDBMS as the VTOC -- as well as for tracking housekeeping data for the Array (i.e., when was the last time this datum was VERIFIED?).

And, the rules are just software -- algorithms.

These looser constraints give me more flexibility in what I can store, how I store it and how much effort I want to have expended (on my behalf) to preserve any particular datum ("file"). I can choose to store five different copies of an object ("file") in five different places ("containers") on five different media ("volumes") of five different physical characteristics ("media types") behind five different interfaces hosted by five different OS's. Yet, to *me*, the user, they are all functionally interchangeable! Give me ANY of those copies and I've got the data that I want/need!

[I.e., the CD/DVD originals from which I created each of the ISOs can be genuine parts of the NAS store -- even though they are read-only, removable, slow, small, etc!] If an ISO gets corrupted, the RDBMS tells me where to find another copy of the files IN the ISO that have been affected -- so the ISO (which is a container!) can be recreated. Ditto a RAR/ZIP archive, etc.

I can implement any variety of RAID (no striping) on any collection of files/objects just by building those desired relationships in the RDBMS that tracks all of the contents of the various volumes. And, at a granularity appropriate for the data involved. Volumes need not be the same size or have similar access characteristics (e.g., one half of a RAID 1 can be a CD while the other half is made of rust)

Because the RDBMS is available 24/7/365, I can browse the NAS's contents without having any of those media actually on-line. And, I can define the schema to let me annotate entries in any way that makes sense, to me (beyond just a "filename").

I can expand the array one volume at a time without having to "rebuild" anything. E.g., the 12 2T SAS drives I picked up this week magically increase the array's capacity without any special overhead -- just start copying files onto them like any other media!

Finally, I can deal with subsets of the array outside of its scope -- just move the drive to another machine and access it as a "normal" drive (e.g., a USB disk can be carried to a friend's home and used there; then reattached to the array as if it was never gone!). Because of this, I can recover/repair/access multiple volumes concurrently WITHOUT requiring special appliance hardware! (what happens when the power supply in your appliance shits the bed? or, the NIC fails?)

[Curious.George]

Re: Partitioning based on media transfer rate zones

Each system pulls 280w off the grid. The second system in this NAS is a backup/mirror of the first one, and is off 99% of the time.....but I have considered ways to make it more efficient, such as ITX PC-on-a-chip setups, they don't require much raw horsepower. I run WD greens and WD reds, all 5400RPM spinners. THey run cool, efficient, and plenty fast enough for multiple HTPC's to stream from simultaneously....and well, of course store files.

I've had bad luck with Greens (I don't even bother rescuing them, anymore).

What sort of sustained throughput can you get from them (or the reds)? I'm building a 50T datastore for the RDBMS in my "disk sanitizer" to store compressed disk images from which to mass-produce machines. I'm stuck trying to balance power consumption (it would be nice to only have the drives that were serving up images AT THE TIME actually spinning) vs throughput -- I want to be able to build multiple machines concurrently. I either have to actively manage power on the individual drives (i.e., spin down those that aren't needed at the moment) or opt for drives that will do this on their own (though they are usually lower performing).