Today, new BETAs are out for both StableBit DrivePool (1.2.0.6864) and the StableBit Scanner (2.2.0.2706 – for Windows Home Server 2011).
Available at: http://stablebit.com
In this post we’ll take a look at some new integration features between both products and we’ll look at 3rd party balancing plug-ins support in DrivePool.
Balancing in DrivePool
DrivePool 1.2 introduced a new balancing infrastructure that I described in the last post. For those of you unfamiliar as to where you would set that up, you would click here:
I should note that everything is set up properly out of the box and you really don’t need to delve into any of the settings that I’ll be describing shortly if you’re just interesting in the “it just works” user experience.
The more advanced functions that I’m going to describe here are for those of you who want a bit more control over DrivePool.
Balancing Settings
The new DrivePool BETA now features a Balancing Settings tab under the balancing window. Here you can configure when you want to allow the automatic balancing process to run and the conditions that trigger it.
You can also control whether balancers can force a balancing run at any time, regardless of the triggers that you set up. The StableBit Scanner makes use of this and we’ll talk about that shortly.
Reset to Default
If you’ve been experimenting with the new balancing settings or balancers and you think that you might have “messed something up”, don’t worry. Just click Reset everything to defaults and then click Save. Everything will be reset back to normal. This includes the balancing settings, the balancer order, whether each balancer is enabled or not and each balancer’s individual settings.
Basically, absolutely everything about balancing will be reset to the out of the box experience.
StableBit DrivePool + Scanner Integration
DrivePool 1.2.0.6864 now includes a new built-in balancer that provides integration features with the StableBit Scanner.
The new balancer is responsible for 2 things, File Evacuation and Temperature Control. Let’s talk a bit about how these work.
File Evacuation
As soon as any problems are detected with one of your drives by the StableBit Scanner, DrivePool will start moving your files out of the compromised drive to the other drives in the pool.
It does this using the existing balancing framework, which means that all your files will remain available on the pool during this process. Nothing needs to be unmounted for file evacuation to work.
You can choose whether to evacuate files as soon as an unreadable sector is detected, or when a S.M.A.R.T. warning is issued. You can also select whether to evacuate just un-duplicated files or all pooled files.
All file evacuation features are optional and can be enabled or disabled by the user at any time. By default we evacuate all pooled files as soon as the StableBit Scanner detects an unreadable sector on any one of your disks.
It’s worth noting that because duplicated files are already protected by the virtue of being available on multiple hard drives, file evacuation is not as critical for duplicated files. But by evacuating all of the pooled files from a compromised drive ahead of time, including duplicated files, you are not only protecting them from multiple consecutive drive failures but are facilitating faster and safer drive removal.
Evacuation Types
By default, the limit placed on the compromised drive by the StableBit Scanner balancer is a bit different than a limit that you would normally set using the File Placement Limiter.
When you limit file placement on a certain drive by using the built-in File Placement Limiter, the limit will only be respected if there is enough space on all the other drives in the pool to hold all of the limited files. If there isn’t, the limit will be broken and files will be allowed to go onto the limited drive in order to keep the pool functioning.
For example, if you have 2 1TB drives in the pool and there is 500GB of un-duplicated files on the pool, you can easily limit the storage of those files to only one of those drives. But if you copy another 1TB worth of data onto the pool, the system will break your limit instead of showing you an “out of disk space” message.
In DrivePool this is called a “soft limit”.
However, with the StableBit Scanner we assume that the drive’s integrity is compromised and that it’s not safe to store new data on the drive any longer. So we set a “hard limit” instead. In other words, even if all of the other drives in the pool are completely full, no new files will be placed on the compromised drive.
You can change this to a “soft limit” in the settings pictured above.
Note that if your pool doesn’t have enough free space to evacuate all the files from the compromised drive in the first place, DrivePool will have no choice but to leave the pooled files on that drive.
Temperature Control
Another aspect of the StableBit Scanner balancer is temperature control.
Using the StableBit Scanner, DrivePool monitors the drive temperature of each drive in the pool and dynamically hints to the real-time file placement algorithm where to place new files.
The limit will be respected as long as there are other cooler drives in the pool with sufficient free space.
This process can force files to be placed on a “non-optimal” disk, as determined by the balancers below it. In this case, the new files will be moved to the most optimal disk on the next re-balancing run.
Temperature control is disabled by default, but you can easily enable it if you like this feature.
An Immediate Balancer
The StableBit Scanner balancer is different than any other balancer written for DrivePool so far in that it forces a balancing run immediately if it needs to evacuate files, regardless of what your automatic balancing triggers are set to.
We do this because it is prudent to start moving your data off of any compromised drives as quickly as possible. This behavior can be controlled by the user in the aforementioned balancing settings tab.
DrivePool Balancing Plug-ins
Right now, DrivePool ships with 5 built-in balancers. But in addition to the built-in balancers, DrivePool can now accept 3rd party balancing plug-ins which can be installed at any time.
Today there are 2 balancing plug-ins available for download from our development wiki.
Get DrivePool balancing plug-ins here:
http://wiki.covecube.com/StableBit_DrivePool_-_Download_Balancing_Plugins
Let’s briefly talk about each plug-in and what it does.
Disk Space Equalizer
DrivePool comes with some sophisticated built-in balancing algorithms out of the box, but some people want something much simpler. They simply want all of their drives in the pool to be used equally.
Well, that’s exactly what the Disk Space Equalizer does. It can equalize either by the free space available on each disk or by the percentage of disk space used.
It’s a very simple plug-in and there’s really nothing more to it.
Archive Optimizer
This one is a bit more interesting. Here the idea is that you are using your pool to archive data and would like to optimize, or speed up, the process of placing new data into your archive.
The plug-in is designed around the idea that most of your high capacity disks, that are used for archival storage, are slow. While at the same time you have one or more faster disks that you would like to use for copying new data to the server.
In the Archive Optimizer plug-in you group your pooled disks into “Feeder” disks and “Archive” disks. All new data will be written to the fast feeder disks and the balancing system will move the data from the feeder disks to the much slower archive disks some time later.
You can set up a schedule to balance overnight, or when a certain amount of data needs to be moved in the overall balancing settings tab.
The obvious use for this is with SSDs.
You can add one or two SSDs to the pool and have those serve as “landing zones” for new files giving you super fast write times. You can even combine this with read striping for an even greater boost in performance.
Write Your Own Balancing Plug-in
Writing your own balancing plug-in is simple. All you need is Visual Studio 2010 and WiX 3.6 (for packaging).
To get started, visit the development wiki at:
http://wiki.covecube.com/StableBit_DrivePool_-_Develop_Balancing_Plugins
There’s a sample solution there with a step by step readme on how to get your plug-in set up and compiled into a .wssx installation file.
It’s really easy and you could have a plug-in written in a day.
What’s Next
There are a lot of BETAs out right now. Next we’ll start moving everything towards release final builds.
The DrivePool BETA may get an extended testing period due to multiple pool support, if that makes it into the current release cycle. If not, it will ship in the next version, which will be version 1.3.