The latest BETA of StableBit DrivePool 2.0 adds some major new back-end features, improved read striping and a new Performance UI.
Get the latest BETA here: http://stablebit.com/DrivePool/Download
The Performance UI
The new performance UI lets you see exactly what’s happening on the pool, at a glance. It shows you the most important information in real time.
Of course, it works perfectly with the remote control feature, letting you connect to any other machine on your network that is running StableBit DrivePool, and see exactly how the pool is performing on that machine.
Let’s take a look at the new UI in detail, and what all the visual cues actually mean.
File Performance
The performance UI is accessible from the Horizontal UI (and not the Vertical UI). To get into the horizontal UI, simply stretch the window wide.
As you can see from the screenshot above, the performance UI is split up into two categories, File Performance and Pool Performance.
File performance is designed to show you the top 5 file operations on the pool, that are consuming the most resources.
It shows you the read and write speeds of each file operation.
But in addition to the speed, it shows you something that’s even more useful, the disk activity percentage.
In fact, the list shows you the top 5 file transfers on the pool, with the highest disk activity. The longer the bar, the more “busy” the disk is.
Let me explain what that means. Every hard disk or SSD can only read and write so fast. In addition drives have different performance characteristics. For example, SSDs can jump around between different parts of the disk (i.e. seek) faster than hard drives.
This means that the maximum speed that any drive can read or write at, depends highly on the type of drive it is, and what file operations are being performed on it.
In short, the disk activity bar shows you how busy that disk is, considering all those factors.
So reading this performance chart we see that:
- Disk I:\ is very busy, and DrivePool is reading from it at 96 KB /s
- Disk F:\ is not so busy and DrivePool is reading from it at 12.2 MB/s
DrivePool happens to be reading the same file from both disks, which is called read striping, but more on this later.
Pool Performance
On the left of the performance UI, you will see Pool Performance.
In pool performance, you can see:
- The total combined read and write speeds that are occurring on the pool at this time.
- Out of all the read / write operations, how many of those are utilizing Fast I/O.
- Out of all the read operations, how many are utilizing read striping, and what algorithm is being used to stripe the reads (more on this later).
- And finally, out of all the read operations, how many are being prioritized by Network I/O Boost.
Let’s go over the 3 vertical bars in detail, in order to understand exactly what they mean.
Fast I/O
The notion of Fast I/O is something that Microsoft coined for a technology that is designed to bypass the file system and retrieve your data directly from the Windows cache.
The reality of the situation, is that this is not the case. In fact, Fast I/O can and mostly does interact with the filesystem and disk (albeit in a roundabout way).
A better way to think about Fast I/O is I/O that requires no additional processing and bypasses some of the overhead that is imposed on regular I/O. For example, when DrivePool is performing any kind of complex manipulation of the I/O (such as read striping or I/O prioritization) it will bypass Fast I/O.
Saving every CPU cycle was important back in the heyday of Windows NT, because computers were slow and every CPU cycle counted. But this may not be as relevant today.
Read Striping
Read striping is a feature in DrivePool that is designed to improve the performance of reading duplicated files from the pool. In essence it can read from multiple disks at the same time in order to improve (or maintain) performance.
Read striping in DrivePool actually operates in 2 different modes:
- Bulk transfer – For sequential access of large files.
- Optimal disk – For streaming and random access of data.
A mode is chosen when a file is first opened on the pool, based on the hints that the application gives us.
DrivePool shows you exactly what each read striping algorithm is doing in real-time, by coloring the read striping bar with 3 different colors.
Here’s what these mean:
- Hold – During a bulk transfer, DrivePool has decided that it is best to hold on to the current disk and not switch over to another disk.
- Switch – During a bulk transfer, DrivePool has decided that it is best to switch to another disk while the current disk services its outstanding requests.
- Optimal – During random file access or media streaming, DrivePool has chosen the optimal disk based on outstanding I/O requests, and an adaptive algorithm designed to pick the fastest disk at this time (this is new).
Bulk Transfers
This is what a typical bulk transfer looks like.
Here we can see that DrivePool is reading from 2 disks at the same time (I:\ and E:\), and that 100% of the read requests are being processed by the read striping algorithm. Most I/O requests are going to the same disk, and the disk context is being switched very rarely. In addition, we can see that both disks are saturated (this is exactly what we want).
Media Streaming (or Random Access)
This part is brand new and very cool.
Let’s say for example that you’re streaming a 50 GB uncompressed Blu-Ray that is in a duplicated folder, with read striping enabled.
So far, everything is going according to plan. Read striping has switched into streaming mode and your data is being read from 2 disks at the same time (I:\ and F:\), and both disks are not busy at all.
Now, let’s say some process starts on disk I:\, something that completely saturates the disk. Normally at this point your media would start to stutter very badly. And because the media player is reading the data rather slowly, the standard bulk mode read striping algorithm would have no outstanding I/O to lock onto, in order to tell it to switch disks.
Well, here’s what the new system does in this case.
As you can see, we’re still streaming at a very good 5.75 MB /s and more importantly, DrivePool has all but stopped reading from I:\ because it’s very busy. Your media keeps playing without stuttering and you would never know that anything happened at all.
This functionality is brand new in this build of DrivePool and works by passively monitoring the speed at which each disk is able to read at, and at the first sign of a slow down, it will prefer to read from the faster disk.
I/O Boost
The last bar under the pool performance UI is I/O boost.
This feature is disabled by default, but can be enabled in pool options.
Starting with Windows Vista, Microsoft has added the notion of I/O priority to Microsoft Windows. I/O calls that are made with a higher priority are always serviced before any I/O calls made with priorities lower.
This is very important, because the slowest component of your computer is typically your hard disk, and prioritizing requests to the hard disk has a great potential to speed up your everyday computing experience.
DrivePool utilizes this functionality by boosting the priority on I/O request packets coming from the network. We only do this for reads and not for writes. DrivePool only does this for requests that are not being serviced by the read-ahead cache (which is typically the case, unless the disk is very busy).
The main goal of this feature is to reduce stuttering on streaming video, but it can also be used to improve the network I/O throughput on busy servers.
Performance Impact of the Performance UI
You may be wondering, with all this data collection going on and being processed, isn’t this going to slow down the pool?
Well, not really. The entire performance monitoring system shuts down 60 seconds after the last UI window has closed. So it’s only processing the performance data when you need to see it.
Technical Notes
Finally, I’d like to end by pointing out some technical differences between how Pool Performance and File Performance collect their performance data.
Pool Performance collects the data from the pool’s point of view. In other words this includes everything that goes on and comes off of the pool. While File Performance records everything that goes on and comes off of each disk part of the pool.
This is an important distinction because, for example, files that are cached will not show up under File Performance while Pool Performance will show activity on the pool. This is because those files are not being read from any disk, but rather from memory and so there really is no disk activity to show.
Conclusion
This is going to be the last major addition to DrivePool 2.0 BETA. Next, the existing features will be polished up and DrivePool 2.0 will ship as a final.