The next testing milestone of StableBit CloudDrive is here and it’s version 18.104.22.1687 BETA.
Get it here: http://stablebit.com/CloudDrive/Download
Automatic updates should be going out to older versions within the next day or so.
As you can probably tell from the large jump in the version number (from 22.214.171.1243), a lot of changes have gone into this new BETA, and a lot of testing has gone into it as well. While there are some new features in this version, first and foremost, this version is primarily focused on fixing bugs and improving stability. Writing some new tests was also a big priority for this BETA, and this version has passed a number of important tests, including new data consistency tests and power failure tests.
If you’d like to see the details, the full change log with all of the fixes is available here, as always: http://stablebit.com/CloudDrive/ChangeLog
But aside from the fixes, here’s a summary of the major new features in this version (as compared to 126.96.36.1993):
- New cache types (expandable, fixed, proportional).
- ReFS support for cloud drives (Windows 8 and newer).
- FTP / FTPS / SFTP provider.
About Backwards Compatibility
While version 188.8.131.527 is fully backwards compatible with 184.108.40.2063, you should know that some under-the-hood features may not be enabled on drives created with a version prior to 220.127.116.117. For example, always-on encryption and file ID optimizations will not be enabled if you’ve created your drive with version 18.104.22.1683. This is mostly a technical distinction, and functionally, any drives created with version 22.214.171.1243 or older should continue to work in version 126.96.36.1997.
Now let’s dive into the new cache types.
New Cache Types
StableBit CloudDrive now supports a new setting that will let you specify the cache type to use for your new cloud drive.
3 cache types are now supported:
- Expandable (default)
Let’s see how each one works.
Let’s begin by imagining that you have a 500 GB volume that you would like to use for your cloud drive’s cache, and that it already has 200 GB worth of files on it.
Your existing files on that drive won’t be affected by the StableBit CloudDrive cache.
Let’s talk about the fixed cache type first because this is the easiest one to understand. Fixed simply means that the on-disk cache will strive to never exceed the preset cache size. So whatever you set the cache size to, that’s the maximum amount of local disk space that it will consume.
Let’s imagine that you created a 100 GB cache on your 500 GB drive:
As you use your cloud drive, StableBit CloudDrive will learn which data on your cloud drive is accessed most frequently and it will automatically cache that data locally in the on-disk cache for faster access.
The fixed cache is very simple, but it has some disadvantages. Let’s see what happens when you copy some new data onto the cloud drive:
As you write new files onto your cloud drive, StableBit CloudDrive will store the newly written data in the local cache and queue it up for upload. By writing the newly copied data directly to the cache, StableBit CloudDrive ensures that the file copy operation to your cloud drive completes as quickly as possible.
Once uploading completes, the data that was just uploaded remains in the cache:
As you may have noticed, by simply writing data to the cloud drive, you have overwritten some of the learned portion of the cache (the adaptive part of the cache that holds the most frequently accessed cloud data).
Let’s see what happens when you try to copy some more data to a cloud drive that is using a fixed cache:
As you can see, the entire learned portion of the cache has now been overwritten with data that needs to be uploaded:
Once uploading completes, the cache has now lost all of its learned data, and it must relearn and re-download the data that is accessed most frequently.
Fixed cache advantages:
- Has a predictable fixed size.
- Maximizes write speeds by utilizing the entire size of the fixed cache.
- Writing to the cloud drive will overwrite any learned data in the cache.
Overall, the fixed cache is optimized for accessing recently written data over the most frequently accessed data. If that’s what you’re looking for then the fixed cache is perfect for that. But for a more balanced approach, let’s take a look at the proportional cache type.
The proportional cache type is similar to the fixed cache in that it also has a fixed size. But in addition to the size of the cache, a proportional cache allows you to define how much of the cache should be used to store data that needs to be uploaded versus data that is learned.
Once that proportion is defined, when you write new data to the cloud drive, only a part of the cache will be used to speed up the writes, while the other part will always be used to store the most frequently accessed data.
Once uploading is complete, one part of the cache is used to store learned data and the other part will contain new data.
The proportional threshold ensures that newly written data never overwrites the learned data.
Proportional cache advantages:
- Has a predictable fixed size.
- Does not overwrite the entire portion of the learned cache with newly written data.
- This cache type is the least optimized for writes to the cloud drive.
Next, let’s look at the most optimal cache type, the expandable cache.
The expandable cache is optimized for the fastest reads and writes to and from the cloud drive. However, unlike the fixed and the proportional cache types, the expandable cache does not have a predictable fixed size.
When you write data to a cloud drive with an expandable cache, those writes will expand the cache’s size past its set limit. While the cache will expand as new data is written to the cloud drive, it will never consume all of the free space that is available on the volume. It will always maintain some free space as a buffer at all times by throttling the writes to the cloud drive when necessary.
This ensures that the newly written data never overwrites any previously learned data, while at the same time, it optimized the writes to the cloud drive by utilizing most of the free space that is available on the cache volume.
Once uploading completes, the cache shrinks back down to its preset size and the previously learned data is not affected.
Advantages of the expandable cache:
- Optimizes writes by utilizing most of the free space on the cache drive, while making sure that the cache drive doesn’t run out of disk space.
- Optimizes reads by never overwriting previously learned data.
- Does not have a predictable fixed size.
- May consume large amounts of disk space when a lot of new data is written to the cloud drive.
The expandable cache type is the default and the recommended cache type in StableBit CloudDrive 188.8.131.527, and it was the only (implicit) cache type in 184.108.40.2063.
Another new feature in StableBit CloudDrive version 220.127.116.117 is ReFS support. StableBit CloudDrive can now format newly created cloud drives with the ReFS file system. When creating a new cloud drive, look under “Advanced Settings”, and you will be able to choose ReFS as the file system when using Windows 8 or newer.
ReFS is a Microsoft file system that is designed to be more resilient in the face of data corruption, but it may reduce the cloud drive’s overall performance by introducing some additional overhead. ReFS is only compatible with Microsoft Windows 8 and newer. Cloud drives formatted with the ReFS file system will not mount on older operating systems. So if you ever expect to attach a cloud drive to Windows 7 or older, do not use ReFS.
FTP / FTPS / SFTP Provider
In addition to ReFS support, StableBit CloudDrive now has comprehensive support for FTP, FTPS and SFTP. You can now create new cloud drives that store their data on FTP sites.
FTP over SSL is also supported (in both implicit and explicit modes), which includes the optional use of client certificates for authentication.
FTP over SSH support is available as well with optional private key authentication and certificate based authentication.
Kerberos based authentication (Domain\User) for FTPS can be used as well:
One potential issue with FTP is that it’s not uncommon for FTP servers to limit the number of connections that a user is allowed to make, and this can present a bit of a problem for StableBit CloudDrive. While there was always a way to configure the number of download and upload threads that StableBit CloudDrive uses, those threads do not necessarily correspond to connections. In the StableBit CloudDrive I/O pipeline, threads can be split or joined depending on the exact I/O operation being performed, and in certain instance, extra threads can be spawned on demand in a “thread boost” operation as well.
In order to address this strict connection limit requirement for FTP, version 18.104.22.1687 introduces a new “Maximum connections” setting to the I/O performance window.
This setting is only available for providers that are sensitive to the connection count (FTP only for now), and it starts off at a very conservative default of 2. You can of course increase or decrease this connection limit (or turn it off altogether) depending on the FTP server that you’re connecting to, but if enabled, it must be greater than the upload thread count. The reason for that, is that you always want to have at least one connection available for downloads in order to maintain reasonable drive performance.
I expect this to be the last public BETA of StableBit CloudDrive and hopefully a 1.0 release final is not far behind. Of course the exact timing of that will depend on any future feedback and bug reports that we receive.
If you do experience issues with this BETA, as always, please let us know here: https://stablebit.com/Contact
Aside from any critical bugs that are found, fit and finish is going to be the focus as we approach the 1.0 release final. This includes finishing up the documentation and making usability tweaks to the UI.
Finally, I’d like to thank everyone for testing all of the StableBit CloudDrive BETAs that we’ve had so far, and reporting any issues encountered. Quite a bit of the fixes in 22.214.171.1247 came straight from user feedback, and that just makes the software that much more stable for everyone.