Feature Request: Support accelerating writes with L2 Storage

Suggestions around PrimoCache
BrainSlugs83
Level 2
Level 2
Posts: 5
Joined: Thu Nov 13, 2014 6:56 pm

Feature Request: Support accelerating writes with L2 Storage

Post by BrainSlugs83 »

As a PrimoCache enthusiast, I would like to be able to use L2 cache for deferred writes.

If possible I would like to be able to use one L2 storage to do all the read/write caching (like how we can do with memory), but even if I had to divide it into chunks (or partitions) one for reading, one for writing, that would be fine too. -- I just want to do it all from PrimoCache. (Intel IRST has this feature, and it works fine with PrimoCache but it has a lot of limitations and is painful to setup; it maxes out at 64-GB, and requires that you have a supported Intel CPU and Southbridge and requires a second service to run in the background, and that all the drives are hooked up to specific SATA ports on your board... again: it's a huge, pain, I would rather just use one program to do everything, so that no data ends up being cached twice / there's no wasted space / and so that I could use the feature on my AMD machines as well...)

I would like to be able to configure a much more lenient deferred write timer (one for RAM which is short, less than 30 seconds, one for SSD which could be much longer, hours even).

Because it's L2 and stored on an SSD, it should be tolerant of system crashes / power-outages, etc. (i.e. when the system powers back on, if there is still data on the SSD that needs to be written, that data is considered newer than the data on the accelerated volume, and the deferred-write should continue where it left off).

(As mentioned there are existing technologies that do this; Intel Rapid Storage Technology is one example but it's very proprietary; there's another software called "Dataplex" which I understand can do this, but it's even more proprietary than IRST; and there's also a bunch of open source Linux implementations of hybrid-drive software, but it's not much help for a Windows user.)
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: Feature Request: Support accelerating writes with L2 Sto

Post by Support »

We do have this feautre on our task list. It's not very difficult to simply forward write to L2 storage. The difficulty is how to handle all possible crash cases and restore data from L2 storage. This feature is still being developed. Anyway, thank you for the suggestion.
BrainSlugs83
Level 2
Level 2
Posts: 5
Joined: Thu Nov 13, 2014 6:56 pm

Re: Feature Request: Support accelerating writes with L2 Sto

Post by BrainSlugs83 »

Awesome. Is there an ETA (or roadmap, etc.) for this feature?

Even if this feature required a special driver (at windows install time -- via F6 or whatever), I would love to use this.

If my PC crashes, I kind of expect to loose a little bit of data -- as long as it supports the most likely recovery scenarios (i.e. does the same thing NTFS or any otherjournaled file system would do).

Any possibility to get in on a beta? :D
Shen
Level 1
Level 1
Posts: 1
Joined: Mon Mar 14, 2016 7:05 am

Re: Feature Request: Support accelerating writes with L2 Storage

Post by Shen »

I'd also like to see get writeback cache on SSDs. You mentioned "It's not very difficult to simply forward write to L2 storage. The difficulty is how to handle all possible crash cases and restore data from L2 storage." I think you should enable this feature without implementing any crash restoration for now, because isn't that the same case as ram writeback caching? If the computer crushes we lose all the data. You could have a toggle to enable ssd writeback cache as an option.
newtekie1
Level 1
Level 1
Posts: 1
Joined: Wed Mar 23, 2016 4:27 pm

Re: Feature Request: Support accelerating writes with L2 Storage

Post by newtekie1 »

This is the only feature that is stopping me from buying this software.

I have a large RAID5 array. The read speeds are actually just fine, but the write speed is annoyingly slow. I don't want to use RAM to accelerate data writing because a crash or power outage would cause corrupt data. Writing the data to an SSD cache would mean the data could finish being written after a crash/power outage.
Vano
Level 2
Level 2
Posts: 7
Joined: Wed Jan 27, 2016 4:49 am

Re: Feature Request: Support accelerating writes with L2 Storage

Post by Vano »

Was the key factor in me declaring the software "not ready for production use", too.
Vano
Level 2
Level 2
Posts: 7
Joined: Wed Jan 27, 2016 4:49 am

Re: Feature Request: Support accelerating writes with L2 Storage

Post by Vano »

With a boot-time-load driver, I don't readily see any pitfalls except dealing with interrupted writes to files that are also boot-time loaded and do so before our driver (these are always in use and are typically replaced with PendingFileRenameOperations)... as well as of filesystem metadata related to these files and the directories they are in.
Axel Mertes
Level 9
Level 9
Posts: 180
Joined: Thu Feb 03, 2011 3:22 pm

Re: Feature Request: Support accelerating writes with L2 Storage

Post by Axel Mertes »

I also agree that a deferred write could and should be improved.

IMHO we should see:

Deferred writes from L1 RAM to L2 SSD and then from L2 SSD to HDD (each with individually adjustable delays L1->L2 and L2->HDD)

or alternatively
Deferred writes from L1 RAM to L2 SSD AND L1 RAM to HDD (each with individually adjustable delays L1->L2 and L1->HDD)


Some examples:

1. By setting L1->L2 to 0 seconds and L2->HDD to 1 day, you would end up in a safe L2 cache that should not get corrupt at all, but have a lot of wear on the SSD. Using SLC might help a bit in that context.

2. By setting L1->L2 to 10 seconds and L2->HDD to 1 day you have the same as above, but with minimized wear on the SSD and the risk of loosing the last 10 seconds of changes.

Idea: What about implementing a 2 level SSD cache (L1 RAM->L1.5 SLC SSD->L2 SSD->HDD)?
One small "L1.5" SLC SSD that gets immediate writes from L1 RAM cache and does simply mirror the RAM cache. Then deferred writes to true L2 SSD cache (large, TBytes...) and finally HDD. That would clearly reduce write performance from RAM speed to SSD speed, but given there are SSDs between 2 to 10 GByte/s now (PCIe, NVMe) that is not a big problem IMHO. A small SSD might take the wear and gets replaced sooner than later, while a larger SSD cache will stay alive for a long time (physically). Such an approach might be fairly simple, as its more or less cascading existing caching software code & schemes. At best there would be a battery buffered RAM as L1.5 cache...

So as a bottom line we might think about if we can actually cache the L1 RAM cache with an superfast L2 SLC NVMe cache and then a somewhat slower, larger L3 SSD cache and L4 is HDD disk/raids.

3. By setting L1->L2 to 10 seconds and L1->HDD to 0 seconds you would have a safe HDD array (slow, but really safe writes, as the HDD is always up to date as without having a cache at all) and minimized wear on the SSD and immediate reads from SSD cache if data has been written (not read) recently.


We should not divide cache into read and write cache, I think thats a bad approach. We should simply use the entire cache using the common approaches, such as least used cache blocks gets overwritten first etc. [Beside that we require a copy of the L2 cache map on the L2 cache itself (for journaling reasons).] By writing fresh "write data" to the L2 cache too, that would at the same time inherently implement another very import feature I am waiting for:

Write fresh data immediately (or deferred, to protect SSD lifetime) to the SSD L2 cache, so subsequent reads of data will be fed by the SSD cache and no longer require to read data from HDD into read cache.

Presumeably in most bigger data environments that could benefit from caching at all its pretty likely that recently recorded data will be accessed soon after. In our case its rendered images from visual effects movie production. We render, then we review the results. Right now we render, write to disk, then read - at this time it gets carbon copied into the L2 cache - and maybe read again and again in a review process. So if I write 100 GByte of fresh data and review it 5 times, it gets written once to HDD, then read once from HDD and 4 times from SSD L2 cache. By carbon copying the data immediately when its being written to the L2 SSD cache, we have only reads from the L2 SSD cache afterwards, making the whole process significantly faster.

If I think about other use cases, e.g. collecting data from sensors and then processing the very same data afterwards, it ends up in a similar scenario. Or imagine scanning images or medical scanners etc. and then reading back the data immediately after recording. Its a quite common scenario.
Axel Mertes
Level 9
Level 9
Posts: 180
Joined: Thu Feb 03, 2011 3:22 pm

Re: Feature Request: Support accelerating writes with L2 Storage

Post by Axel Mertes »

Btw, in example 2

2. By setting L1->L2 to 10 seconds and L2->HDD to 1 day you have the same as above, but with minimized wear on the SSD and the risk of loosing the last 10 seconds of changes.

I would really consider using 1 day as delay, because my cache is designed large enough to usually keep 2 full days of changes. Each night we free up disk space by emptying cash trans and afterwards defragmenting the volume. Have you ever seen defragmentation running on a 2 TByte SSD cache? Hellfast to say the least. The underlying disk would be written to only once in best case - and in total non fragmented manner. As a result - in a true desaster - it is very unlikely to destroy much data if you miss the last 10 seconds of writes. And from a defragmented volume you can recover very very safely and easy. Remember that NTFS usually places two copies of the MFT on a disk. How likely is it that both copies are affected?
kalua
Level 4
Level 4
Posts: 35
Joined: Thu Aug 19, 2010 1:38 pm

Re: Feature Request: Support accelerating writes with L2 Storage

Post by kalua »

This is the sole feature that's keeping me from buying too. The work I do involves a LOT of writing to very large directory structures. Because of the size of the datasets, SSDs everywhere are cost prohibitive so I have RAIDs of spinners.

When I first added PrimoCache with write cache enabled, it was the single greatest upgrade to my workflow in years...until the RAM cache filled up. But even on a system with 64GB, I can't give up more than 8GB for cacheing and once the 8GB write cache fills up, the poor raids get slaughtered again.

I'd even be willing to pair an SSD to a single underlying hard drive in order to have an 256GB SSD-sized write cache for the spinner.
Post Reply