Safe write caching usage - in a way

[Axel Mertes]

Hi All & Support,

I repeatetly suggested a few things to PrimoCache, from which I hope most landet on the feature list for the next release:

- Preloading custom amount of 'n' blocks AFTER the last requested block to L1/L2 cache -> improve read performance on defragmented volumes strongly.

- Copying any written block to L1/L2 read cache, as most often recently written data gets re-read soon after (at least in my scenario video editing).

- Safe write caching via L2/SSD only (Database and cached blocks). Enabling write cache "pickup" after a crash/power outage, kind of a "journaling" of unfinished write operations. This would require to keep the database of what is cached or not on SSD, as well as all cached write blocks (makes sense as soon as Intel releases its first Optane PCIe SSD cards, you will see).




So, while I am currently testing a lot of software/configurations for my new servers, I played e.g. with Diskeeper Intelliwrite. I have no idea yet what exactly and how they do it, but it brought me to an idea:

1. Enable the above mentioned feature "Copying any written block to L1/L2 read cache".
2. Enable the above mentioned feature "Preloading custom amount of 'n' blocks".
3. Enable the above mentioned feature "Safe write caching via L2/SSD only (Database and cached blocks)".
4. Use a defrag software to defragment the volume e.g. every hour.

What would happen?

1. You write any data, even concurrent writes with tons of fragmentation happening on HDD. Data is written "through" and immediately safe on einther the SSD or directly to the HDD.
2. The defragmentation process will be able to perform all its read operations from the cache, as its very unlikely that the data is not still in the cache (clearly depends on the size of your cache, but in my case, absolutely sure about this).
3. The defragmentation process will lead to a clean volume with sequential files.
4. When reading sequential files which may not be cached, the preload "n" blocks feature will apply and allow to read the file signficantly faster, without the host application needing to perform any preloading - as this would be done by PrimoCache automatically.
5. The above scenario is extremely helpful if you experience any kind of file system desaster or e.g. an undesired format or partitioning operation: The less fragmented a file system is, the less data can be affected by sequentially overwriting the partion (be it through/after formatting, or through partioning). Further its way more likely you can recover your old HDD content using R-Studio or similar recovery software, and its more likely you won*t seen millions of no longer existing files from the past.
6. After all you would have a fast and quite reliable storage solution, that is both fast and secure.

I hope this description gives a clear example what is my goal.

And by the way - it would be way more powerful than this Intelliwrite thing from Diskeeper, as Diskeeper can't utilize L2 cache at all.

Cheers
Axel

[Support]

Many thanks for suggestions and the detailed description!
We do have the plan to implement a smart preload feature, but it needs time as currently we're working on other tasks and each task is not easy to be implemented...

Copying any written block to L1/L2 read cache, as most often recently written data gets re-read soon after (at least in my scenario video editing).

PrimoCache does. When the cache strategy is "read-data & Write-data" or "Write-data Only", write-data is always stored into L1 cache whether defer-write is enabled or not. When we support write caching on L2, these written blocks can also be stored into L2 cache.

[Axel Mertes]

Many thanks for suggestions and the detailed description!
We do have the plan to implement a smart preload feature, but it needs time as currently we're working on other tasks and each task is not easy to be implemented...

Copying any written block to L1/L2 read cache, as most often recently written data gets re-read soon after (at least in my scenario video editing).

PrimoCache does. When the cache strategy is "read-data & Write-data" or "Write-data Only", write-data is always stored into L1 cache whether defer-write is enabled or not. When we support write caching on L2, these written blocks can also be stored into L2 cache.

Thanks for the quick feedback - as always

The point is that I currently do not use any write caching at all, due to security reasons. But even then it would make a lot of sense (in my "video editing" scenario) to instantly place a copy of any written block into the read cache, as it is extremely likely I will request that block soon after.

But you are right, being able to have have *safe* write caching via L2 only would solve things best.