Wish: Replace L1 RAM caching with L1 SSD caching (along L2 SSD caching)

[Axel Mertes]

I have mentioned some of the below in a support question already, but I think we should discuss this here, as its the appropriate place.

I consider using ReFS instead of NTFS. Is that supported by PrimoCache as well?
Do I need to add each single disk of a storage pool to the caching in order to perform true caching of the ReFS partition?

My point for aiming for ReFS is following:
With NTFS I do risk corruption and unresolveable issues, especially when I consider using write caching or deferred write caching.
ReFS is said to be able to fix its issues itsef. However, we don't know if that is still true with PrimoCache deferred write caching enabled. If I understood ReFS right, it will write a new stream for e.g. the file index, and only if that happens to be successful, it will also change the pointer to the last version of the file index. If writing the file index is finished, but not yet written to physical disks (deferred writes), then in theory we could get a corruption if the machine powers off before the deferred writes can be secured. On the other hand, then this scenario should likely include the pointer change to the latest file index as well, which would mean in the case of the power loss the file index is not officially refreshed, so the self-healing of the drive may work (with the exception of some data loss for the period of deferred writes).

I know that we may use PrimoCache for read only caching, but the performance increase due to (deferred) write caching is outstanding. I also wonder if using e.g. HP z Drive Turbo with its power backed up architecture will help to prevent data loss.

In that context it would strongly make sense to enable write caching to simply run through L2 SSD cache only and bypassing L1 RAM caching optionally, including the index of the write caching (which should then be held on the L2 SSD too).


However, what would that imply for the state of re-powering the system?
Valid write data might exist in the SSD, but has not yet been written to HDD (deferred writing in a sense).

Would that be possible?

The performance gain of SSDs over HDDs is still tremendous. Clearly RAM is even faster and deferred writing from L1 RAM to L2 SSD would reduce wear on the L2 SSDs. IMHO its far better to consider using only L2 SSD cache and bypass L1 RAM cache, if I can prevent data loss.

Another option would be to use SSD for L1 cache as well, as for L2 cache. By replacing RAM with SSD we will gain security. With the advent of high performance 3D XPoint SSDs - even small ones - will be likely as fast and significantly larger than RAM for L1 cache, but non-volatile like SSDs. So rebuilding PrimoCaches architecture to support L1 cache on SSDs rather than RAM only will make it really future proof.


With the upcoming Intel/Micro 3D Xpoint / Crosspoint Optane SSDs (1000x faster than current Flash SSDs) there will be no question to bypass L1 RAM forever, because the Optane memory is non-volatile, so will keep its content after a power loss. For PrimoCache users this is big news IMHO.

The companies claimed it was a new type of memory that is up to 1,000 times faster than NAND flash and has 1,000 times the endurance. One thousand times the endurance would be about 1 million erase-write cycles, meaning the new memory would essentially last forever.

...which would make deferred write disabled writing to the SSD cache very feaseable, both performance and endurance wise....

At its Intel Developers Forum in Shenzhen, China, the company said Optane has up to 10X the density of NAND flash and will enable SSDs to store more than a terabyte of data in an M.2 card that is only 1.5 millimeters thick.

... which will result in easy integration of todays systems, as M.2 NVMe SSDs become the standart now, with lots of notebooks and mainboards having the M.2 NVMe connector, or support PCIe adpater cards from a few dollars/euros to extreme ones like the HP z Drive Turbo Quad...

See here:

http://www.intel.com/content/www/us/en/ ... =3d+xpoint
https://newsroom.intel.com/press-kits/i ... -products/
https://newsroom.intel.com/chip-shots/c ... 3d-xpoint/
http://www.computerworld.com/article/30 ... mance.html

First Intel Optane products are scheduled for Q4 this year, so almost "around the corner".

If I can use a SSD that is as fast as my DRAM, then I would bypass DRAM. This solves two issues at once:

1. No more risk to loose data in case of a power fail.
2. No size limitation regarding indexing of cache, as it will be surely significantly cheaper than DRAM (and possibly larger too).

So whatever you have in petto for Version 3 - please consider enabling us to not need to use L1 RAM cache at all and run only through L2 SSD cache if desired, including the actual cache index. Make sure that the index & cache can be picked up from their last state in the case of a power loss/blue screen etc. so that outstanding writes from cache to storage can be resumed.

If that will work, then PrimoCache write caching becomes immediately secure and enterprise ready. And of xourse, with the new technology, read caching can be improved as well, as you L2 cache does not need to be refilled at all, so you cache survives any reboots. I think that works in parts already, right?

Cheers,
Axel

[Axel Mertes]

http://www.digitaltrends.com/computing/ ... oint-demo/

http://www.digitaltrends.com/computing/ ... -3dxpoint/

https://www.youtube.com/watch?v=VsioS35D-HY

[Support]

@Axel Mertes, thank you for some new ideas and discussions!
Actually in current versions, you can disable L1 cache by set L1 cache size to 0, and only use SSD cache (though it namely called L2 cache).
And regarding ReFS, PrimoCache can support it because PrimoCache is a block-level caching program. However, if you enable Defer-Write feature in PrimoCache, it still introduce the risk of data loss.

[Axel Mertes]

Hi Support,

thanks for lettung us know. I will give it a try using SSD only caching and how performant that will be.

Given the hardware promises of the fast Intel/Micron XPoint technology, do you think you will manage at some point to recover a *write cache* after a crash/power outage as I described above?

Of couse the same is true for the read cache - so no longer re-filling read cache, but trusting the content is still vaild on SSDs and having a log of its content held on SSD too.

Another question:
Can I define a RAM L1/SSD L2 read cache in one task, and a SSD only (L1 size = 0) write cache without deferred writes in another task, caching the very same drive?
May that cause corruptions or should that work?
Would that kind of write caching without deferred writes be useful at all?
I hope it will allow to safe data as fast as the SSDs are, and will write back to HDD as fast as the HDDs are. However, it still can cause a loss, if there is a power outage before the SSD cache has been completely written to HDD, right?

Could be a workaround for the time being, but I think for security reasons I stay without write caching for now.

Best
Axel

[Support]

Given the hardware promises of the fast Intel/Micron XPoint technology, do you think you will manage at some point to recover a *write cache* after a crash/power outage as I described above?

Yes, we always think to support recovering write cache from L2 ssd cache in order to reduce the risk of data loss.

Can I define a RAM L1/SSD L2 read cache in one task, and a SSD only (L1 size = 0) write cache without deferred writes in another task, caching the very same drive?

One volume can only be cached by one cache task. And in current verison, SSD-only cache doesn't support write cache.

[Axel Mertes]

@support
Thank you for the quick reply.

So I hope to see support for write cache recovery soon

For the time being we stay with read only caches and try to get our HDD RAID performance up a bit. Its quite a split between 390 MByte/s writes to HDD and 10.000+ MByte/s reads from PrimoCache...