PrimoCache & Nvme in 2018, enough to compete with optane 905P?

[GammaTron]

Hi

So could PrimoCache with alot of ram actually enhance nvme performance in theory? I am particularly interested in random low queue reading for nvme, as typically nvme never hits ever past 82mb/s with that, while intels xpoint optane claim to fame is that low queue performance with q1t1 being around 200 to 300 mb/s in performance, and hence that real tangible performance benefit. I havent found much here regarding the possibility of enhancing nvme performance with primocache since most aims here are enhancing large hdd drives in combination with sata III caching, so I am the first to ask.

So if primocache can be used to enhance hdd performance to compete with large ssd's, could it also be used to have nvme compete with optane? I only have 32gb of ram total to work with, though I usually never use it up

I am currently messing round with a trial version, but perhaps I need to do things differently in a manner that is for nvme? I am very new to this, and excited.

[Support]

RAM performance is much better than a SSD. When using ram caching, 1) For reading, when data is hit in cache, of course the access speed will be faster than SSDs. However, usually the amount of installed ram is much smaller than a nowadays SSD. So you cannot assure 100% hit rate.
Besides, ram is volatile storage device which means cached data will not be present after shutdown/restart. So for the first time of reading you still have to read from source disks.
2) For writing, when Defer-Write is enabled, the write performance will be improved a lot. However, you have to ensure no power loss and no system crash/suspend/freeze.

[GammaTron]

RAM performance is much better than a SSD. When using ram caching, 1) For reading, when data is hit in cache, of course the access speed will be faster than SSDs. However, usually the amount of installed ram is much smaller than a nowadays SSD. So you cannot assure 100% hit rate.
Besides, ram is volatile storage device which means cached data will not be present after shutdown/restart. So for the first time of reading you still have to read from source disks.
2) For writing, when Defer-Write is enabled, the write performance will be improved a lot. However, you have to ensure no power loss and no system suspend/freeze.

Worst that rarely happens is the OS may crash, but does this count as a system suspend?

So to experiment with this, what configuration parameters do you recommend for an NVME drive? I take it L2 cache wouldnt serve any purpose since that operation is meant for a sata ssd configured with a hhd, yes?

[Jaga]

Keep format clusters and Primocache L1 block size at 4k for maximum IOPS performance. Deferred writes (with a UPS on the machine) can still help, though it's less of a concern with a NVME. I usually recommend no less than a 10% RAM-to-Data sizing, so if you have 100GB of data to cache on your NVME, use a 10GB L1 or larger. Also set the L1 to prefetch the last cache on boot so it carries data across shutdowns and achieves higher hitrates. Use a Read & Write caching strategy, even if not using deferred writes.

[jussssx1]

I have tested primocache extensively with samsung nvme drives and optane 900p. Primocache
further increases low queue depth transfer speeds to 600-700Mb/s (from optane's 200-300 Mb/s)
but the real-world performance does not increase (at least with my workloads).

the issue seems to be with a) windows filesystem filter driver causing small additional delay,
and b) the windows+cpu mitigations for spectre/meltdown (which ***really*** kill performance of
4K transfers, i.e. 70% drop in performance)

because of these issues, I have disabled OS level mitigations for spectre/meltdown and I am
currently using primocache with HDDs only.

[Support]

Worst that rarely happens is the OS may crash, but does this count as a system suspend?

Sorry, forgot to mention the crash case. I modified my post.

I take it L2 cache wouldnt serve any purpose since that operation is meant for a sata ssd configured with a hhd, yes?

Yes, L2 purpose is using a fast drive to speed up a slow drive.

[GammaTron]

I have tested primocache extensively with samsung nvme drives and optane 900p. Primocache
further increases low queue depth transfer speeds to 600-700Mb/s (from optane's 200-300 Mb/s)
but the real-world performance does not increase (at least with my workloads).

the issue seems to be with a) windows filesystem filter driver causing small additional delay,
and b) the windows+cpu mitigations for spectre/meltdown (which ***really*** kill performance of
4K transfers, i.e. 70% drop in performance)

because of these issues, I have disabled OS level mitigations for spectre/meltdown and I am
currently using primocache with HDDs only.

What in theory could get around these bottlenecks? Typically people say there is no real world performance going past sata III even, but I think its nvme for certain at this point at least. I dont have any optane drives given their price, as I merely made that as a sort of analogy as to what nvme drives would compete with.

[cichy45]

Well, given the fact that some SATAIII SSD (good controllers + fast MLC/TLC NAND) are faster than NVME M.2 (bad controller + weak, slow TLC NAND) when it comes to high load, you should look at benchmarks that hammer SSD with lots of work, like encryption (encryption is showing real-world NAND performance).



As you can see here, Samsung 860 EVO has faster NAND than some M.2 NVME when it comes to really hard usage (volume encryption using VeraCrypt in this particular case, when you cant use any pseudo-SLC buffers or data compression "magic tricks").

[jussssx1]

well.. there are couple of items which make Optane a lot faster than typical SSD:
First - the consistency of low latency: Regardless of my load type, optane 900p will always
have less than .01 ms latency (and I run some heavy transaction loads).
Secondly - optane will not use nor require TRIM - so even if you write to the areas of the
drive that were previously occupied, there is no penalty in performance. This begins to
be important if your drive is >75% full.

Having said these, I think primocache will offer similar performance benefits as optane
(it's even faster with some loads - and certainly with benchmarks). But the next
bottleneck is Windows (filesystem filter driver, and crippled CPU interrupt management).

-Jussi

[GammaTron]

Well, given the fact that some SATAIII SSD (good controllers + fast MLC/TLC NAND) are faster than NVME M.2 (bad controller + weak, slow TLC NAND) when it comes to high load, you should look at benchmarks that hammer SSD with lots of work, like encryption (encryption is showing real-world NAND performance).

you can see here, Samsung 860 EVO has faster NAND than some M.2 NVME when it comes to really hard usage (volume encryption using VeraCrypt in this particular case, when you cant use any pseudo-SLC buffers or data compression "magic tricks").

That is actually an 860 "PRO" and not a consumer Evo in that picture. Typically server grade equipment will perform better then consumer in these said area's, but I dont typically transfer 100+ gb files to really notice. My new nvme should even compete with a 970 pro based off of a few tests, and the ssd is a SX8200 Pro and the controller is a SMI SM2262EN, to which curiously adata provides the only one on the market for that controller. Typically most people stay away from TLC in server/workstation area and use mlc as its reliability is just as good as SLC, but that cost goes up even for sata III, well past nvme prices.

What program would you recommend to stress my my ssd to see what results I would get?