Which benchtest software to evaluate caching advantage for NVMe drive?

[BonzaiDuck]

I just built this new system -- Skylake, Z170 mobo, 16GB TridentZ DDR4-3200, SATA SSD dual-boot system drive, 2TB 2.5" HDD and 1TB 2.5" HDD.

For months, waiting for the release of the Samsung 960 models of M.2 NVMe, I'd been debating what to do. I decided to experiment and keep some money in my pocket for the time being. So instead of buying a 1TB 960 Pro or EVO, I got the little 250GB 960 EVO and a PCI-E x4 expansion card for it to use in a PCI-E 3.0 slot.

There's a psychology behind this. "Whether or not to use RAM-caching." The benchtests apparently don't reflect the resulting cache speeds of an SSD alone. So one has a tendency to define caching tasks with RAM assigned as well as an L2 caching-SSD.

Correct me, but this seemed to be the behavior of several benchtest programs with PrimoCache caching tasks. I've run Magician benchmark, ATTO, AS SSD, Anvil and CrystalDiskMark.

Of course, with Magician, an SATA SSD cached to the NVMe M.2 SSD and 1,500MB of RAM-cache shows a sequential read rate of 18,000 to 29,000 MB/s. Then I had two HDDs connected -- one with an existing caching task with 75GB of L2 and 1,500 MB RAM, and a second HDD with only 1,500 MB of RAM and no L2. The latter under Magician shows ~ 3,100 MB/s; the former shows about 11,100 MB/s. So we know that the NVMe caching is working.

But I'd like to move from "qualitative" evaluation to "quantitative." But which benchtest program will actually show it with no RAM-cache allocated?

[Support]

Benchmark tools such as CrystalDiskMark, Anvil, and AS SSD test disk speeds by following two steps.
1. Preparation step: Create/write a test file with a specified test size first. This test file is usually 1GB size, and users can change it by setting the test size.
2. Testing step: Read/write the test file multiple times, and then calculate the speed.

When you set PrimoCache's level-1 cache size bigger than the test file size, the test file will be fully stored in the level-1 cache during the preparation step. So later in the testing step, benchmark tools actually read/write the file from the level-1 cache. That's why you see big performance boost.

For level-2 cache, in order not to affect other applications' tasks, PrimoCache only populates level-2 cache when it detects Windows is idle. Besides, currently level-2 cache doesn't support write-data. So during the preparation step the test file will not be stored into level-2 cache and in the testing step all reads/writes will be completed on the target disk.

In the coming PrimoCache 3.0 version, we'll support level-2 write caching. You'll see the different testing results then. Currently I think you may just see the performance boost with level-2 cache in real usage scenarios.

[BonzaiDuck]

This is really quite good to know, and it would be even better if I knew when you will release v.3.0.

The most of TBW for the NVMe M.2 250GB disk occurs from altering or deleting, then recreating L2 volumes. So for less than 2 weeks operation of trials and errors, I've racked up about 800GB of writes. You can see the use of the drive by Primo, but the numbers of MB written don't seem all that large.

Because if this confusion which you explain, I was looking at other software. There was one program called eBoostr, which isn't validated for Windows 10 and therefore -- thumbs down. An outfit in Germany incorporated in UK is EliteBytes, with their program MaxVeloSSD. Their web-page doesn't provide enough information, and their trial period includes a "2-hour" stage after download, from which you then "register" for a full 15-day trial.

I think I'm sticking with Primo. And if I seem impatient for v.3.0, I can calmly wait with hopes it won't have any bugs. Haven't found any in the versions I've used so far!

[Axel Mertes]

I am also strongly looking forward to 3.0 as I think it will introduce a lot of features I have asked for.

For the time being I use PrimoCache 2.x (currently 2.4 on my server) as its a huge timesaver. I have never found a real bug so far, too.

[BonzaiDuck]

Well, I think there are a lot of loyal customers here. Your average "Mainstreamer" user may not grasp it or see an advantage.

Everyone agrees that tiered-caching or at least RAM-caching is useful for servers. Among the enthusiasts, there are a sizeable number who use it for workstations, but they don't brag about it so much that you see that number right away, yet you don't find among those anyone complaining or foreswearing it. The other enthusiasts are skeptical: "Why not just buy top-tier, state-of-the-art solid-state storage?" But nobody disagrees -- among the enthusiast community -- that a desktop/workstation system is better for integrating established technologies. You still get the greatest capacity and a certain guaranteed unpowered data-longevity from an HDD. Price per gigabyte is lowest; speed may be the slowest. But there's a place for it.

I wouldn't accelerate or RAM-cache a DVR Media drive -- it would serve no purpose, unless I were editing video. And I think, if RAM were plentiful and you had more than enough, you could still cache such storage. You wouldn't want to do it with the L2 feature. How could it otherwise be at a disadvantage? I'm thinking that even the RAM-caching would mean less wear and tear on the source disk.

The only caveat concerns hardware reliability and stability. Memory must be flawless. For L2, you want something that has a decent number of TBW expectations, but limited capacity. If machine has to hibernate, you must accept larger minimum HIBERFIL.SYS on boot-media that itself has limited TBW expectations, but then you might spend more money on such a drive anyway -- like a Samsung 960 Pro 1TB with something up to 1.2 Petabyte TBW expectations. Then, there's the pocketbook or IT budget.