Can PrimoCache rival the Intel Optane System Acceleration ?

FAQ, getting help, user experience about PrimoCache
Zyxel
Level 1
Level 1
Posts: 4
Joined: Fri Mar 16, 2018 9:48 pm

Can PrimoCache rival the Intel Optane System Acceleration ?

Post by Zyxel »

I have a Z370 motherboard with Intel Optane System Acceleration support and a 32GB M2 NVMe Optane module and a slow 4TB mechanical SATA drive.

I cannot use the Optane module to accelerate the mechanical disk with Windows7 x64 because Intel's Rapid Storage Technology driver(v15.9.1) GUI displays:
"Your system appears to be Intel Optane memory ready but no Intel Optane memory modules were detected in your computer. Please install a compatible module"

The above message is blatantly incorrect because the M2 Optane module is installed and visible in BIOS (in RAID/Optane mode) and in Windows 7 Disk Management. I can even format it and write/read files to/from it.

Also, I have temorarily test-installed Windows 10 x64 and then the Intel's RST driver GUI does not complain and does not display this inaccurate message and the Optane System Acceleration works as adverised.
However, Windows 10 is shite and I will not use it. So, I wiped the SATA mechanical drive and reinstalled Windows 7 x64 on it.

The Intel's Optane FAQ states that System Acceleration is NOT SUPPORTED on Windows 7 and on Windows 8 despite these Operating Systems not having reached their end of life. See below:
https://www.intel.com/content/www/us/en ... pkw=optane

What differences in driver API would make it possible for Optane System Acceleration to work with Windows 10 x64 but not with Windows 7 x64 ?
...or is withholding support for older versions of Windows a result of some kind of collusion between Microsoft and Intel aimed at coercing people to migrate to Windows 10 ?

Anyway, all of the above nonsense brings me to this forum with the following questions:
Q1: Can the performance of PrimoCache with Windows7 x64 rival the performance of Intel Optane System Acceleration with Windows 10 x64, in the system outlined above ?
Q2: If "yes" - how should PrimoCache be set up with Windows 7 x64 in order to rival the performance of Intel Optane System Acceleration with Windows 10 x64 ?

P.S.
I do not have RAM to spare...
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: Can PrimoCache rival the Intel Optane System Acceleration ?

Post by Jaga »

No reason not to migrate to Windows 10 at this point. It's just as stable and fast as Windows 7 for practical purposes. If you still absolutely refuse to use it - stop reading here and ignore the rest of my post. I am however a tech by trade, having supported everything from desktops to Datacenter rack server systems for the last 20+ years. I'm also a system builder, so can say I've done it dozens or hundreds of times and the results are consistent.

Yes - you're probably experiencing "forced upgrade" pathing from the manufacturers. They rarely want to write driver sets for old Operating Systems when releasing new hardware. Once they do, it means they have to support those drivers on the old OSs for a long time. Ultimately it's a waste of time/money for them to do it. If *YOU* want the latest/greatest, you have to upgrade to take advantage of it.


A1: Yes, with a decent MB/CPU/RAM combination, I'd say you can easily rival an Optane module. Here's what I get using run-of-the-mill RAM on an OC'd CPU and Primocache (20gb L1) running with W10:

Image

I have yet to see an NVME drive reach those speeds, though it's not entirely impossible. I haven't had too many clients actually request them yet.


A2: You'll probably want to disable the RST monitoring software no matter what else you do, since it can cause issues with Primocache. 3-4 years ago I sent in a bug report to Romex about an issue I was having, and that I'd resolved it by uninstalling all the Intel RST stuff *except* the driver. After that, setup a single L1 cache in RAM for the NVME drive. You didn't specify how much system RAM you have installed, but giving Primocache less than 8gb is usually going to starve it and slow it's effectiveness down. Since you have Primo covering the NVME, there's no reason to add the SSD as a L2 cache.

If you don't have enough RAM to toss Primocache at least 8gb, stop right here. Small L1 caches won't help a RAM-starved system at all, unless it's an old slow laptop.
Zyxel
Level 1
Level 1
Posts: 4
Joined: Fri Mar 16, 2018 9:48 pm

Re: Can PrimoCache rival the Intel Optane System Acceleration ?

Post by Zyxel »

I already wrote, that in my opinion Windows 10 is shite. I do not want to have anything to do with it.
If I could tolerate Windows 10, I would not be on this forum looking at Primo Cache because Intel Optane System Acceleration works with Windows 10 as advertised.

On my continent, only 35% of Windows users run Windows10 and 49% run Windows7, so for me Windows 7 is the MAJOR Windows operating system here. See this link.
Also, I can provide good technical justification for my decision to stay away from Win10, but that would be off-topic in this thread and in this forum

Also, I clearly wrote, that "I do not have RAM to spare", so all of your statements about L1 caching in RAM are off-topic. (not wrong, just off-topic).
Also, RAM is volatile so it cannot support deferred writes like Optane NV memory, without risk of data loss in the event of power outage or OS crash/lockup, thus L1 write-caching using RAM is a non-solution for me....and for most people, too IMO
Hodor
Level 2
Level 2
Posts: 7
Joined: Mon Mar 19, 2018 2:24 am

Re: Can PrimoCache rival the Intel Optane System Acceleration ?

Post by Hodor »

I've never used Optane but have plenty of experience with SRT on a SATA SSD. In my experience, there's isn't that much difference between PrimoCache and the Intel solutions and you should get more or less the same performance. The major advantage of SRT is that it works at a BIOS level and will try to rebuild your cache before Windows boots in case of system failure. It might also give you better boot times but performance should otherwise be identical once you're in Windows.

Just keep in mind that Win7 is unsupported on the Z370 platform so it's probably not a good idea trying to hack Optane to work on this OS. I wouldn't want to rely on hacky workarounds (that might brick the system) to get Optane/SRT running when PrimoCache is fully supported on Win7.
Zyxel
Level 1
Level 1
Posts: 4
Joined: Fri Mar 16, 2018 9:48 pm

Re: Can PrimoCache rival the Intel Optane System Acceleration ?

Post by Zyxel »

Hodor wrote:Just keep in mind that Win7 is unsupported on the Z370 platform so it's probably not a good idea trying to hack Optane to work on this OS. I wouldn't want to rely on hacky workarounds (that might brick the system)
Do you realize, that there is no natural incompatibility between the Intel Z370 chipset and the Windows7 Kernel or its HAL ?
Do you realize, that there is only an artificially introduced incompatibility between the Windows Update Client (wuaueng.dll) and KabyLake+ CPUs ...namely the return value of the IsCPUSupported() function ?

That's only one user mode component and patching the return value of its purposely crippled function should be considered an effective Windows repair - not a hack, ...unless your definition of a hack is anything not sanctioned by Microsoft.
Hodor wrote: ...to get Optane/SRT running when PrimoCache is fully supported on Win7.
In my experience, there's isn't that much difference between PrimoCache and the Intel solutions and you should get more or less the same performance.
Now, that is a better argument, but do you mean PrimoCache with L1 volatile RAM cache with deferred writes (dw), like the member "Jaga" mentioned, or only the L2 cache (w/ dw) based on non-volatile memory, like I was asking about originally?
Hodor
Level 2
Level 2
Posts: 7
Joined: Mon Mar 19, 2018 2:24 am

Re: Can PrimoCache rival the Intel Optane System Acceleration ?

Post by Hodor »

It's the reason that I used 'unsupported' rather than 'incompatible'. Not supporting Win7 on Z370 was a poor move on Intel's part but I still wanted to mention that Optane has not been validated for Win7 even though it's unlikely to cause issues.
Now, that is a better argument, but do you mean PrimoCache with L1 volatile RAM cache with deferred writes (dw), like the member "Jaga" mentioned, or only the L2 cache (w/ dw) based on non-volatile memory, like I was asking about originally?
I only use the L2 cache and with deferred writes. Windows' own memory manager handles files well enough that I haven't seen any gains from using the L1 cache. It's not something that I'd use unless you explicitly need faster-than-SSD write throughput since Windows' MM only works as a read cache.
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: Can PrimoCache rival the Intel Optane System Acceleration ?

Post by Jaga »

Zyxel wrote:I already wrote, that in my opinion Windows 10 is shite. I do not want to have anything to do with it.
Hence the reason I put this in my first paragraph:
If you still absolutely refuse to use it - stop reading here and ignore the rest of my post.
You can rant at anyone you want, but you seem to like to ignore what's posted. You'd be better served by ranting at Intel.
Last edited by Jaga on Wed Mar 21, 2018 11:05 pm, edited 1 time in total.
Zyxel
Level 1
Level 1
Posts: 4
Joined: Fri Mar 16, 2018 9:48 pm

Re: Can PrimoCache rival the Intel Optane System Acceleration ?

Post by Zyxel »

Hodor wrote:I only use the L2 cache and with deferred writes. Windows' own memory manager handles files well enough that I haven't seen any gains from using the L1 cache. It's not something that I'd use unless you explicitly need faster-than-SSD write throughput since Windows' MM only works as a read cache.
...and how do you like to configure PrimoCache to work optimally with a fast NV L2 cache with deferred writes?
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: Can PrimoCache rival the Intel Optane System Acceleration ?

Post by InquiringMind »

Zyxel wrote:...the M2 Optane module is installed and visible in BIOS (in RAID/Optane mode) and in Windows 7 Disk Management. I can even format it and write/read files to/from it....Q1: Can the performance of PrimoCache with Windows7 x64 rival the performance of Intel Optane System Acceleration with Windows 10 x64, in the system outlined above ?
If you can format and read/write in Disk Management, then you should be able to format the NVMe for L2 usage in PrimoCache. Unless Intel's RST is using special "tricks" to boost NVMe driver performance, the L2 read/write speed should match what you would see with Intel's acceleration.
Zyxel wrote:Q2: If "yes" - how should PrimoCache be set up with Windows 7 x64 in order to rival the performance of Intel Optane System Acceleration with Windows 10 x64 ?...I do not have RAM to spare...
Without spare RAM you would want to avoid L1 caching (set this to zero) and just rely on L2. There's not a huge amount of setup involved (see here for instructions) since most options relate to L1. As to performance, you'll have to test that for yourself - but the trial version allows you 60 days to check. 32GB might be rather small to cache a 4TB hard drive but it is your pattern of data usage that will determine how effective any caching will be.
Zyxel wrote:...is withholding support for older versions of Windows a result of some kind of collusion between Microsoft and Intel aimed at coercing people to migrate to Windows 10 ?
You're talking about an Intel product so it's clearly Intel's decision. However they don't have a monopoly on caching software and PrimoCache is one of several alternatives.
Jaga wrote:No reason not to migrate to Windows 10 at this point.
Hmm - I'm sticking with Windows XP on my main systems. However further debate would be better suited to another thread.
Jaga wrote:If you don't have enough RAM to toss Primocache at least 8gb, stop right here. Small L1 caches won't help a RAM-starved system at all, unless it's an old slow laptop.
PrimoCache can be useful with less (I'd suggest 2GB as a lower limit) but again, it comes down to an individual's data usage. A moderately recent "graphics card intensive" application may only result in 2-4GB cache usage.
Zyxel wrote:Also, RAM is volatile so it cannot support deferred writes like Optane NV memory, without risk of data loss in the event of power outage or OS crash/lockup, thus L1 write-caching using RAM is a non-solution for me....and for most people, too IMO
Primo's L1 caching doesn't defer writes by default - it has to be enabled and then offers a variable (10 second default) timeout. While that does pose an increased risk of data loss, a robust backup system (by which I mean a combination of regular whole-disk image backups combined with automatic file versioning for important data) should cover all possibilities. And people should have robust backup whether they run PrimoCache or not. Being short of RAM is a better reason to avoid L1 (since Windows' file caching is more flexible on memory usage).
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: Can PrimoCache rival the Intel Optane System Acceleration ?

Post by Jaga »

InquiringMind wrote:
Zyxel wrote:...the M2 Optane module is installed and visible in BIOS (in RAID/Optane mode) and in Windows 7 Disk Management. I can even format it and write/read files to/from it....Q1: Can the performance of PrimoCache with Windows7 x64 rival the performance of Intel Optane System Acceleration with Windows 10 x64, in the system outlined above ?
If you can format and read/write in Disk Management, then you should be able to format the NVMe for L2 usage in PrimoCache. Unless Intel's RST is using special "tricks" to boost NVMe driver performance, the L2 read/write speed should match what you would see with Intel's acceleration.
I'm not certain what Romex did to get around issues with the RST driver/software when I reported a fatal bug many years back - perhaps Support can chime in on it. Whatever the case, uninstalling the RST software (and allowing Windows to retain the driver only) is step 1 in getting around any issues. And yes - if you can see the drive in Windows Disk Management, theoretically it should be available for use as an L2 target. Theoretically.

InquiringMind wrote:
Zyxel wrote:Q2: If "yes" - how should PrimoCache be set up with Windows 7 x64 in order to rival the performance of Intel Optane System Acceleration with Windows 10 x64 ?...I do not have RAM to spare...
Without spare RAM you would want to avoid L1 caching (set this to zero) and just rely on L2. There's not a huge amount of setup involved (see here for instructions) since most options relate to L1. As to performance, you'll have to test that for yourself - but the trial version allows you 60 days to check. 32GB might be rather small to cache a 4TB hard drive but it is your pattern of data usage that will determine how effective any caching will be.
Which is why I didn't recommend any L2 in this case - the chances of getting high hit rate if even just 50% of the mechanical drive is populated is very slim. 32GB caching 2TB of data is going to get stuff flushed out of the cache exceedingly often, leading to very low hit rates.

InquiringMind wrote:
Jaga wrote:No reason not to migrate to Windows 10 at this point.
Hmm - I'm sticking with Windows XP on my main systems. However further debate would be better suited to another thread.
I have an old 2003 Dell laptop still running XP, but it's just as a collectible. I can't actually envision trying to run a production system (any type) on XP nowadays. With the stability performance available in Win7, it's pretty much the lowest common denominator for "old OSs" from Microsoft. Feel free to start up that discussion elsewhere here if you like - I'll gladly chime in on why I'd never run an XP system connected to the Internet today.

InquiringMind wrote:
Jaga wrote:If you don't have enough RAM to toss Primocache at least 8gb, stop right here. Small L1 caches won't help a RAM-starved system at all, unless it's an old slow laptop.
PrimoCache can be useful with less (I'd suggest 2GB as a lower limit) but again, it comes down to an individual's data usage. A moderately recent "graphics card intensive" application may only result in 2-4GB cache usage.
Given he's trying to cache a 4TB volume with an unknown population percentage (guessing 50%), a 2-4GB cache would give laughable hit rates. Not even worth trying in my opinion.

InquiringMind wrote:
Zyxel wrote:Also, RAM is volatile so it cannot support deferred writes like Optane NV memory, without risk of data loss in the event of power outage or OS crash/lockup, thus L1 write-caching using RAM is a non-solution for me....and for most people, too IMO
Primo's L1 caching doesn't defer writes by default - it has to be enabled and then offers a variable (10 second default) timeout. While that does pose an increased risk of data loss, a robust backup system (by which I mean a combination of regular whole-disk image backups combined with automatic file versioning for important data) should cover all possibilities. And people should have robust backup whether they run PrimoCache or not. Being short of RAM is a better reason to avoid L1 (since Windows' file caching is more flexible on memory usage).
A L1 write cache might be the most advantageous portion of this solution, though he's not willing to do it. Since you can't get a large enough L1 to cover all needed reads (or even a large enough L2), at the very least you could cache writes via L1 on a shortened timer, to gain on writes. Granted, the NVMe would help accelerate writes a little bit by itself, so it's not a total loss.



Problems I see with any solution here:
  • Windows 7: No support (?) for the hardware in use. Need to check Disk Management in Windows.
  • Size of the NVMe hardware vs size of the mechanical drive: Depending on how full the mechanical is, you'll probably need a MUCH larger cache volume (either L1 RAM or L2 SSD/NVMe) to cover all that data with a decent hit rate. I'm caching 112GB of data on my C: volume with 20GB of RAM. That could just as easily be L2 on another volume. But it needs to be a certain percentage of the cached data size (18% in my case - more cache would be even better). Using 32GB to cache 2TB data means the cache is only covering 1.6% of the data. That's highly inefficient. Even assuming just 1TB of data means cache coverage of only 3.1%.
  • No free RAM: Any caching strategy is going to have to track cached blocks, either on L1 or L2. That takes up a chunk of system RAM, more depending on block size. He'll have to keep block size fairly high in order to have a lower hit to system RAM.
Post Reply