[2017-05-27] PrimoCache 3.0.0 Beta is available now!

First hand news related to PrimoCache
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by InquiringMind »

I have to confess I didn't test this before posting (away from my main PC for the next couple of days), but clearing Windows cache would certainly be necessary for a proper test - Task Manager should show the current size of the System Cache so you may be able to check whether that changes (though 6.23GB/15 secs comes to 425MB/s, pretty quick even for uncached SSD). Task Info has a couple of option (Free RAM Fast and Free RAM Slow) which can achieve this, but I've only used it on 32-bit systems (where the cache is limited to around 2GB).

I'd certainly agree that being able to disable Windows own caching is highly desirable - aside from duplication of caches there is also the problem of when it relinquishes memory to applications. On a long running system (where Windows cached data has dropped out of PrimoCache resulting in both caches having different contents), this would then result in the most frequently used data (up to 2GB on a 32-bit system, more on 64-bit) no longer being cached. Unfortunately, there seems to be only one way of doing this which has several complications.
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by InquiringMind »

Did a test earlier and both creating and verifying a checksum file appears to use PrimoCache (I saw available cache dropping on creation and cached reads rising during verification). If using this solely/mainly to load data into cache, then I'd suggest creating a CRC32 file as this has the lowest CPU overhead.
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by Jaga »

Seems then like using a high-RAM system (mine is 64 gig) defeats that purpose, and has Windows utilizing it's cache before Primo gets a chance. I saw zero (literally no cache) use with Primo while trying a crc verification. And as I have no way to disable the Windows 10 disk cache, it's back to the original request of building it into Primo.
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by InquiringMind »

Jaga wrote:Seems then like using a high-RAM system (mine is 64 gig) defeats that purpose, and has Windows utilizing it's cache before Primo gets a chance. I saw zero (literally no cache) use with Primo while trying a crc verification. And as I have no way to disable the Windows 10 disk cache, it's back to the original request of building it into Primo.
In that case, you shouldn't be seeing any benefit from PrimoCache at all, as Windows' caching would come into play with any disk access.

You mention having 64GB RAM, but what matters is how much you have free (after allocating RAM to PrimoCache and any used by applications/OS). If you have a lot free, then increasing the amount allocated to PrimoCache (thereby reducing the amount available to Windows' own cache) may be beneficial.

Another possibility to check is whether you are running the 32-bit or 64-bit version of HashCheck - try starting a 32-bit version of Windows Explorer (or whatever alternative file manager you prefer) and see if the results differ.
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by Jaga »

Primocache already eats over half of the total system RAM. The cache task on the volume used for testing (C:) is 24 gig in size, which should be sufficient. After allocating RAM to Primo at startup, available system memory sits around 50%.

My explorer runs as a 64-bit process, and Hash Check shows up as a child entry in Task Manager when it is run. I can only assume then that it's utilizing the 64-bit DLL version of Hash Check. Though I don't know how that would matter for disk access - if it's running a TRUE MD5 hash comparison, it has to access every bit of every file it checks, which requires reading from "disk" (whether the actual or in-memory disk data).

I'll re-try the test, but I can tell you with certainty - I cleared Primo's volume cache on C:, and re-initiated a MD5 check on over 6 gig of files (and a ton of them in quantity). It should have hit at least one file for reading, and Primo still showed zero reads.

Edit: No luck on the test. A 1.15mb hash check file on ~2 gig of files/folders showed no file reading in Primo on the cache task or the cache volume. There was a 4.5kb read during that time, which I can only assume was from Windows or an already-loaded program.

The Hash Check software simply isn't forcing any files to be read from the drive. I would like it if it worked, but at this point it doesn't appear to be reliable.

Going to leave my request for the feature to be built into PrimoCache, as the best solution for pre-caching files/folders.
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by InquiringMind »

Jaga wrote:...I can only assume then that it's utilizing the 64-bit DLL version of Hash Check. Though I don't know how that would matter for disk access - if it's running a TRUE MD5 hash comparison, it has to access every bit of every file it checks...
Speculating wildly here, the 64-bit version of Windows Explorer could be accessing the disk in a way not intercepted by PrimoCache (e.g. using its own driver - even so, you should see Windows' cache, reported under Physical Memory/Cached in Task Manager, increase as checksums are created/checked).

If the 32-bit version does show cached reads in PrimoCache on your system, then that would confirm this hypothesis - and highlight a possible limitation with PrimoCache.
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by Jaga »

I don't see any way to launch a 32-bit version of Explorer.exe on a 64-bit version of Windows 10. I was somewhat thorough in researching a method, even the one posted here did not work. Neither do I see a way to force the OS to use a specific version of Hash Check through a shortcut - it didn't make any and doesn't have an obvious install directory.

I think we're trying to "herd cats" here. A workaround like Hash Check would be nice, if it worked under all circumstances and was reliable. Unfortunately it either seems to be unreliable, or to allow Windows to circumvent 3rd party tools and go straight to the Windows' file cache. You wouldn't think that with only 32 gig free RAM, Windows would tolerate >6 gig read into it's cache and give no reads at all to Primo with an empty cache task. To me that's unheard of behavior.

A simple shell extension for Primocache with a "load this now" entry to do a straight "read to volume cache" command would solve the request easily. It wasn't what Primo was originally designed for, but it does have a large use, and plays in directly to how Primo helps speed up systems. Basically it's serving as a human-directed read-ahead, for when we know what we want cached before Primo does.

The only other way to accomplish it directly, is to copy/move the files to another volume, move back if needed, and then hope that Primo cached/retained all the files it read. It creates unnecessary writes and wait time, and isn't a graceful solution when compared to a built-in feature.

But I digress - we're somewhat hijacking the topic now for discussion around an undiscussed and unplanned feature (request), with no comment from Support on it. I'll wait for 3.0 to land proper, and then perhaps make the request in another topic. I still see it as directly within the system functions that Primo handles as useful and appropriate.
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by InquiringMind »

If HashCheck doesn't work on your system (as it does on mine), then any similar feature provided by PrimoCache may encounter a similar issue. If none of the options for running a 32-bit version of Windows Explorer work on your system, then try a 32-bit version of Total Commander (this will offer the same right-click options as Explorer does).

Edit: Also, according to your post above, you've allocated 24GB (out of 64GB) to PrimoCache and have about 32GB free (and usable for Windows' file caching). As an experiment, try allocating more memory (say, 50GB, which should leave around 6GB free) to PrimoCache since that should then leave less for Windows' caching.
escaward
Level 3
Level 3
Posts: 17
Joined: Thu Aug 01, 2013 1:29 am

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by escaward »

Hello,

Is PrimoCache 3 close to final release? I am looking forward to trying it out but I would prefer not to use a beta version at this time.

Regards,
Eric
geneo
Level 4
Level 4
Posts: 27
Joined: Sat Oct 31, 2015 5:19 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by geneo »

Yes I am interested if the final is near too. This has been asked many times. Is anyone minding the store? :D
Post Reply