Primocache stops writing to L2 after a while

[Pentium100]

Hello, I am trying the trial version of PrimoCache 1.0.1 and initially I set it to use RAM only - that worked perfectly. Now I borrowed an SSD to test whether I should buy one. I created a single partition on it (55GB) and set it as L2 cache. At first i seemed OK, but then Primocache stopped writing to the SSD after about 400mb or continued to write very slowly, even though the disk was being read at ~4MB/s). After reboot it works for a while and then stops. It seems that the writing stops, but reading continues.

I have set it to zero L1, 64KB block, Read data only.

OS: Windows 2012 R2 x64
SSD: Intel 330 series 60GB configured as JBOD on Adaptec 5805Z

Performance counters are OK as I have been using the Performance monitor to see how the SSD and HDD is accessed (then I tried without starting the Performance Monitor after reboot - no change).

Thinking that there is a problem with my SSD, I plugged in a 146GB HDD to use as cache (good thing that Primocache does not require a SSD). The "L2Storage Write" counter is equal to the "Total Read" counter up to about 600MB, then the writing slows down. Now it's 8GB read and 700MB written. And the HDD (or SSD) is idle all this time, the light comes on once in a few seconds even though the cached disk is read continuously.

(I will update this post as i find out something new)

I tried connecting the SSD directly to the motherboard (as opposed to the RAID controller) - no difference at all.

So, for the 146GB hard drive, the stop occurs at about 600MB, for the 45GB SSD partition, the stop occurs at ~350MB (some relation to the size?). However, once the stop occurs, the only way to restart the cache is to reboot the PC, if I just reset or delete and recreate the cache task it does not work (2.5MB written for 1.6GB read), even if I choose a different device for cache.

Also, it does not seem to matter whether I first create the cache task and then restart the PC or first reboot the PC and then create the cache.

OK, I created a RAID0 array from two 146GB hard drives (to see it this is capacity related). I also noticed that the stop is not abrupt, instead the writes gradually slow down after reaching about 400MB and then essentially stop. Also, with the 270GB array it also stopped at about 600MB/s. So, increasing capacity does not seem to make a differece, how about decreasing it...

Created a 10GB partition on the SSD, the stop occurred at 400MB. So, it's 400MB for the SSD and 600MB for the HDD and does not really depend on capacity.

[Support]

I think this may be caused by busy source disk. In order to not affect the disk perfomance, PrimoCache only writes data to L2 cache when source disk is idle.

Also please make sure that all volumes in source disks that you want to be cached have been added to a cache task. If reads happen on a volume which is not added to a cache task, no data on this volume will be stored into L2.

Just for your reference.
http://www.romexsoftware.com/en-us/prim ... cache.html

[Pentium100]

That's interesting - so it writes to L1 cache all the time (does not matter whether source disk is busy or not), but only writes to L2 if source disk is idle. Would having some L1 cache speed up the filling of L2 (because I would expect it to write to L2 from L1). I added 8GB of L1 cache to see how that works out.

OK, so if the source disk is overloaded (one of the reasons why I would want a large cache) I have to build the L2 cache gradually, reducing the load on the source disk until the cache is full (or at least the most accessed blocks are cached).

EDIT: Well, it's true. I stopped the program that accesses the disk a lot and the cache started filling up (not very fast though). It still is weird - I mean if L1 cache is filled up immediately, that means that Primocache knows about the contents on first read (and does not need to do a second read just for caching), but it still does a second read to write to L2. So, if I expect to saturate the source drive, I have to periodically interrupt the activity to let Primocache catch up. Weird. Also, adding L1 cache did not help - sure it filled up and worked, but L2 was not filling up any faster (so, if the source disk is saturated, Primocache won't write from L1 to L2 - both of which have nothing to do with the source disk).

Interstingly, for the first few hundred megabytes after reboot, Primocache does not care about the source disk load (I tested this by first loading the source disk and the creating the cache).

[Jaga]

I think this may be caused by busy source disk. In order to not affect the disk perfomance, PrimoCache only writes data to L2 cache when source disk is idle.

This is a problem. My C: drive is almost *never* completely idle. It has tasks that read/write to it in tiny bits every second, almost continually. It may explain why my L2 has failed to load more than 500mb despite 2 days of work/idle time. Essentially PrimoCache has stopped functioning for me - even the L1 is only 25% loaded.



I have a separate topic open here about the L2 to discuss.

[Pentium100]

L1 for me fills up as soon as the data is read. If I read a 7GB file from a disk, the free space in L1 shrinks by 7GB. However, I need more cache (and it does not need to be very fast, just lower latency than a hard drive) and RAM is expensive, so I would rather use a SSD. However, the L2 cache strategy makes PrimoCache kinda useless - Let's say I am constantly random reading a 25GB collection of files as fast as possible. Ideally, PrimoCache should cache the blocks that make up the files to L2 immediately so after every block is read once, the subsequent reads should come from SSD. I would get low performance at first (until every block is read once and cached) and high performance later. Now, PrimoCache does not write blocks to L2 unless the source disk is almost idle, which means that I get low performance for a much longer time.

With L1, PrimoCache behaves exactly like it should be. It's just L2 that is weird.

OTOH, I found another program that seems to work correctly - VeloSSD (SSD only) or MaxVeloSSD (SSD and RAM). While it looks worse than PrimoCache and does not have the nice statistics, it writes to cache at the first read (if I am reading from the source disk at 5MB/s, the cache is being written at 5MB/s).

[Davey126]

even the L1 is only 25% loaded.

It's possible I am misinterpreting the data but it appears L1 is populating at a reasonable rate. Of the 7.68GB read in your screen snip some portion is likely from the system page file which Primocache excludes. Cached reads are ~4GB which closely mirrors the amount of data in the L1 cache. In short, L1 is probably only 25% loaded because that represents all the of the blocks directly from the source disk since the cache was last started. Give it some time - L1 should continue to fill as fresh blocks are read from the source disk.

That said, I fully agree the sluggish population of L2 is a significant problem and really messes up the value proposition of PrimoCache. Unless you have a ton of memory dedicated to L1 PrimoCache is essentially useless for high volume operations on busy systems. Let's hope the developers due some tuning for the next point release. As others have noted there seems no good reason why L1 contents can not be promptly copied to L2 provided the processor and/or data bus are not saturated.

[Jaga]

I've been using PrimoCache since it was called FancyCache Davey - it used to behave much differently than it does now with 1.0.1. Something changed recently, and not for the better. The L1 cache used to fill up VERY quickly, and the L2 used to fill to at least 75% in the course of one day. That screenshot I posted was after at least an hour's worth of use after successive boots, so both the L1 and L2 should have been significantly populated.

I just booted this morning and the L2 is still sitting at only 500mb of 30gb filled (roughly 1.7% ?), it only read 5.64mb from the L2 for the boot total. And the L1 for me yesterday never filled more than 30%, all day long.

Here is what it looks like while I'm typing this post out - 30 minutes after booting the machine:



Something is seriously wrong with the software in 1.0.1 from what I'm seeing, based on comparisons with earlier versions.

I remain hopeful that things will get fixed, as it ran well for quite some time during development. But it is unfortunate it's "gone paid" now and there are bugs that prevent it from being useful at all.

[Davey126]

I've been using PrimoCache since it was called FancyCache Davey - it used to behave much differently than it does now with 1.0.1.

Not challenging that ! Clearly something isn't right with L2 population in version 1.x regardless of how it performed before (FancyCache days). L1 population still looks ok based on the figures presented in your latest screenshot (~1GB read; ~1GB cached). Of course, PrimoCache may be under representing the amount of data read. Then all bets are all.

I wonder if Windows caching is skewing some of the PrimoCache numbers. If Superfetch caches a file does PrimoCache see it? I honestly don't know ...

[Jaga]

Don't know about Superfetch - one of my criteria for thoroughly checking PrimoCache recently was to completely disable Superfetch. There's no point in populating both memory and memory-disk-cache with the same sort of data, unless that is partly what L2 keys off of (I really don't know).

Would be nice to have Support chime in and let us know what kind of tests we can do, or other things we could be checking.

[Support]

Well, we have noticed this issue in L2 cache when souce disks keep busy. I think a tuning will be made in the next version.
BTW, when populating L2 cache, PrimoCache does get data from L1 cache instead of source disk if L1 cache has already got the data.

Thanks all!