caching behavior with windows built-in cache; and other generic questions Topic is solved

FAQ, getting help, user experience about PrimoCache
SnowReborn
Level 4
Level 4
Posts: 34
Joined: Sun Dec 02, 2012 3:13 am

caching behavior with windows built-in cache; and other generic questions

Post by SnowReborn »

Hello. I been using Primocache for a long time and I am really loving this software, specially with version 4.3 and the freeze feature. couple years ago, I used to use ISLC in conjunction of primocache. ISLC's purpose is to clear window's built-in cache, but with very old forum thread I posted, primocache support recommended to use primocache WITH windows cache. Windows cache is definitely nice due to it's dynamic behavior, but the reason at the time i was using ISLC was solely to track better static from primoecache's GUI to see cache utilization and so on; so with support's recommendation, I stopped using ISLC and started to use primocache WITH windows cache. However, recently with my memory upgrade(128 to 192gb) i've ran into some issues with L1 /L2 caches due to larger initial free memory. The issue I ran into is that, during first clean restart or boot up of the windows session, for files and read / write activity that has not yet been cached(read directly from hard drive) will be cached in for both primocache L1(if has space, if not L2) and windows default cache for unused memory. Now we have 2 copies both in primo and windows builtin cahce. However, when the file was being read again, it is read directly from windows's built-in cache, which is fine; but the issue with windows cache is that, it does not get logged into primocache(read request). So i suspect eventually primocache will kick out that copy of the cache because it was "never used" and was being served by windows cache. So now I only have 1 copy of the cache stored in windows builtin cahce. Then once my memory utilization goes up(i have drastic memory usage pattern, sometimes I maxes out 170GB, sometimes it lowers back to 70GB), windows frees up memory by deleting caches, then my last copy of the cache also gets kicked out. So now when I try to read the file again, I have no cache and is reading directly from the disk again; and that cycle repeats, then eventually I am just dealing with window's caching and fighting for free memory.

I would like to know primocache's caching behavior, as well as cache preservation behavior. Let's keep it simple WITHOUT defer write, and cache is configured in default mode(read write mixed). From my observation correct me if i am wrong, primocache will save anything regarding disk activity that it can record(excluding read activity served by windowsd builtin cache since it doesn't see it), into L1 as long as it has space, but things I would like some explanation is that, once L1 is full, how 's the L2 cache going to store any data and cache, what's the behavior(it seems different from L1), and how does primocache decide which cache to kick out first for both L1, and L2? is it FIFO? LIFO? or LFU? under what condition?(L1 L2 full? or there is a timer to kick it out dependent frequency of being used?) If I am not mistaken, I have also seen behavior that despite my L2 is not full(200GB space left) I see primocache releases some of the cached content(is it because the cached target file was physically deleted from the disk? or is because the algorithm has decided it hasn't seen any read request for that cache for a long time, and is never being used, so it gets deleted?) because if target(original file on disk) were deleted, then the cache probably serves no purpose and should be deleted, that's fine; However, if the cache was deleted because it was not being used, while there are still plenty space on the L1 or L2, then that's not a behavior I want to see. I am again tempted to use ISLC to clear windows memory cache to ensure primocache to read all the data and store them properly so I don't have to deal with reading from hard disk when my RAM utilization is very high and windows clears all it's cache.

Another question is, I understand standard license can be upgraded to pro license, but can standard / pro license be able to upgrade to server license? or is sever license separate and has to be bought separately? Thanks!
SnowReborn
Level 4
Level 4
Posts: 34
Joined: Sun Dec 02, 2012 3:13 am

Re: caching behavior with windows built-in cache; and other generic questions

Post by SnowReborn »

I forgot to mention another strange / potential problematic behavior I observed. It seems like if I have large amount of defer write data being "infinite" defer write(i am well aware of the risk but I have the need and proper risk management(PSU, stable system, image backups, etc)). Let's say I have accumulated 30GB defer write data in L1, but I forgot to manually delete the cache task, or manually flush it; and proceeding to restart my windows. Then Windows will start to close down all the application, and shows "restarting", and primocache will automatically engage in flushing the defer write tasks to the local disk. This behavior is fine, but the issue is that, primocache is only flushing, on the windows's blue screen(NOT BSOD) of spinning "shutting down" or "restarting", at this point there is nothing the user can do to stop windows from trying to restart while waiting for primocache to flush; and sometimes this process can be long dependent on the amount of unflushed data. I think if the time it take is too long, say like 10 minutes, or 15 minutes, windows itself will think something is wrong and it cannot be properly restarted, and raises a BSOD(blue screen of death), the exact message I don't remember, but I think I've encountered this issue 3 times. Any input or advise would be greatly appreciated!
tverweij
Level 6
Level 6
Posts: 74
Joined: Thu May 10, 2018 9:27 am

Re: caching behavior with windows built-in cache; and other generic questions

Post by tverweij »

For the flush on shutdown:

You can set the time that Windows will wait for services to shut down, before it starts killing them.
The registry entry is: [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control] and the value (32 bits DWord) WaitToKillServiceTimeout

The value is the time that windows will wait in ms, the max value is one hour - if it is higher, the wait time will be 30 seconds.
Without this value, the system will wait only 20 seconds before killing it unless the service itself asks for more time.
tverweij
Level 6
Level 6
Posts: 74
Joined: Thu May 10, 2018 9:27 am

Re: caching behavior with windows built-in cache; and other generic questions

Post by tverweij »

SnowReborn wrote: Thu May 11, 2023 9:11 pm how does primocache decide which cache to kick out first for both L1, and L2? is it FIFO? LIFO? or LFU?
That is a really good question that I would like the answer on too.

Based on this question I'd like to propose a new cache strategy, especially for servers with lots of available RAM: the Most Often Used scenario..
This scenario prevents the cache from being trashed by backups and other big reads that occur one in a while.

This scenario needs a map of the complete disk where the disk is split into cache-size blocks.

This map tells how often each disk block is read; every read will increase the number (32 bit unsigned integer) of the block by one. When the max value is reached for a block, all numbers will be decreased by 2^31 (if a block has a lower number it will be reset to zero).
These values should be decreased by one every 10 milliseconds, to make sure that the score is removed over time.

For a 32k block cache this will cost about 32 Mbyte RAM per Tb of disk space.

For a disk block to become cached, it should reach a value that is higher than the lowest value of a currently cached block (or when there is room for new blocks). In a read/write shared scenario, written blocks should stay cached until they are replaced by reads of other blocks and should not use more than 50% of the cache (unless there is more available space). In a separated read write cache scenario, the write cache should use FIFO.
Nick7
Level 5
Level 5
Posts: 46
Joined: Sun Jun 25, 2017 7:50 am

Re: caching behavior with windows built-in cache; and other generic questions

Post by Nick7 »

... or simply look how ZFS does it, as it combines both MRU (Most Recently Used) and MFU (Most Frequently Used).
tverweij
Level 6
Level 6
Posts: 74
Joined: Thu May 10, 2018 9:27 am

Re: caching behavior with windows built-in cache; and other generic questions

Post by tverweij »

Nick7 wrote: Fri May 12, 2023 2:51 pm ... or simply look how ZFS does it, as it combines both MRU (Most Recently Used) and MFU (Most Frequently Used).
Whats the fun in that? :D
Yes, you are right - don't invent the wheel.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: caching behavior with windows built-in cache; and other generic questions

Post by Support »

SnowReborn wrote: Thu May 11, 2023 9:19 pm This behavior is fine, but the issue is that, primocache is only flushing, on the windows's blue screen(NOT BSOD) of spinning "shutting down" or "restarting", at this point there is nothing the user can do to stop windows from trying to restart while waiting for primocache to flush; and sometimes this process can be long dependent on the amount of unflushed data. I think if the time it take is too long, say like 10 minutes, or 15 minutes, windows itself will think something is wrong and it cannot be properly restarted, and raises a BSOD(blue screen of death), the exact message I don't remember, but I think I've encountered this issue 3 times. Any input or advise would be greatly appreciated!
As far as I know, PrimoCache has implemented an mechanism to prevent Windows from raising a such BSOD before the flush completes.
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: caching behavior with windows built-in cache; and other generic questions

Post by Support »

SnowReborn wrote: Thu May 11, 2023 9:11 pm From my observation correct me if i am wrong, primocache will save anything regarding disk activity that it can record(excluding read activity served by windowsd builtin cache since it doesn't see it), into L1 as long as it has space, but things I would like some explanation is that, once L1 is full, how 's the L2 cache going to store any data and cache, what's the behavior(it seems different from L1), and how does primocache decide which cache to kick out first for both L1, and L2? is it FIFO? LIFO? or LFU? under what condition?(L1 L2 full? or there is a timer to kick it out dependent frequency of being used?) If I am not mistaken, I have also seen behavior that despite my L2 is not full(200GB space left) I see primocache releases some of the cached content(is it because the cached target file was physically deleted from the disk? or is because the algorithm has decided it hasn't seen any read request for that cache for a long time, and is never being used, so it gets deleted?) because if target(original file on disk) were deleted, then the cache probably serves no purpose and should be deleted, that's fine; However, if the cache was deleted because it was not being used, while there are still plenty space on the L1 or L2, then that's not a behavior I want to see. I am again tempted to use ISLC to clear windows memory cache to ensure primocache to read all the data and store them properly so I don't have to deal with reading from hard disk when my RAM utilization is very high and windows clears all it's cache.
L2 caches data differently than L1. L2 caches data asynchronously. Even data has already been cached in L1, it still can be cached in L2. For reading cache, we use a dedicated LFU-based algorithm for both L1 and L2. If a file is deleted, its corresponding cached data will also be removed from the cache.
SnowReborn
Level 4
Level 4
Posts: 34
Joined: Sun Dec 02, 2012 3:13 am

Re: caching behavior with windows built-in cache; and other generic questions

Post by SnowReborn »

tverweij wrote: Fri May 12, 2023 12:46 pm For the flush on shutdown:

You can set the time that Windows will wait for services to shut down, before it starts killing them.
The registry entry is: [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control] and the value (32 bits DWord) WaitToKillServiceTimeout

The value is the time that windows will wait in ms, the max value is one hour - if it is higher, the wait time will be 30 seconds.
Without this value, the system will wait only 20 seconds before killing it unless the service itself asks for more time.
Thanks for the reply! However, like what support mentioned, it seems like windows is only raising BSOD under extreme wait times. not 20 seconds, not even 5 minutes.
SnowReborn
Level 4
Level 4
Posts: 34
Joined: Sun Dec 02, 2012 3:13 am

Re: caching behavior with windows built-in cache; and other generic questions

Post by SnowReborn »

Support wrote: Sun May 14, 2023 6:24 am
SnowReborn wrote: Thu May 11, 2023 9:19 pm This behavior is fine, but the issue is that, primocache is only flushing, on the windows's blue screen(NOT BSOD) of spinning "shutting down" or "restarting", at this point there is nothing the user can do to stop windows from trying to restart while waiting for primocache to flush; and sometimes this process can be long dependent on the amount of unflushed data. I think if the time it take is too long, say like 10 minutes, or 15 minutes, windows itself will think something is wrong and it cannot be properly restarted, and raises a BSOD(blue screen of death), the exact message I don't remember, but I think I've encountered this issue 3 times. Any input or advise would be greatly appreciated!
As far as I know, PrimoCache has implemented an mechanism to prevent Windows from raising a such BSOD before the flush completes.
Thanks Support, It seems like I only encountered BSOD under very long flush, that is probably more than 10+ minutes. I am not sure under such extreme condition would have unexpected behavior even with the mechanism implemented, I don't know if I would want to try to reproduce the issue, but maybe I was thinking, it would be probably better implementation if primocache can detect a restart / shutdown command has been issued, and give a pop up or intercept the command before user gets locked in blue "restarting" or "shutting down" phase where nothing can be done.
Post Reply