Freeze problems when using primocache

[Jaga]

Just a quick update. Am now using 2.4.0, and have been for about a week - no problems. I think the issue was a 'phantom' one. Re-installing Windows 10 was a good idea, and I would recommend it to anyone whose machine has been 'upgraded' to Windows 10 from a previous Windows version. Thanks to support.

Just a comment about re-installing Windows 10 clean:

Before doing so you should double-check and make SURE your version of W10 is activated with your Microsoft account, so that when you go to login to your new 'clean' installation with that MS account info, it automatically activates it for you. If your activation info isn't tagged onto your MS account, it will NOT re-activate cleanly, and you'll end up having to call MS on the phone for a manual activation. Sometimes they deny them.

[gotterdammerungly]

Just a quick update. Am now using 2.4.0, and have been for about a week - no problems. I think the issue was a 'phantom' one. Re-installing Windows 10 was a good idea, and I would recommend it to anyone whose machine has been 'upgraded' to Windows 10 from a previous Windows version. Thanks to support.

Just a comment about re-installing Windows 10 clean:

Before doing so you should double-check and make SURE your version of W10 is activated with your Microsoft account, so that when you go to login to your new 'clean' installation with that MS account info, it automatically activates it for you. If your activation info isn't tagged onto your MS account, it will NOT re-activate cleanly, and you'll end up having to call MS on the phone for a manual activation. Sometimes they deny them.

Good advice indeed. I did discover all of that in the reading I did before proceeding, and since my Win10 is Microsoft account activated, I did not have a problem. Of course before doing any significant OS operation one should avail oneself of all the relevant information; in the end I was surprised at how painless and relatively quick it was, and my Win10 installation now seems happier than before. I used to do it all the time with Win 98!! Those were the days....

[InquiringMind]

You need an online account to use Windows now? Good grief, makes me glad I've stuck with XP/2K on my main systems.

[gotterdammerungly]

You need an online account to use Windows now? Good grief, makes me glad I've stuck with XP/2K on my main systems.

Luddite!
I still have an XP system too, to run out-dated but still very good audio hardware that is not compatible with modern hardware or OS's, and it does its job very well, but I wouldn't want to use it all the time....progress has definitely been made in many areas - although I think there have been backward steps too. Online activation and its related issues can be, I agree, a major PITA, which is one reason I go for open source and/or non-copy-protected software whenever possible. I suppose I should be using Linux, but Windows is the devil I know. However, if MS continue to make Windows increasingly difficult to control I might jump ship one day.

[InquiringMind]

Ugh, me hit you with XP club!

My main reasons for sticking with XP are the availability of top-notch security software allowing me to have total control over what is run and what it can do (System Safety Monitor and Process Guard, both no longer available alas, though a copy of SSM can still be found with a generic license key) which need to hook into the kernel/SSDT to function (and therefore would have to bypass PatchGuard on 64-bit systems).

Then there is the activation issue which started with XP - though for that the bypass is simple and straightforward (later Windows versions seem to require BIOS modification to appear like OEM systems).

And then there is the architectural ugliness of 64-bit Windows which insists on splitting programs between 2 "Program Files" folders and having 64-bit code in \Windows\System32 and 32-bit code in \Windows\SysWOW64...

As for Linux, it lacks firewall software that works on an application level (e.g. able to create and enforce network access rules like "Allow Firefox to send traffic to port 80 but block anything else") - TuxGuardian is the closest I've found but that doesn't allow detailed rules, only yes/no network access.

[npelov]

I want to get back to the freeze problem. I made some tests. While for SSD it's nto a problem, most flash drives can't read and write at the same time very well. I added a USB3.0 flash drive to OS drive cache and I tested it's read speed while primo cache was writing to L2 (see read-07-September-2016_11-32.png). As you can see there are some significant drops in speed (to about 1 MB/s) and one drop is for more than 5-10 seconds. The drops are caused by writing. I've tested many flash drives before. They all have this problem. The cause is probably limited chipset to reduce cost/power draw/space. I attached a read test without anything writing to see the real flash performance (only-read-07-September-2016_11-46.png).
Keep in mind that these numbers are for sequental read. It may get worse for many smaller random reads.

The file "only-read-07-September-2016_11-46.png" is taken while the usb flash wasn't used. There are no significant speed drops. Dropping to 50MB/s is probably some feature/bug of the flash (maybe the flash is faster at the start for marketing purposes), but there is big difference between 50MB/s and 1 MB/s

My suggestion is to put 2 more options for gather interval:
At shutdown - the contents are only written at shut down or manually when you click a button.
Manual - the contents are never written automatically to L2 cache.

And add a "write contents to L2 storage" button. When clicked L2 contents are flushed to flash/ssd storage.

Keep in mind that this is just an additional option. It won't change the current behaviour unless a user selects it.

Meantime what has to be written in L2 can be stored in RAM in a structure similar to "prefetch" (adress ranges?) - it doesn't take a lot of space.

Let me know what you think.

[Support]

@npelov, well, that's true. Usually, the native performance of flash drives is not as good as that of SSDs, especially when reading and writing at the same time. We're also thinking about this problem and may have some optimizations in future versions. And thank you for your suggestions for this problem, we'll consider them.

[InquiringMind]

...As you can see there are some significant drops in speed (to about 1 MB/s) and one drop is for more than 5-10 seconds. The drops are caused by writing. I've tested many flash drives before. They all have this problem. The cause is probably limited chipset to reduce cost/power draw/space...Dropping to 50MB/s is probably some feature/bug of the flash (maybe the flash is faster at the start for marketing purposes)...

The cause seems likely to be erase-block related (the need to wipe a 16KB block to free it up for future 4KB writes). Might be worth checking if TRIM is enabled (if available on USB drives) but for SATA-connected SSDs, background garbage collection seems to have removed the need for TRIM (at least in my experience).

In such a case, it might be worth prioritising the caching of smaller files (where the initial access time has a greater impact on HDDs) over larger ones (where the transfer rate matters more). Having more options for delayed L2 writes increases the likelihood of data corruption in the event of power failure or system crash.

[npelov]

well, making primocache filesystem-aware will open a lot of doors. I would be glad to be able to set rules for how files are cached. Currently I have troubles when watching movies - firstly they override the important cache (I want to exclude them from the cache) and sencondly they write a lot of data to L2, which is USB flash on my laptop.

I meant delaying what to be "cached" on L2, not "written". I something is not cached it won't be corrupted. Although it depends on implementation.

[InquiringMind]

...Currently I have troubles when watching movies - firstly they override the important cache (I want to exclude them from the cache) and sencondly they write a lot of data to L2, which is USB flash on my laptop...

Suggestion - movies (plus other media and program/game installers) don't benefit from caching, as long as your storage media is fast enough for playback - place them on a uncached volume or disk to avoid wasting cache space.

Editing video is a separate issue where caching can help out, but not playback. As long as your disk/SSD/flash drive can handle the data rate required (the highest, for UltraHD 4K HEVC level 6.2, High tier is around 90MB/s), there is no benefit to speeding it up further.