[2017-05-27] PrimoCache 3.0.0 Beta is available now!

First hand news related to PrimoCache
Davey126
Level 7
Level 7
Posts: 99
Joined: Sun Mar 23, 2014 3:40 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by Davey126 »

sinus wrote:Any assumptions in science and practice always either are confirmed experimentally, or are rejected. Without having results of experiments, there is no sense to argue because it is not soccer and not policy. The experiment must be clean and clear: established checkbox of "Enable Defer-Write", the "Latency" value is enable, in "Advanced Defer-Write Options" select Write Mode "Intelligent", "Idle-Flush" or "Buffer" (with condition of "Windows idle"). Some background task is carried out not to allow "Windows idle", and start of a procedure, for example, copy files on cached disk; (total size of files has to be more than Cache Size).
Thank you for a grade school introduction to the scientific method along with the implicit suggestion such practices are not being followed. I have used PrimoCache on multiple workstations for several years and have closely observed behaviors with various settings over many hours. As previously stated it is my belief that all of the write commit methods generally conform to the flush methodology stated in the accompanying documentation.

By way of example I have a 256 MB L1 60s write only cache established on a laptop configured for 'average' commits. Data volumes regularly exceed the cache capacity resulting in urgent writes. However, during quieter periods it is easy to observe the cache flush behavior with the various write modes. All appear to work as designed (but see footnote, below).

Why such a small cache and accompanying long retention interval? Boils down to the number of small incremental writes which are often temporary or redundant. The automatic trim and disembiguation of cached data results in a 30-40% reduction in actual writes to physical media. I could configure a larger cache but the incremental benefits would not offset the loss of physical memory which is better used by the host operating system.

Analyzing behavior on a smaller scale is often easier although findings/results do not always scale. My experience with huge L1 caches (>4 GB) is quite limited. However, I have consistently observed behavior that is in line with the above example with L1 caches ranging from 128 MB to 2 GB.

Footnote on Native, Intelligent and Buffer: each of these write modes leverage Windows "idle" time to commit data. However, many conditions must be met for Windows to be "idle" which may not be consistent with casual observation. Anyone who has used Task Manager or extended sleep timeouts has experienced seemingly unexplained delays as Windows frequently resets the idle timer(s) due to background activity that may not be noticeable.
sinus
Level 3
Level 3
Posts: 19
Joined: Sun Aug 20, 2017 10:24 am

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by sinus »

Jaga, You right. But more than 32 Gbytes of RAM can address not all systems. And, forgive for banality, however there will always be non enough memory. Instead of accumulation of memory size it is better to optimize available. Here some compromise is necessary.
We like PrimoCache because it helps to increase a data exchange rate, but any urgent record in case of the crowded cache is a time delay. You understand that increase of RAM size defers the moment of urgent saving data, but doesn't cancel it.
InquiringMind
Level SS
Level SS
Posts: 477
Joined: Wed Oct 06, 2010 11:10 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by InquiringMind »

Just given this a try and the big improvement is the "write amplification" problem is gone, which is a relief (I was seeing figures as high as 40GB written for 8GB of data). However I would expect the quantity of data written to be a multiple of the PrimoCache block size (64K in my case) or at least Windows NTFS cluster size (4K) so I would be interested in an explanation of the following screenshot showing 172.5KB having been written.

Performance seems slightly improved for writes, but nothing perceptible. Downsides are that the graph is still as useless as ever (the cumulative cached percentage can be shown as a single figure, not needing a graph - if a graph is to be used, have it show something more useful like the percentage read from cache per minute over the previous x minutes).

I'd also give a thumbs-down to the garbage PrimoCache writes to the Windows registry (since version 2.2.0) making its keys undeletable by standard means.
Attachments
PrimoCache300.png
PrimoCache300.png (29.64 KiB) Viewed 5677 times
Axel Mertes
Level 9
Level 9
Posts: 180
Joined: Thu Feb 03, 2011 3:22 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by Axel Mertes »

Hi!

I have not been visiting the forum since a while, so I missed Beta 3.0.0 at the beginning. Now I have downloaded and installed it.

Are any blocks written to a read & write cache enabled device are immediately available for reading as well?
Think of rendering an image sequence and wanting to play that back immediately afterwards. So my hope that writes are going through write caching, those cached blocks will be preserved in cache and when I try to read these again, the blocks are already in the cache and will be served from the cache.

Is that now implemented in PrimoCache 3.0.0 beta?

Further, can we use volumes bigger than 2 TBytes for SSD caching?

I am about to buy one of these:
Amfeltec Squid
http://amfeltec.com/products/pci-expres ... d-modules/
http://barefeats.com/hard220.html

Highpoint 7101B oder 7101B-80 mit 8 TByte
http://www.highpoint-tech.com/USA_new/s ... erview.htm
https://www.newegg.com/Product/Product. ... 6816115217
https://www.hardwareluxx.de/index.php/n ... yte-s.html

ASUS
https://www.hardwareluxx.de/index.php/n ... -ssds.html
https://www.youtube.com/watch?v=PTibIjzm15I

(Amfeltec available since a long while, Highpoint seems to hit the stores now, ASUS unknown)

and add several Samsung 960 Pro M.2 cards to it, all to be used for caching on our server. So 8 TByte isn't out of reach at all. So I want to know if and how we can increase cache size beyond 2 TByte... with PrimoCache 3.0.0?
Axel Mertes
Level 9
Level 9
Posts: 180
Joined: Thu Feb 03, 2011 3:22 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by Axel Mertes »

...and another thing I want to see was preloading of "n" blocks after the last requested block.

As I frequently defragment my volumes (the biggest 64 TBytes in one single volume/piece - defragmentation runs hell fast with PrimoCache...) my data is mainly sequential (audio/video production). So implementing read ahead with customizeable "n" blocks would be a game changer, as I do no longer rely on third party software to implement (or mostly not implement) meaningful read ahead caching for smooth and reactive work.

I would expect this to help about anyone dealing with large files signficantly.
minhgi
Level 10
Level 10
Posts: 256
Joined: Tue May 17, 2011 3:52 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by minhgi »

Axel Mertes wrote:...and another thing I want to see was preloading of "n" blocks after the last requested block.

As I frequently defragment my volumes (the biggest 64 TBytes in one single volume/piece - defragmentation runs hell fast with PrimoCache...) my data is mainly sequential (audio/video production). So implementing read ahead with customizeable "n" blocks would be a game changer, as I do no longer rely on third party software to implement (or mostly not implement) meaningful read ahead caching for smooth and reactive work.

I would expect this to help about anyone dealing with large files signficantly.
That interesting. What software you use for the readahead with Primocache? I am always interest getting the extra speed.
Axel Mertes
Level 9
Level 9
Posts: 180
Joined: Thu Feb 03, 2011 3:22 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by Axel Mertes »

No, I have no software for preloading. Thats exactly the point:

Either my software is having preloading by its own or there is no preloading (as most of the time).
So my playback speed is limited to the actual drive speed "just in time". So my (old and often mentioned) wish is that a new version of PrimoCache allows us to preload "n" blocks beyond the last blocks read. Thats an option that would accelerate mostly sequential files on defragmented disks (such as in my video production environment). It would be pretty simple to implement and it will not allow faster "long term" playbacks, but in an editing scenario it would help. I could even imagine to read "n" blocks before the lowest block number and "n" blocks after the highest block number. This is the way editing usually works, jumping back and forth form a specific position, repeatably.

If you want this too, we are two ;-)
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by Jaga »

I'd simply like the ability to preload a folder/file by right-clicking it, and choosing the option from a Primocache context link. You could preload games, videos, whatever you wanted just before you used them, speeding it all up tremendously.
Davey126
Level 7
Level 7
Posts: 99
Joined: Sun Mar 23, 2014 3:40 pm

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by Davey126 »

Jaga wrote:I'd simply like the ability to preload a folder/file by right-clicking it, and choosing the option from a Primocache context link. You could preload games, videos, whatever you wanted just before you used them, speeding it all up tremendously.
That capability is better aligned with a ramdisk IMO.
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: [2017-05-27] PrimoCache 3.0.0 Beta is available now!

Post by Jaga »

Davey126 wrote:
Jaga wrote:I'd simply like the ability to preload a folder/file by right-clicking it, and choosing the option from a Primocache context link. You could preload games, videos, whatever you wanted just before you used them, speeding it all up tremendously.
That capability is better aligned with a ramdisk IMO.
Disagree.. but that's the spice of life. I enjoy having faster boots, overall snappier and responsive system speed. But sometimes I'd like to ensure that a file/game/program/etc runs as fast as possible, without having to create a separate ~30-40 gig RAMdrive for it. I use a lot of different software, and would need somewhere on the order of 256 gig of RAM if I was going to do that, and that just isn't practical. On the other hand, a simple feature that requests a cache load of a file/folder wouldn't be difficult at all.
Post Reply