[2021-01-30] PrimoCache 4.1.0 released!

First hand news related to PrimoCache
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: [2021-01-30] PrimoCache 4.1.0 released!

Post by Support »

@BrightSilence, I think the best solution is trying to find out what caused ungraceful shutdowns.

Please note that after an ungraceful shutdown, PrimoCache verify L2 data based on the last boot. For eg. if last boot up there were 50MB L2 data, and during running PrimoCache gathered new 100MB L2 data, 150MB in total. PrimoCache only verifies 50MB initial L2 data, new gathered 100MB data will not be verified and just be discarded.

Besides if iSCSI drives were disconnected from the computer during Windows running, PrimoCache will remove its L2 cache. Even you reconnected drives quickly, PrimoCache will not use previous L2 cache. So you should keep iSCSI drives connected until you shutdown the computer in order not to clear its L2 cache.
BrightSilence
Level 2
Level 2
Posts: 5
Joined: Tue Mar 02, 2021 12:56 pm

Re: [2021-01-30] PrimoCache 4.1.0 released!

Post by BrightSilence »

Thanks @Support for the additional info, I appreciate it. That does help explain some of the behavior I'm seeing and I'll try to reboot more often to make sure not all L2 is lost. Of course preventing the ungraceful shut downs is priority 1. But for reasons unrelated to PrimoCache I'm having them more often than I would like atm. That's not your problem though, however they clearly can't always be prevented. In my case it's not about MB's, but I have 800GB of L2 cache available. I use this to install my games on the iSCSI drive. So it can take years to fill that all up to use it to its full potential. And I don't think anyone goes years without an ungraceful reboot.

Furthermore I had set up my PrimoCache with no write cache in memory, hoping it would ensure me that no data was lost. Obviously wiping the L2 makes this pointless. But I think this would be an awesome feature if it could be made to work like I expected it would. So I really hope a feature could be added that lets the end user ensure data never changes on the target disk without PrimoCache knowing about it and assuring it can just trust the L2 cache. This would also solve the issue with disconnects of an iSCSI drive.
I understand why it doesn't work like this right now. The contents of the read cache can't be assured to be correct. But it currently also means you will certainly lose all deferred writes. Which I initially had set to only flush when idle. Which again would be a great option if the L2 could be relied on to be persistent.

Alternatively, it would be nice if there was an option to also prefetch L2 cache in case it was wiped. While this is not nearly as great as not wiping it at all, it would at least ensure that given enough time the cached read data would be available again. It still means you can't rely on deferred write data being persistent though.

I understand that this may require significant development effort. But it could make the product do more than it does now. I'm currently a little hesitant to recommend it and still evaluating whether to purchase or not myself. I previously used CloudDrive to do a similar thing, but it had issues of its own. Until I saw the L2 being wiped I thought I had found the perfect replacement. For now I'll try to work around the issue with the additional info provided, though it feels a little like walking a mine field. I like PrimoCaches approach a LOT more than CloudDrive, so I really hope I can make this work. And perhaps you'll consider expanding functionality on the L2 persistence during ungraceful shutdown. Thanks for your time!
User avatar
Support
Support Team
Support Team
Posts: 3623
Joined: Sun Dec 21, 2008 2:42 am

Re: [2021-01-30] PrimoCache 4.1.0 released!

Post by Support »

@BrightSilence, I'm sorry for the late reply! PrimoCache does have an option to preserve L2 cache by ignoring sync check at boot. It is on L2 advanced options dialog, but you have to enable it via CLI. Assuming your cache task index is #1, type in the following command in the CLI.
rxpcc edit 1 -j ignoresync
For information about the CLI, please see
https://www.romexsoftware.com/en-us/pri ... rface.html
However, this still doesn't apply to the disconnection issue of iSCSI drives. Logically, if a drive is removed, all occupied cache should be released for other drives. We have submitted a feature request to address this iSCSI disconnection issue.
Thanks.
QuantumX
Level 2
Level 2
Posts: 5
Joined: Mon Jan 11, 2021 6:30 am

Re: [2021-01-30] PrimoCache 4.1.0 released!

Post by QuantumX »

Support wrote: Tue Mar 09, 2021 2:25 am PrimoCache does have an option to preserve L2 cache by ignoring sync check at boot. It is on L2 advanced options dialog, but you have to enable it via CLI.
Will ignoresync prevent Primocache from running the L2 verification scan added with V4.0.1 after ungraceful shutdown? Verifying 900GB L2 takes a while and I live in a country where the government cuts everyone's power two to three times a month...
BrightSilence
Level 2
Level 2
Posts: 5
Joined: Tue Mar 02, 2021 12:56 pm

Re: [2021-01-30] PrimoCache 4.1.0 released!

Post by BrightSilence »

Support wrote: Tue Mar 09, 2021 2:25 am @BrightSilence, I'm sorry for the late reply! PrimoCache does have an option to preserve L2 cache by ignoring sync check at boot. It is on L2 advanced options dialog, but you have to enable it via CLI. Assuming your cache task index is #1, type in the following command in the CLI.
rxpcc edit 1 -j ignoresync
For information about the CLI, please see
https://www.romexsoftware.com/en-us/pri ... rface.html
However, this still doesn't apply to the disconnection issue of iSCSI drives. Logically, if a drive is removed, all occupied cache should be released for other drives. We have submitted a feature request to address this iSCSI disconnection issue.
Thanks.
I really appreciate the helpful response. I enabled the ignore sync option and am testing with it now. Thanks for also creating a feature request for the iSCSI disconnect case. That would get rid of all cases of L2 cache being wiped.
Will ignoresync also retain and flush deferred writes to disk after an ungraceful shutdown? Or should I still not expect it to work like that? If it does, I might turn off write caching in L1 again and switch back to only flush deferred writes to the iSCSI disk when idle to squeeze out just a little more performance. This would be a little too risky if those deferred writes aren't resumed after an ungraceful shutdown though. (though, I would consider this icing on the cake)
For what it's worth, my evaluation period is running out and I've decided to buy the license after you've been so helpful!

@QuantumX I don't know the answer to your question, but I don't think there is any reason to scan if it's not going to wipe what can't be verified anyway. I assume it doesn't do the scan at all. That said, I wouldn't use this setting if I weren't ok with losing data on this disk. It's just game installs for me, which I could just reinstall if I lost something. The warnings stated by the CLI as well as the GUI are there for a good reason. So I'd say only use it if you know it's a good fit for your scenario.
User avatar
Jaga
Contributor
Contributor
Posts: 692
Joined: Sat Jan 25, 2014 1:11 am

Re: [2021-01-30] PrimoCache 4.1.0 released!

Post by Jaga »

BrightSilence wrote: Tue Mar 09, 2021 9:34 pmWill ignoresync also retain and flush deferred writes to disk after an ungraceful shutdown? Or should I still not expect it to work like that?
Any writes in L1 that aren't committed when the machine encounters an ungraceful shutdown are lost. If however they are committed first, then they are retained on the drive. The real answer comes from asking "why am I getting ungraceful shutdowns in the first place?" Knowing that may tell you whether or not the STOP is occurring before or after Primocache has had a chance to flush writes at shutdown.
BrightSilence
Level 2
Level 2
Posts: 5
Joined: Tue Mar 02, 2021 12:56 pm

Re: [2021-01-30] PrimoCache 4.1.0 released!

Post by BrightSilence »

Jaga wrote: Wed Mar 10, 2021 6:21 pm
BrightSilence wrote: Tue Mar 09, 2021 9:34 pmWill ignoresync also retain and flush deferred writes to disk after an ungraceful shutdown? Or should I still not expect it to work like that?
Any writes in L1 that aren't committed when the machine encounters an ungraceful shutdown are lost.
Thanks for your response.
Of course this is true for L1 cache as it is in RAM. I was hoping to be able to switch L1 to read only and only use L2 for deferred writes in the hope that the deferred writes will be retained through ungraceful shutdowns with ignoresync on.
xinwei
Support Team
Support Team
Posts: 28
Joined: Mon Jan 25, 2021 8:35 am

Re: [2021-01-30] PrimoCache 4.1.0 released!

Post by xinwei »

I am sorry. IgnoreSync has been turned off in the current 4.x and will be improved in the future
BrightSilence
Level 2
Level 2
Posts: 5
Joined: Tue Mar 02, 2021 12:56 pm

Re: [2021-01-30] PrimoCache 4.1.0 released!

Post by BrightSilence »

That's unfortunate, but I'll keep an eye on updates. I think I may have a way to solve the ungraceful shutdowns I've been having and I'll reboot frequently to ensure I don't lose all L2 cache. Looking forward to what's coming.
Stateless
Level 3
Level 3
Posts: 10
Joined: Wed Dec 28, 2016 3:17 pm

Re: [2021-01-30] PrimoCache 4.1.0 released!

Post by Stateless »

Are there any plans to add back in the ability to use 4Kn drives as cache drives? I had been using Storage Spaces to RAID-0 NVMe drives for a large and fast cache for large spinning disk volumes. It's keeping me from upgrading at this point.
Post Reply