Automatically caching certain types of devices...?

FAQ, getting help, user experience about Primo Ramdisk
Post Reply
FlowersEverywhere
Level 2
Level 2
Posts: 5
Joined: Sat Nov 05, 2022 10:35 am

Automatically caching certain types of devices...?

Post by FlowersEverywhere »

I'm gonna ramble a bit, but that sometimes ticks people off, so TLDR:

Can I somehow configure PrimoCache to automatically cache any VHD file I mount?

The too long part:

I'm one of those people who can't throw anything away and who hoard like crazy. I have a Linux NAS with 68 TB of storage attached. That storage is 2.5" SMR USB drives, i.e. slow. They're also using NTFS (for cloud backup purposes), so even slower when on Linux.

Because 68 TB is not enough for any true hoarder, I compress as many things as I can. Right now, I'm trying to find a good way to keep all my games stored on this NAS in NTFS VHDs, using LZX compression as this is very efficient. It also speeds up reads, as I now have to read less data over the network and from the slow spindles. However, it's still terribly slow for some of these massive multi-tens-of-gigamabytes games that are normal today.

So, I've been looking for some way to speed up this thing. I've installed trials of this PrimoCache software and of that CloudDrive thing, which is super cool, but it requires me to create one big virtual volume backed by encrypted files, which is okay, except I'm afraid what happens when something gets corrupted. I prefer a little less obfuscation, and PrimoCache seems pretty cool here.

I've tested it with caching iSCSI mounts of a couple of the drives from the Linux box, and it works with this setup. But then I noticed that if I mount a VHD, it too actually turns up as a cacheable device in PrimoCache. So, I can mount a VHD and cache it. It's not that I don't want to use iSCSI, but I have both a gaming laptop and a gaming desktop, and these NAS drives aren't just used for games, so if there was some way to not use iSCSI and instead mount VHDs via Samba, and then automatically have PrimoCache cache that mount, that would be bleedin' awesome.

For the record, it doesn't even have to some kind of GUI thing. I'm a programmer, and I've already written scripts to enable me to run games automatically off VHDs mounted via Samba, so if there's a CLI part to PrimoCache that I could use, that would work as well. The only requirement, of course, would be that - given say, a 500 GB cache device - I can keep cached data from a previous mounting of a VHD for the next time I mount it, so that if I wanna reboot the computer and play the game I just played before, it'll already be cached.

Okay, so I warned you, I'm a ranter. Call me grampa :) Thanks for any feedback!
User avatar
Support
Support Team
Support Team
Posts: 3394
Joined: Sun Dec 21, 2008 2:42 am

Re: Automatically caching certain types of devices...?

Post by Support »

It's a frequently asked feature request. However, I'm sorry that currently this feature is not available. PrimoCache is a disk-sector-level caching program and it only caches data stored in disk sectors which might be fragments of a file.

PrimoCache does support CLI. Please see https://www.romexsoftware.com/en-us/pri ... rface.html.
FlowersEverywhere
Level 2
Level 2
Posts: 5
Joined: Sat Nov 05, 2022 10:35 am

Re: Automatically caching certain types of devices...?

Post by FlowersEverywhere »

Hello, Support, and thank you for the response :)

Thank you for the CLI link - that's great! I'll read it over and try some stuff out.

I just want to make sure you didn't misunderstand me (my wordings are sometimes not as goodly as me likey) as you mention this thing about "only caching disk sectors."

I'm not trying to cache the VHD file per se. Once the VHD is mounted, PrimoCache can cache this virtual drive, since it has virtual sectors (it's now a valid drive in Windows). This I already confirmed. My question is more about whether I can say, basically, "whenever a virtual drive is mounted, start caching the resulting virtual drive - or resume an existing cache for it."

If I create a script that adds a mounted VHD drive to an existing cache configuration using the CLI version of PrimoCache, will this destroy the cached data of previously configured other VHDs that were added in the same way? I notice there's also a limit imposed on the number of drives one configuration can cache... Is this limit in the CLI version as well?

Like, me wordly badly methinks... Consider this (I haven't yet read the CLI docs or tested anything WRT that):
*) I create a cache configuration: "My VHD Cachey Thing" using a 500 GB SSD.
*) I mount a game in a VHD using my scripts (only one game VHD will every be mounted at once)
*) I extend my current scripts that mount VHDs. I add commands to use the CLI to add the mounted drive to the "My VHD Cachey Thing" cache if it's not already cached (assuming I can test if a VHD virtual drive is already in the cache with the CLI as well).

Am I still wording badly? Like, consider I have 1 million VHDs with all the games in the world. I'd like to script it so that I can cache all of them in the same cache. But, of course I will only be playing a handful of them in the same week, let's say. So having cached data on less often mounted VHD drives being evicted is perfectly fine and actually good. But I'd like for the latest handful of mounted VHDs to have cache data waiting for them to be mounted again, so the most recently played games are always SSD-speed fast. Does this make sense?

Sorry for the ranting XD And thanks for the reply :)
User avatar
Support
Support Team
Support Team
Posts: 3394
Joined: Sun Dec 21, 2008 2:42 am

Re: Automatically caching certain types of devices...?

Post by Support »

I understand now. Once a volume is cached, PrimoCache will automatically cache the volume the next time it is mounted, as long as you do not remove the volume from the cache task. However, the cached data of this volume will be cleared after the volume is unmounted. PrimoCache doesn't know when the volume will be mounted again, so it doesn't keep its cache forever. It will release cache so they can be used for other volumes in the cache task, or release resources to the Windows system.
FlowersEverywhere wrote: Mon Nov 07, 2022 12:02 pm I notice there's also a limit imposed on the number of drives one configuration can cache... Is this limit in the CLI version as well?
A cache task can cache up to 16 volumes. CLI programs and GUI programs have the same functionality and have the same limitations.
FlowersEverywhere
Level 2
Level 2
Posts: 5
Joined: Sat Nov 05, 2022 10:35 am

Re: Automatically caching certain types of devices...?

Post by FlowersEverywhere »

All right! Thank you very much for the detailed explanation :thumbup: :wave: Then it won't work like that, but I'm also considering some kind of iSCSI route, so perhaps :think:
Post Reply