Virtual disk as L2Storage

Suggestions around PrimoCache
Post Reply
points
Posts: 35
Joined: Wed Mar 16, 2016 10:51 am

Virtual disk as L2Storage

Post by points » Mon May 27, 2019 12:49 pm

Following situation: 1 large SSD but not wanting to use it all for PC.
So normal way would be to partition and create dedicated partition(s) for PC.
However this requires repartitioning every time if volume sizes need to be adjusted.
So I only created a single volume and put a VHDX on it which serves as disk for PC.
This does work however there is no built in feature in Windows to auto attach the VHDX on boot.

So here comes my suggestion: PC should be able to detect wether the cache resides on a real disk or on a virtual disk. And then it should auto attach that virtual disk on boot as soon as PC requires it.

Edit: It seems that PC has a function built in that wipes L2Storage between boots if a disk has been detached which happens with the VHDX between boots. So my suggestion includes to not to wipe the L2Storage between boots if it resides on a VHDX so it keeps being persistent.

User avatar
support
Posts: 2406
Joined: Sun Dec 21, 2008 2:42 am

Re: Virtual disk as L2Storage

Post by support » Fri May 31, 2019 7:42 am

Virtual disks are not recommended as level-2 cache because this might cause deadlock on certain conditions.
Primo Ramdisk | PrimoCache
Romex Software Support

points
Posts: 35
Joined: Wed Mar 16, 2016 10:51 am

Re: Virtual disk as L2Storage

Post by points » Mon Jun 03, 2019 11:31 am

support wrote:
Fri May 31, 2019 7:42 am
Virtual disks are not recommended as level-2 cache because this might cause deadlock on certain conditions.
Virtual disks should be supported natively. Partitioning is troublesome and always poses the risk of data loss.
I have just been trying to change the size of an existing partition to create an additional one. Plenty of space available but disk management is telling me that there is not enough space for repartioning. So there is no way to create the desired partition. There might be 3rd party tools available but one of them recently messed up a partition on another drive so badly that it was beyond repair.

That does not happen with virtual disks and that's why PrimoCache should natively support them as L2 storage instead of making me messing around with creating and resizing partitions.

User avatar
support
Posts: 2406
Joined: Sun Dec 21, 2008 2:42 am

Re: Virtual disk as L2Storage

Post by support » Mon Jun 03, 2019 1:03 pm

Actually in initial releases of PrimoCache (was named as FancyCache at that time), we used files instead of dedicated partitions. However, we discard this design and use dedicated partitions because of the performance and possible deadlock issues.
Primo Ramdisk | PrimoCache
Romex Software Support

points
Posts: 35
Joined: Wed Mar 16, 2016 10:51 am

Re: Virtual disk as L2Storage

Post by points » Tue Jun 04, 2019 4:29 am

support wrote:
Mon Jun 03, 2019 1:03 pm
Actually in initial releases of PrimoCache (was named as FancyCache at that time), we used files instead of dedicated partitions. However, we discard this design and use dedicated partitions because of the performance and possible deadlock issues.
Hi support. Now that is the second time I come to think FancyCache was much better than PrimoCache. You have been dropping the wrong features :( :roll: . First time I came to that conclusion was here: viewtopic.php?t=4425

I think you should reconsider these decision made. It is always sad to hear that you had these features before but dropped them. From my perspective you made the wrong decisions here. Tech has changed since FancyCache times. Virtual disks are used everywhere and they're not just files, for the OS they behave like real disks you could even use to boot a real system from. And with todays M.2, performance should not be such a great issue.
At least while testing this on a M.2 SSD PCI I had no reasons to complain the system speed increased massively. Just last week PCI specification 5.0 has been finished and for PCI 4.0 we know already there will be SSDs like the Corsair MP600 with speeds up to 5Gb/s. With that and todays or upcoming Intels or AMD cpus I don't see that lack of performance could ever be a reason not to support virtual disks as L2 Storage natively today.

And regarding deadlock I don't know exactly what you are referring to as I did not experience anything like that when I used a virtual disk as L2 storage. But why not supporting virtual disks natively and let the users use this feature on their own risk like you are doing it with ignoresync feature?

Edit: One additional advantage comes into my mind, however I did not test if it really does work: A virtual disk could be placed on an encrypted disk. If this does work all contents would be auto-encrypted and therfore securely stored very easily. This cannot be achieved with the dedicated partition which can't be encrypted at all. You really need to think more about data security and encryption on your products. We are working with sensitive files here.

User avatar
support
Posts: 2406
Joined: Sun Dec 21, 2008 2:42 am

Re: Virtual disk as L2Storage

Post by support » Wed Jun 05, 2019 9:54 am

Again, thank you very much for your feedback and the idea for caching encrypted disks.
We will reconsider this.
Primo Ramdisk | PrimoCache
Romex Software Support

User avatar
Jaga
Posts: 388
Joined: Sat Jan 25, 2014 1:11 am

Re: Virtual disk as L2Storage

Post by Jaga » Wed Jun 05, 2019 5:06 pm

Just as a point of feedback - I wouldn't mind being able to use virtual volumes as L2 storage (as an option, not a replacement for non-virtual ones). But in the big picture I think Romex has "bigger fish to fry" in terms of other requested features that would benefit everyone equally. If it ever does happen it may be a long while from now.

points
Posts: 35
Joined: Wed Mar 16, 2016 10:51 am

Re: Virtual disk as L2Storage

Post by points » Thu Jun 06, 2019 4:29 am

Jaga wrote:
Wed Jun 05, 2019 5:06 pm
If it ever does happen it may be a long while from now.
Hopefully not as it does not seem to require too much work to be implemented. To have it in a working state all that is required to make sure the virtual disk gets auto mounted and attached before PrimoCache starts up. Vice versa for shutdown and either a sync check or an ignoresync to be able to keep the contents.
A skilled programmer should be able to beta that in 1 hour.

User avatar
Jaga
Posts: 388
Joined: Sat Jan 25, 2014 1:11 am

Re: Virtual disk as L2Storage

Post by Jaga » Thu Jun 06, 2019 9:18 am

points wrote:
Thu Jun 06, 2019 4:29 am
To have it in a working state all that is required to make sure the virtual disk gets auto mounted and attached before PrimoCache starts up.
That right there may be highly problematic, since Primocache runs at the kernel level and loads even before Windows does.

points
Posts: 35
Joined: Wed Mar 16, 2016 10:51 am

Re: Virtual disk as L2Storage

Post by points » Thu Jun 06, 2019 6:40 pm

Jaga wrote:
Thu Jun 06, 2019 9:18 am
points wrote:
Thu Jun 06, 2019 4:29 am
To have it in a working state all that is required to make sure the virtual disk gets auto mounted and attached before PrimoCache starts up.
That right there may be highly problematic, since Primocache runs at the kernel level and loads even before Windows does.
I don't think that is much of a problem. First of all you can set a virtual disk as boot drive, so it seems a virtual disk can be mounted very early during boot.
And even if not, that means only to mount it as soon as possible and make PrimoCache aware of it. Generally PrimoCache does not complain when a cache gets added while it is running.

Post Reply