PrimoCache and VMware direct access

[mrjrt]

I'm currently evaluating PrimoCache and think I've found a slight issue.

One of the things I do is have installs of an OS that are bootable natively on the hardware, but also under VMware as a VM. This is largely achieved by using direct-access to partitions. It has come to my attention that subtle corruption was occurring that was clearly down to PrimoCache, but I couldn't determine the cause as the native partition involved wasn't being managed by PrimoCache...(clearly, as VMware had exclusive access to it)...or so I thought.

...I just discovered that if I disable the deferred-write caching it now seems to work fine, but this obviously disables it for all drives in the cache task. It seems that if the PrimoCache management UI is started whilst VMware has exclusive access, the partition doesn't appear, but is still being managed (and cached!). If it is started before, then it is visible and continues to provide management stats whilst the VM is active.

Now, I have created a second cache task and placed this partition into it, and now I can disable deferred write caching purely for this partition, and all is well, however by doing so I need to define a full set of separate caches for it, which is wasteful as it won't always be active, hence why I wanted all drives using the same cache. The most problematic being that I have given my whole SSD over to the main L2 cache, and having to cut a chunk of that out of it for a single partition reduces the amount available to all the other partitions that overall will get more use.

So...is there any way of having individual volumes/partitions in a cache task each having separate cache settings (i.e. c: being read/write, d: being read, e: being write only and any combination of deferred write settings) so I can disable deferred write-caching for just this volume whilst retaining the read/write caches shared with all the others? (or somehow fix things so VMware's direct read/writes are serviced from the cache!)

...if you want to reproduce this, create a partition on your host, configure primocache with full caching, then create a VM in VMware using direct access to the partition and start something like Windows XP setup - the initial format of the partition will fail every time until deferred writes is disabled.

[Bjameson]

This kind of corruption is often reported when accessing a VMWare physical disk from the host, or vice-versa. That is, without PrimoCache. The corruption is a logical and fully predictable result of two independent systems accessing the same physical file system at the same time. Both systems have a different idea of the disk layout and are therefore prone to overwiting each other's clusters and each other's $MFT. I'm quite sure you are fully aware of this.

Recent versions of VMWare Server have only a limited ability to use physical disks because VMWare is well aware of the problem. Yet users have found a way around this limitation. Other VMs Like Virtualbox or Hyper-V are also able to use physical disks. So the problem is not exclusively related to VMWare and PrimoCache is guilty of nothing at all.

This means that Romex cannot quickly create a standard solution. They could do it, but only at the expense of constantly monitoring the system and trying to anticipate when a VM is going to access a Primo-cached volume. They would have to quickly switch off delayed write, flush the write buffer, and automagically switch it back on later. Not only for VMWare but also for other VMs. While theoretically possible, it would be a temporary solution for a problem which isn't even theirs. Corruption will occur sooner or later, with or without PrimoCache. It is your own responsibility to guarantee exclusive access for the VM.

Having said that, you already found a way to -at least partially- solve the problem. I'm not familiar with Primo's command line options. Maybe someone else could shine their light on this. I was thinking about a batch file which zaps your current cache configuration and quickly builds another one which accommodates your direct access volume. And vice-versa. This way you would only have to push a button to switch between modes. And to remember to push that button before firing up VMWare. On-demand reconfiguring L2 would be the biggest problem. Hopefully Romex can point the way. If not, I'm afraid you'll have to live with certain limitations.

Given the fact that fatal corruption is only a matter of time, I would personally never use such a setup for a production environment. At least, not without having recent backups at hand. How about reserving a volume exclusively for direct access experiments and never ever allowing PrimoCache to touch it in the first place. Some waste of resources cannot be avoided, unfortunately.

[mrjrt]

Thanks for the reply. VMWare locks the drive, or so I thought, hence it disappearing from the PrimoCache UI. I think the bug here is that somehow primocache is only partially interacting with the drive and/or isn't ignoring it completely. No reads come from the cache, and thus the deferred writes will obviously cause problems. Writes are copied to the cache, (seemingly), but as they're never read from it's pretty much a pointless exercise - so I'm better off disabling it altogether on the host side of things. There isn't an issue in the general sense with direct access to the partition as VMWare has exclusive access to it, so I'm not clear on your issues with it. Has worked perfectly well for me for several years without PrimoCache involved as VMWare locks the partitions out from the host.

Given I had to set up a separate cache for the partition anyway, I suspect it actually works out better as in the guest OS I can configure it to cache directly by giving it direct access to the SSD partition. So not all bad.