Some guidance, please

FAQ, getting help, user experience about FancyCache
Lawnmower1
Level 2
Level 2
Posts: 7
Joined: Tue Mar 19, 2013 2:10 pm

Some guidance, please

Post by Lawnmower1 »

Hello All,

I've devoted a while reading the support information for FC as well as reading many of the posts here. I *think* that FC will work in my intended application, but would like to get some feedback from actual users in a couple of gray areas.

First, let me describe what I want to do: I have a file server machine running Windows7 tweaked for this purpose. It has a C2-Q8400 processor and 8GB RAM. It has a low-power profile that spins down all disks after 10 minutes. (I have no intention of changing this power profile due to the excessive power consumption caused by idle spinning disks.) System disk is an SSD. "User" disk (holds files frequently accessed such as DOCs, pictures, blah blah) is a 512GB SSD. Then there's a 1TB disk that holds a small video library. A 1TB disk that's used for automated backups of other machines on the LAN. And finally a 1TB disk (WD Black FAEX) that holds the software installation library (infrequently accessed) and the Music Library. The Music library is where I'm looking at using FC. It's a little under 400GB and 50,000 individual tracks. Due to the spindown setting, accesses of the hard drives often involves a wait to spin them up. In addition, I'm always doing some kind of maintenance on the music library (i.e. working on the tags) so it's not unusual to have to read the first few clusters of each music file, which as you can imagine, takes quite a while.

As it turns out, the SSDs work great as a system disk (no surprise there) and as the "User" disk which is frequently accessed. Even though they are powered-down too, they can resume R&W operation without any discernible pause.

The problem is the disk with the music files. Startup is annoyingly long, and the Tag-Reading operation (one small piece of each of ~50K files) takes forever. So there's this unused SATAII Force 120 SSD sitting here without a home. It seems that this, along with FC, may go a long way to dealing with the music files. 120GB ought to be enough to cache those files in their entirety that are frequently accessed, and (here's the tricky part) ought to be able to cache the first few clusters, that contain the tag information, of each of the ~50K files, without attempting to cache the entire file.

Will FC do this?

Also -- will FC have its cached information available without waiting for the HDD to spin up? I can't find any guidance on this issue at all. This is the Deal Breaker. If it's necessary to wait for HDD spinup to retrieve information that's already on the SSD, then this exercise probably isn't worth it.

So how does FC work in this situation? Can anyone clear this up for me?

Yes, I know that I could install it and try it, but TBH, I don't want to take any unnecessary software installation risk if there's no opportunity for a payoff in the first place. OTOH, if it looks like this is a viable use model, then I'll gladly wring it out and report back on the results.

BTW, it's really encouraging to see a small company like this go through the time/effort to interact with the user base to refine their product. There are way too many examples of the "Here it is, take it or leave it" approach, including many examples for this same type of product. Good for you, Romex. I'm hoping that this translates into a higher probability of getting something that works well in my situation. Which, BTW, I'll willingly pay for in its finished form.
Lawnmower1
Level 2
Level 2
Posts: 7
Joined: Tue Mar 19, 2013 2:10 pm

Re: Some guidance, please

Post by Lawnmower1 »

Forgot to ask: Would I be correct in assuming that FC also caches metadata such as directories and the MFT?
Lawnmower1
Level 2
Level 2
Posts: 7
Joined: Tue Mar 19, 2013 2:10 pm

Re: Some guidance, please

Post by Lawnmower1 »

Okay, curiosity got the better of me, so here's what I did: 1) Assembled a test machine out of some unused Socket 775 parts, 8GB RAM. 2) Installed Win7 x64 SP1 and updates. Minimal driver set for functionality. 3) Copied entire disk with music files to an unused disk. So what we have is Win 7 on SSD. Music files on WD Black 640GB, and FC on Force 120 (SATAII) SSD.

Installed "Disk" version of FC. Configured it to use 128MB of "Level 1" and 100GB of the Force 120. Everything looks okay so far. Key file download and installation clumsy and poorly implemented.

MANY reboots. (well, actually two, but that's about one too many)

Using the Tagging software, read the entire 50K-track music file. It takes about 5 minutes, which is even better than usual due to (unexpected, probably directories and MFT causing less seeking) cache hits in the initial reading process.

Shut down Tagging program and restart it. Sit there with amazed look on face while it loads up all 50K tracks in about 30 seconds, without accessing the HDD at all. WOW!! It was so fast that it choked Resource Monitor. Awesome. And, it would have been even faster had there been more processing power for the Tagging program. It maxed out one processor core, which I have never seen before.

Now for the bad news.

1) It's completely unusable due to a GIGANTIC memory leak. I've attached a picture of Taskmgr where the memory usage as the Tagging program hits the cache -- note the memory "Commit" value. There's a HUGE leak while it's building the cache, followed by another HUGE leak as it delivers blocks from the cache. Dang. Obviously this is a Deal-Breaker.

2) The read cache doesn't survive reboots. Say What? This is pretty much unacceptable too, as it defeats the whole purpose of using the cache. I see no reason for this either. If you mark the cache entries with the disk serial, you can easily verify that the same disk is available and safely deliver blocks from the cache. Even so, if that's not good enough, you could verify cached blocks versus what's on the disk for a short while (in background) to implement an additional layer of verification. The other bad part of this is that when using an SSD, it burns up a whole lot of unnecessary writes rebuilding a cache file that's identical to the one that was already there. I regard this as another Deal Breaker.

Bottom line: I'd really REALLY like to use this because it does exactly what I was looking for. But because of these two issues, I'm just going to tear down the experiment and wait for a (hopefully) fixed version.
Attachments
FC-memory-leak.JPG
FC-memory-leak.JPG (274.36 KiB) Viewed 9259 times
Lawnmower1
Level 2
Level 2
Posts: 7
Joined: Tue Mar 19, 2013 2:10 pm

Re: Some guidance, please

Post by Lawnmower1 »

Okay, a couple more things....

Apparently FC prevents power-down of the cached drive. At least the HDD in my test setup isn't powering-down when it ought to. There are no processes running that write to this drive, outside of FC. This is pretty disappointing, as a spinning drive can cost as much as 20W at the mains power, and in my usage, the drives are unused for many hours during the day.

Using an SSD for the FC cache file presents an interesting situation. When you configure FC, it reserves one large file of the size you request. Now think about this: TRIM no longer works as this is a permanent file and TRIM works only with deleted files. So ALL blocks of the file are considered "In Use" by the SSD and not subject to TRIM or background garbage collection. The only thing that maintains SSD performance is wear-leveling. Consider, for example, the situation where FC would be writing to this single file starting at Block 0 every time. That means that, with the exception of wear-leveling, it's pounding those same memory locations over and over again. Seems to me that this isn't a great idea, but I'd like to hear what the developers at Romex have to say about it. The fact that FC has to rebuild the cache, from the same starting location of the file on EVERY reboot, just makes this situation a whole lot worse.

So, given all of the above -- the memory leak, the abuse of the SSD, the interference with power management -- and despite the great potential of this product, I have to say that this is in need of significant additional work and nowhere near ready for release. It would be nice to see what's been going on over the ~11 months since this Beta was released.
Last edited by Lawnmower1 on Wed Mar 20, 2013 7:57 pm, edited 2 times in total.
Lawnmower1
Level 2
Level 2
Posts: 7
Joined: Tue Mar 19, 2013 2:10 pm

Re: Some guidance, please

Post by Lawnmower1 »

Tried one last experiment -- Network access to cached drive. It doesn't work, far as I can see -- it ignores the cache and reads the disk directly. So that puts the nail in the coffin for me.
Manny
Level 6
Level 6
Posts: 62
Joined: Tue Nov 13, 2012 11:42 pm

Re: Some guidance, please

Post by Manny »

So many letters... :)

Yes it has memory leak with L2 cache.
Yes l2 cache is not persistent.
Yes it keep hdd online, there is a reason for that, as well as checkbox to disable it.
Yes it can cache part of file.
Yes it should cache metadata as well.
It should cache network access as well, but i'm not sure if it actually does. Please try to read same file many times.

Also please read what is block level cache to understand FC better. As well as official user manual.

P.S. data in cache don't stay even if it is frequently used, if cache is 3gb, and If you read 5gb file, then at the end of operation cache will contain 3gb part of this file, no matter what was in the cache. So far it is pretty dumb. And works well when you read some restricted amount of data all the time.
Lawnmower1
Level 2
Level 2
Posts: 7
Joined: Tue Mar 19, 2013 2:10 pm

Re: Some guidance, please

Post by Lawnmower1 »

Hiya Manny, Thanks for the information.

I didn't notice the checkbox that applies to cached-drive power state. If/when I try this again, I'll pay closer attention to that.

As far as caching part of a file and metadata, yes, I verified that it indeed does. That part worked well.

No joy on network access caching. After a half-hour playing around with that I couldn't get it to work at all. That killed my interest in the product in its current state.

The cache file on the SSD was 100GB, BTW. I didn't read anywhere near that amount of data off the HDD, so there's no chance that anything aged out of the cache.

You have to wonder.... with this list of complaints over the last year, and no update in sight, has Romex lost interest in this product?
Manny
Level 6
Level 6
Posts: 62
Joined: Tue Nov 13, 2012 11:42 pm

Re: Some guidance, please

Post by Manny »

Lawnmower1 wrote:No joy on network access caching. After a half-hour playing around with that I couldn't get it to work at all. That killed my interest in the product in its current state.
Looks like a bug, please post in in bug reports section.
Lawnmower1 wrote:The cache file on the SSD was 100GB, BTW. I didn't read anywhere near that amount of data off the HDD, so there's no chance that anything aged out of the cache.
Sorry did not get it.
Lawnmower1 wrote:You have to wonder.... with this list of complaints over the last year, and no update in sight, has Romex lost interest in this product?
No, in fact they are still working, focusing on issues and GUI redesign.
dustyny
Level 8
Level 8
Posts: 118
Joined: Sun Sep 02, 2012 12:54 am

Re: Some guidance, please

Post by dustyny »

No joy on network access caching. After a half-hour playing around with that I couldn't get it to work at all. That killed my interest in the product in its current state.
This is not a bug, SMB is a networking protocol and has nothing to do with the storage system. If you want to use a disk caching tool with it you will need to mount a VHD(x). However if you aren't using 10Gb+ ethernet it's not worth the time.
Now think about this: TRIM no longer works as this is a permanent file and TRIM works only with deleted files. So ALL blocks of the file are considered "In Use" by the SSD and not subject to TRIM or background garbage collection.
I think you might be confusing hardware (block) level with the file system level. While the file system might lock the space (meaning the 10gb it takes can't be used until the file is deleted), on the hardware level data is written & deleted across the disk as necessary. This is how you get file fragmentation, file data will be stored in contagious blocks depending on what is available at the time of write. Over time as the available contiguous space becomes smaller, the data will be broken up across different non-contiguous disk blocks.

There is a misconception that SSDs are easy to wear out. One of the big Hardware blogs released a good article debunking this but I forgot who (Anandtech?). Unless you write to it nonstop 24x7 an SSD will outlive its usefulness (as in larger cheaper faster drives will replace it) far before you wear it out.
Manny
Level 6
Level 6
Posts: 62
Joined: Tue Nov 13, 2012 11:42 pm

Re: Some guidance, please

Post by Manny »

dustyny wrote: This is not a bug, SMB is a networking protocol and has nothing to do with the storage system. If you want to use a disk caching tool with it you will need to mount a VHD(x). However if you aren't using 10Gb+ ethernet it's not worth the time.
It is bug if FancyCache is used on server side, where shared folder is hosted.

On client side it is not a bug, but how can you enable FC for network share without VHD...
Post Reply