Re: What the people are asking for + a few others.
Posted: Fri Nov 16, 2012 10:25 pm
Just bought a ocz synapse 128GB... too bad fancycache didn't come out with their persistent cache first... could have saved some money and just bought a solution from them (I've been waiting for over a year...), the market is there...
However...
I had a theory that an ssd should hold all small files (say <=128KB), and the large files should be left on the storage medium. (I had a batch file that did just this using symlinks). the small files is what eats up most of the seek time on a hard drive.
After reading bcache stating it doesn't cache sequential reads...
I was thinking caching could take it a step further, say for larger files (over 128KB), the first 128K should be stored on the cache, so the hard drive can spin up and grab the remainder of the file (i.e. the 7ms time).
I'm not sure how much of an improvement that would make in performance, and the 128KB size might actually need to be higher, it might need to store up to the first 1MB to bridge the 7 ms gap between NAND and HDD access times.
Some stats...
on my pc, windows folder holds 28.8GB with about 80,000 files... comes out to be an average file size of 280KB
doing a simple *.* ; <=128KB search, I find this many files...
results in 4028 files. I didn't select them all to get a total size, but lets say they are 64KB average... hell, even at 128KB each, that's only
64KB average = 251MB cached for 28.8GB
128KB average = 900MB cached for 28.8GB.
So... there's a lot to play with here... Keep the small files on the SSD, and the large files on the platter and use the SSD mainly for small sequential reads/writes that it excels at, let the platter do all the large reads/writes...
I know there's a lot more involved with caching technology that I'm not aware of, such as fifo and queue's and all sorts of other things, but this just seemed like a simple solution that could be implemented into a ssd caching structure that wanted the hd to still do some work for maximum throughput.
Update
Pre-emptive caching
cache small files written during write-back caching
for example, new install, installing a bunch of files, first written files... never been read, just written
cache the small files (not to overwrite current cache that has higher priority) just in case that file is re-read later so as not to have seek time penalties
However...
I had a theory that an ssd should hold all small files (say <=128KB), and the large files should be left on the storage medium. (I had a batch file that did just this using symlinks). the small files is what eats up most of the seek time on a hard drive.
After reading bcache stating it doesn't cache sequential reads...
I was thinking caching could take it a step further, say for larger files (over 128KB), the first 128K should be stored on the cache, so the hard drive can spin up and grab the remainder of the file (i.e. the 7ms time).
I'm not sure how much of an improvement that would make in performance, and the 128KB size might actually need to be higher, it might need to store up to the first 1MB to bridge the 7 ms gap between NAND and HDD access times.
Some stats...
on my pc, windows folder holds 28.8GB with about 80,000 files... comes out to be an average file size of 280KB
doing a simple *.* ; <=128KB search, I find this many files...
results in 4028 files. I didn't select them all to get a total size, but lets say they are 64KB average... hell, even at 128KB each, that's only
64KB average = 251MB cached for 28.8GB
128KB average = 900MB cached for 28.8GB.
So... there's a lot to play with here... Keep the small files on the SSD, and the large files on the platter and use the SSD mainly for small sequential reads/writes that it excels at, let the platter do all the large reads/writes...
I know there's a lot more involved with caching technology that I'm not aware of, such as fifo and queue's and all sorts of other things, but this just seemed like a simple solution that could be implemented into a ssd caching structure that wanted the hd to still do some work for maximum throughput.
Update
Pre-emptive caching
cache small files written during write-back caching
for example, new install, installing a bunch of files, first written files... never been read, just written
cache the small files (not to overwrite current cache that has higher priority) just in case that file is re-read later so as not to have seek time penalties