Breaking your hard drive/data into multiple drives with Fanc

[RRTRACKS]

The logic or madness for my computer’s drive organization layout follows.

A relatively small 100gb partition for Windows as Drive C: was created for the purpose of making it easier to back up the Windows OS. Currently the only way I know of reliably backing up the OS is by making a disc image the OS is installed on. Using this method it has taken me about 15 minutes to either back or restore (And I had to do this recently) the OS using an external SATA drive.

Given the importance of backing up your system with the use of a Beta program, a few more comments on backup methods follow.

Windows 7 provides a backup program capable of creating a disc image of the OS. The problem with this feature is if any files perceived to be system files are installed with the files on your program partition, that partition is required without option to be part of the disc image. MS Office of all programs puts system files with the program files. If the Windows 7 backup program works for you, I would go with it.

For my needs the free version of Paragon Backup has worked very well. Having automatic cloud backups or synchronization of data files including Outlook emails provided by Norton 360 has also worked very well.

All program installed files go on to a 500gb partition. One problem with this setup is software installation programs always defaults to the OS drive letter since the OS does not provide any other option. Therefore, you have to manually select the drive letter for every program you install. If you want to complain to Microsoft about this issue, good luck.

A 4gb partition is used for all data files created including email. Everything on this drive as previously mentioned is automatically and regularly backed up to cloud storage.

I have about 800 cds worth of music and close to 7500 cds worth of audio books that went on two other separate partitions. This could easily be done with separate folders on the same partition, but when dealing with a huge number of files I prefer separate partitions. It might not make any difference with Windows 7’s ability to index at the folder level, but not knowing any better, this is what I have done.

Whenever it is possible index data, it is always going to be more efficient than caching data. If you use Find in Windows, all the search options will be indexed for any drive or folder you select to be indexed. In almost all cases including Outlook, files for indexing have to be manually selected. If you have Gmail, big surprise all your email is indexed for searching. This is Google thing but one area you would have thought Microsoft would have also built into Outlook given its use and competition.

If you are going to be caching a system I/O effectively it might be of some help to have more than one drive and/or partition. Everyone’s need is going to be different. Using some of my system’s criteria for any comments you might have would be appreciated.

All the Best,
Rich

[dustyny]

Rich,


You have the right idea in some ways but there are a couple of issues that I can foresee with your setup. Yes you do want to segregate your data from your OS but I'd like to suggest you are taking it a bit to far. Keep this in mind, the more complicated you make your setup the more difficult it is to troubleshoot when things go wrong.

Lets say one of your drives or partitions is damaged, you could find yourself in a very confusing situation when you are running a repair tool. Most of the best drive repair tools run on Linux LiveCD (as DrDOS doesn't support modern hardware as well) and they don't display drive information in the same way windows does.

You could very well find yourself looking at a print out like this; now imagine you need to fix just one partition.
Device Boot Start End Blocks Id System
/dev/hda1 1 39 157216+ 6 NTFS
/dev/hda2 40 293 1024128 83 Unknown
/dev/hda3 * 294 332 157248 83 NTFS
/dev/sda1 1 17 136521 82 NTFS
/dev/sda2 18 170 1228941 83 Unknown
/dev/sda3 171 261 1959930 5 NTFS
/dev/sdb1 1 17 136521 82 CDROM
/dev/sdb2 18 261 1959930 83 NTFS

It's not unsolvable of course but you'll most likely waste a lot of time trying to find the right partition. Trust me when I say it is SOOOO easy to pick the wrong one and destroy your data..

General best practices for Windows drive/partitioning is roughly the following.

OS & Programs should be kept one partition/drive - The reason why you keep your OS & Programs on the same partition is you want to back both of them up. The programs that you install are dependent on your registry. In the event that you restore your OS from backup there is a very good chance your registry will have a lot of out of date data. This might not be an issue for all programs but if you have registry settings for an earlier version of an application, it could cause major problems such as data corruption. In the instance that your program partition is damaged, then you'll end up with a lot of orphaned entries in your registry which slows down your machine. However the way I do this is, I use Portable apps (free) & Thinapp (a pro tool). That way I don't install anything to my machine. The nice thing about this is, when I need an app on another computer I just copy the app folder over, no install necessary (don't try to run from a SMB share, it's to slow)http://www.portableapps.com/

Ideally (but not necessary) you put your page.sys on it's own hard drive. I generally use this with servers but since ram is so cheap and ssds are so fast, I haven't seen any reason to do this for a while. Even though the page.sys file isn't accessed as often or intensely as most users think (gamer's seem to be overly concerned about it), putting it on it's own drive makes sure you don't run in to drive I/O contention issues. This is really most useful when you exceed the amount of memory your machine has and you're machine is swapping a lot of memory pages in out and of the page file. For example if Photoshop is trying to use 4GB when all you have available is 2GB. You could put it on a different partition so you don't include it in your image but I vaguely recall that you need to set a registry setting first or Windows will spit out errors when you restore.

Put data on a separate drive or partition - You can use as many as of these as you want but each one you add, will make your life more difficult as mentioned above. Personally I'd suggest using a data drive with a reasonable file system hierarchy because it's simpler to maintain. Since your OS/Program drive is accessed the most, it is at most risk for failure. Putting data on a separate drive, reduces I/O contention and it reduces your risk of failure (how much is arguable). Ideally you'll mirror the drive or partition using RAID 1 or Storage Spaces mirroring (Win 8 & 2012).

Whenever it is possible index data, it is always going to be more efficient than caching data.

I get the feeling you're mixing things up a little here.. You can't really compare the two since they do two very different things. I can't say for sure but the index probably utilizes the Windows Cache (tons of things do) in order to speed up searching.

Index data = Reads data from your files and it stores search terms in to a local database that windows uses for quick look ups. So when you do a search instead of reading data from every file on your drive or a particular folder checking to see if your search term exists in those files, it queries a database that returns hits with pointers to the files that contain the data you searched for.

Caching = There are lots of types of caching but essentially means you are placing a copy of data in an easy to access place (such as memory) where there will be a speed advantage when accessing that data. For instance Fancycache will cache recently accessed data first to your ram, then when you request that data again it reads it back from RAM which is much faster then reading it from the HDD.

[RRTRACKS]

dustyny,

Thank you for the information. Since my use is just for the home, the tolerance (or risk) I have for errors for the sake of convenience is quite a bit higher than yours. My system does not get used nearly as much either so the chances of hardware failure are much less. With me doing things user induced disasters are probably much more likely.

The idea behind making the operating system partition small is so it can be quickly backed up. With the larger program file partition, incremental backups can also quickly be done with an occasional full backup. If an incremental backup of the program partition is done at the same time a system image is made the concerns you mentioned about registry conflicts would be minimal. I do not do backups as often as I should; but I am a heck of a lot better at doing them than 99 percent of my friends.

I have used Windows Disc Tools for checking hard discs for bad sectors and clusters. Is there anything wrong with Windows Disc Tools?

Windows indices are similar to the database indices you described in your post. If anyone would like to verify this, do a search of Windows help for indexing. There you will find a section on moving the location of index files to free up disc space. For anyone interested in using indexing, the instructions given in the help files are short, easy to implement as well as understand.

Thank you again for the time and information you provided.

All the Best,
Rich

[dustyny]

Since my use is just for the home, the tolerance (or risk) I have for errors for the sake of convenience is quite a bit higher than yours. My system does not get used nearly as much either so the chances of hardware failure are much less. With me doing things user induced disasters are probably much more likely.

I'm sure it took you a while to develop your setup and it's working very well for your workflow, however you are putting yourself more at risk than you realize. H/W failures happen for a lot of reasons and "mean time to failure" (usage) is just one of many causes. AC power anomalies are actually a bigger concern but there is always data rot, flipped bits, corruption, and just random H/W failures with no discernible cause. Most causes of failure are unpredictable and are completely out of control of the user.

The idea behind making the operating system partition small is so it can be quickly backed up. With the larger program file partition, incremental backups can also quickly be done with an occasional full backup. If an incremental backup of the program partition is done at the same time a system image is made the concerns you mentioned about registry conflicts would be minimal. I do not do backups as often as I should; but I am a heck of a lot better at doing them than 99 percent of my friends.

I'd encourage you to reconsider this approach, I don't think you fully understand why this will be a problem for you. If you get a proper third party backup tool, it will address all of your concerns and it will help you protect your data. At the very least be sure you are backing up the separate drives on the same backup job. Another thing to consider is backups & the media that holds them often fail and if you don't test them (I do a restore to a Virtual disk) you have absolutely no way of knowing if they are good or not.

I have used Windows Disc Tools for checking hard discs for bad sectors and clusters. Is there anything wrong with Windows Disc Tools?

Windows disk tools are very rudimentary and it will only solve simple problems. If you ever have a major problem chances are you'll need to try a variety of recovery tools until you find one that can detect and address the issue. That's when things will get really confusing for you because many of them display drive data in very different ways. I'd recommend familiarizing yourself with Hiren's BootCD, The Trinity Rescue Kit, The Ultimate Boot CD, System Rescue CD as they will help you troubleshoot just about any problem you'll come across.

Windows indices are similar to the database indices you described in your post.

Perhaps you misunderstoodd what I was telling you, Windows search uses a database file to store its index. In this instance the database is just a file with a specific data structure. Databases come in all sizes, a database can be as simple as a flat text based Comma Space Variable (CSV) file or they can be as complicated as relational data structure, hosted across thousands of machines.

[RRTRACKS]

Dustyny,

Thank you for both the time taken and the information provided. I have saved your comments and will use them for reference should the need arise.

All the Best,
Rich

[dustyny]

Good talking to you.. =)