Data corruption?

[dustyny]

Ased -
Sorry for the hijacking of your thread.

Manny -
When I went to college for the first time, I took a psychology course. All of sudden I had all this new information and I started to "diagnose" everyone around me. This person is probably bi-polar, that person is probably a narcissist, etc. The truth is I had no idea what I was talking about. I understood the basics but not nearly enough to understand something as complex as a psychological disorder. You see, a little bit of knowledge gives you a false sense of confidence but these things are very complicated and it takes a LOT of learning and experience before you can say for certain how things work in the real world.

Think about this, the Windows NT code has been in development for 19 years and it contains anywhere from 50-150 millions of lines of code (how much exactly isn't public knowledge) and it was written by thousands of the worlds most talented programmers. Don't you think that the problems that would concern an inexperienced programmer, would be immediately apparent to an senior programmer (who probably has a PHD in compsci)? Doesn't it make sense that they would have taken these issues in to account and addressed them?

Also i'm not sure that you completely understand how general fault chance is calculated for serial connected elements. The simple rule - general fault chance is higher then the biggest chance among the elements.

I do understand, it's an academic exercise that you learn in your first year of college.
Maybe I should have mentioned that I have a Bachelor of Science in Systems Engineering & Architecture, not to mention two decades of experience as a network & systems administrator.

So ECCs that you have mentioned that used for storing data on hdd, works well, and some time even can repair bed block with remapping and without data corruption. But not always.

Even on a drive with bad blocks, a journaled file systems (NTFS, BTRFS, EXT2-4, Reiser, ZFS) will either self correct or remap the bad blocks. Yes its possible for a drive to develop so many bad blocks that the Filesystem can't address them but that's just a symptom of a failing drive, you can't avoid that. This is mostly an issue for non-journaled file systems such as FAT (16, 32, exFat). So yes you understand there is a danger here but you don't seem to get what that means in the real world. This is a case of experience and you really need to manage a lot of devices over a long period to understand the difference.

Other links in the chain are much less protected. For example RAM for servers has ECC and that cause one extra chip on the module. Home users don't want to pay more for ECC and loose ram size. So it not protected much.

Sorry to tell you but we don't use ECC RAM that much anymore. It adds a premium to the cost of the server, ECC slows down RAM performance, reduces your max ram and the protection it offers has been debated for years. There was a time where I had to have ECC ram in a server to even install server applications (such as M.S Exchange or SQL) but since Win2008 that hasn't been an issue. Bits flip all the time and the more ram you have, the faster the clock rate, the bigger the problem gets. If M.S didn't handle this on the software level, most PCs would BSOD all the time. So ECC adds a level of protection on the hardware level but this is also handled by the OS. These days I don't buy ECC ram unless the vendor requires it and that doesn't happen to much anymore.

That you know something, and i know something, may be you know something better. But i know that no matter how cool you are, there is always something that you don't know.

You and I are not in competition with one another. You are inexperienced and I'm taking the time to explain where you are making your mistakes because in the past were kind enough to do the same for me. You don't have to listen to me but I'd suggest spending some time on a professional forum like Technet.

So you can very long say to me how solution safe is solution, but i will always say: "There is a way how it will go wrong! And it will!"

The problem is you don't understand how to do a proper risk assessment and it's caused you to overestimate how dangerous these issues really are. So if your concern is explaining to people that their data is put at risk, that's fine just don't exaggerate the issue. Do you experience read issues? No you don't, so why are you trying to worry someone about something that COULD be an issue? An electrical spike COULD damage your computer, is the solution to tell people don't turn power on their computers? Of course not. Yes each app is different and they CAN have this issue but it is not a problem with Fancycache at the moment. SO instead of worrying them about some hypothetical possibility, why not actually help them by taking the time to explain how they can protect their data using data parity strategies (RAID, Storage Spaces, Backups, etc)

If some one is smart enough to maintain caching and all possible issues - then he would not post such questions, and if he does, probably it is not good idea to advice him to use caching at all. General rule - "Afraid? - Don't use!" especially when it is in BETA.

This product is drawing a lot of enthusiasts and it's clear they want to learn. Who are you you to judge if someone you don't know anything about is ready for that or not? You think you are helping people by telling them to go away because they don't understand how things work, you don't either. Should I tell you to go away, don't mess with this beta? You aren't helping someone to learn by scaring them away from experimenting. The only real thing a user needs to be told is that they need to have their data backed up before they start testing this product. Anyone who seeks out a product like this has or will break their machine at one time or another and in the process of figuring out how to fix it, they'll learn how computers work. That's the very soul of being a computer hobbyist, build it, tweak it, break it, fix it and repeat.

[Ased]

@dustyny

I don't want to use memory caching to boost benchmarking tools, I want to use it to boost actual application use and general Windows performance.

The PC I tested on is a SB i3, with 2GB DDR3 memory, 1GB DDR5 Radeon HD5550 GPU, and 1TB 7200RPM hard drive.

My own PC is an i5 2500K, with 4GB DDR3 memory, 1GB DDR5 Radeon HD7770 GPU, and 1TB WD Green hard drive. I believe the WD Greens are 5900RPM and dynamically boost their speed if they are being heavily utilized, but this thing is so insanely slow it may as well be a 5400RPM drive.

Both systems have Windows XP SP3 32bit, because reasons.

Before you ask why I got such a high end CPU and didn't go mid range and upgrade hard drive as well, well it was a case of rolling the dice on what funds I had available. I previously had a Pentium 3 and then a Pentium 4, and I was absolutely sick of the CPU being the bottleneck. Suffice to say, I've already utilized the i5 to the point where an i3 would have been a bottleneck, so the upgrade was justified.

Now I don't expect Photoshop to launch instantly (actually I do, FancyCache is supposed to work like a smart ramdisk, but meh) but come on - absolutely no performance boost from stock? Shouldn't I expect at least 2 - 3 seconds improvement?

Photoshop launching isn't actually a big deal. My problem is the way Diablo 3 is designed, where it loads directly from the hard drive instead of loading a level into memory beforehand, means that it stutters horrendously. It's like I'm playing at less than 10FPS. You could suggest I test this on my main system, where Diablo 3 is installed, but I was worried about data loss, because I keep absolutely everything on that system. Alright I do keep a backup, but the backup is typically somewhat out of date.

Many people have reported similar lag in Diablo 3 and they've said that a ramdisk, SSD, or O&O's CleverCache fixes the problem. My only problem with CleverCache is that while you can reduce the cache writing to a few milliseconds, you can't disable it completely, which logically indicates that there may be data loss issues.

There is one other big reason I'd like to use ram cache - I have some folders with massive quantities of files inside, and whenever I nagivate to those folders, it takes several seconds to display anything. In fact, navigating to the one folder upon fresh boot takes nearly 30 seconds. This hard drive is rediculously slow.

-

@Manny

Err no, I never said CleverCache works better. I said neither caching services made any improvement whatsoever to Photoshop's launch times compared with an unoptimized system.

As I said, I tested them on the 2GB memory computer, and I allocated memory to cache using default settings. I think it was under 1GB but over 600MB both times. I checked, and Photoshop's memory usage while open was a total of 91MB. The 91MB may be meaningless because that's the open state, and the resources that require loading could be much higher than 91MB, but as I said, I do expect at least some improvement.

PS. I did not put my cat in the microwave. I like my cat.

[Manny]

[quote="Ased"][/quote]

I would not expect any improvements on system with 2GB. I think CleverCache is good solution for 2gb and win XP. Basically 2gb is low end. When you dedicate ram to Fancy Cache it takes memory from windows cache, and can even move some programs to swap, it can even reduce general speed. Basically you need to have 1gb free, 1gb for windows cache, 1gb for fancy cache, and other used by programs. So take Diablo3 system requirements and add 3Gb to it. You will have rough amount of ram that you need. So i would suggest using Clever Cache that just tweaks your win xp cache. But if Diablo3 has big resource files (100mb and more) then it would be useless. And one more thing Clever Cache and Fancy Cache probably gives the same chance to loose data.

[Manny]

[quote="dustyny"][/quote]
Looks like you did not get a point at all. So far you did not point me to any error, just considered that "my chances" are not good enough for you. Mean while I told you at the beginning that chance is small but still it is possible. Also i have never heard that Windows correct memory errors, please provide proof link.
I don't care much what you client use for servers. It has nothing to problem it self, same as you college or experience till you don't understand simple logical facts.
Also i have no interest in talking about other people teaching or learning, especially when they don't ask for that.
Question was about data loss chance, my answer was about loss chance. So please, if you have facts please provide them, other case i got your emotional point of view, and don't want to talk about it much.

P.S. Your mistakes: journaled file systems can roll back to previous version when write operation was interrupted. But still it can fail when write operations are reordered. Bed blocks are remapped by hdd it self, not by file systems. Memory error leads to BSOD. Memtest x86 don't fix memory issues with ecc, and give same results that in windows memory tests. There are some tasks that can't be solved on logical level, so experience of developers that were working in win 95 mean nothing. Frequency of RAM have increased only twice since 1999. And cpu max Frequency in 5-6 times since then. And would not grow at least till DDR5 (aprox 2016 year).

P.P.S don't use people as you experiment platforms, not every one is hobbyist and ready to break and fix. And for people who want to have stable systems rule is: install as less software as you can. Don't forget that real hobbyist will test what he wants no matter what we say here.

P.P.P.S It was last post about Dustyny and not about FancyCache. Sorry guys for flame.

[dustyny]

I don't want to use memory caching to boost benchmarking tools, I want to use it to boost actual application use and general Windows performance.

I understand but if you don't measure things before and after you don't really know what's changed. Tuning a system is a very time consuming process that involves lots of small changes and you need a way to measure if they change was beneficial or if it actually slows things down. Benchmarking is a way for us to get an idea of what things were like before tweaking and what they are like after.

Now I don't expect Photoshop to launch instantly (actually I do, FancyCache is supposed to work like a smart ramdisk, but meh) but come on - absolutely no performance boost from stock? Shouldn't I expect at least 2 - 3 seconds improvement?

Sorry but there is a lot more to a program loading then just reading it off of storage. The program has to process data and the bigger and more complex the program the longer it will take. I run Photoshop on a Core i7 3930k (overclocked to 4.2Ghz), 64GB of ram DDR3 2400, 8xSSDs (2.8GB transfer rate) and it still takes time for it to load up.

I believe the WD Greens are 5900RPM and dynamically boost their speed if they are being heavily utilized, but this thing is so insanely slow it may as well be a 5400RPM drive.

The WD green line has a very aggressive power saving algorithm and this is going to impact your performance. The same reason why these drives can't be used in RAID configurations. My guess is you're getting a slower performance then you'd get even from a 5400RPM (which is already extremely slow). That hard drive is really your systems major bottleneck and there's only so much you can do to mitigate that.

My problem is the way Diablo 3 is designed, where it loads directly from the hard drive instead of loading a level into memory beforehand, means that it stutters horrendously.

A number of people have mentioned Diablo 3 as for using Fancycache but the problem is always the same thing, there is a huge amount of game data and there's no effective way to cache it before hand because the data is randomly loaded.

My only problem with CleverCache is that while you can reduce the cache writing to a few milliseconds, you can't disable it completely, which logically indicates that there may be data loss issues.

This is one of those cases where people make a recommendation based on worse case scenarios which is very prudent, however data isn't that brittle That said it does take a good deal of know how to figure out how to do this stuff safely.

I have some folders with massive quantities of files inside, and whenever I nagivate to those folders, it takes several seconds to display anything. In fact, navigating to the one folder upon fresh boot takes nearly 30 seconds. This hard drive is rediculously slow.

You have a really slow hard drive and this is what happens. It takes time to read data off the disk.

- Manny
I've been doing my best to be polite to you. You have no clue what your talking about and you want people to think your an expert. I've felt bad about calling you out but you've spouted so much bad information that it really has to be pointed out. Can't say I feel so bad about that now..

[Manny]

My only problem with CleverCache is that while you can reduce the cache writing to a few milliseconds, you can't disable it completely, which logically indicates that there may be data loss issues.

That is ok, because you have cache in HDD it self, so this cache would not rise chance much. The thing that should worry you - how stable CleverCache is on your system. If it don't cause bsods, works well on hibernate and on heavy load then feel free to use it. If we can believe O&O then CleverCache caches only files, so in worst case you will loose only some files that have been written at the moment of crash to hdd, not whole disk.