smart copy verification
-
admin
- Site Admin
- Posts: 67270
- Joined: 22 May 2004 16:48
- Location: Win8.1, Win10, Win11, all @100%
- Contact:
smart copy verification
What would you think of an optional "smart" verification which only checks first 5% and the least 5% of the transferred data? Would be 10 times faster than the current full verification. And be 10 times nicer to your hard disk.
FAQ | XY News RSS | XY X
-
serendipity
- Posts: 3360
- Joined: 07 May 2007 18:14
- Location: NJ/NY
Re: smart copy verification
Maybe randomly check 5% from anywhere? or a tweak to increase 5% to more.admin wrote:What would you think of an optional "smart" verification which only checks first 5% and the least 5% of the transferred data? Would be 10 times faster than the current full verification. And be 10 times nicer to your hard disk.
-
grindax
-
admin
- Site Admin
- Posts: 67270
- Joined: 22 May 2004 16:48
- Location: Win8.1, Win10, Win11, all @100%
- Contact:
Re: smart copy verification
Holy crap, you are right! I had totally forgotten that I had discovered the fast crypt API recently. I short test showed that they are much faster than my own method. Cool!grindax wrote:For backups and copy operations, I think the smartest verification would be to checksum the files rather than do the current byte-by-byte simultaneous comparison.
Calculate the checksum of the source file while it's being read in the first place and then, after copying is complete, read the destination file in one go (on it's own, without any unnecessary seeking), and calculate it's checksum for comparison.
FAQ | XY News RSS | XY X
-
LittleBiG
- Posts: 1848
- Joined: 08 Apr 2011 12:57
- Location: Win10x64
Re: smart copy verification
Just for the record: all serious duplicate finder software have a byte-to-byte comparison as well, because they said that is the most accurate method for checking. However, checksum generating is really faster.
-
eil
- Posts: 1923
- Joined: 13 Jan 2011 19:44
Re: smart copy verification
idea with 5% is good, but checksum is real faster and easier way of solution
byte to byte is not needed here, IMO.
byte to byte is not needed here, IMO.
Win 7 SP1 x64 100% 1366x768|1900x1080
-
nas8e9
- Posts: 2232
- Joined: 21 Jun 2008 14:50
Re: smart copy verification
Accuracy/reliability depends on two things, I guess:LittleBiG wrote:Just for the record: all serious duplicate finder software have a byte-to-byte comparison as well, because they said that is the most accurate method for checking. However, checksum generating is really faster.
- choice of hash algorithm: as far as I know, crc and md5 aren't considered as bullet-proof as the later sha-versions;
- making sure the destination is read from disk rather than the cache (reading the *source* from cache shouldn't matter and could be faster).
If both conditions are met, I'd say hashing should work here.
XYplorer Beta Club