Faster verification in Backup, like in Edit | Compare

Features wanted...
Post Reply
Marco
Posts: 2347
Joined: 27 Jun 2011 15:20

Faster verification in Backup, like in Edit | Compare

Post by Marco »

I'm a huge fan of Backup with verification, but I notice that the verification stage is quite slow, slower than the copy itself. On the other hand "Edit | Compare | Compare Current File with Previous File" is fast. It seems like "Edit | Compare" reads both files simultaneously, while the verification in Backup reads them separately, in chunks. But I may be very well wrong on this, yet the difference in speed is absolutely noticeable. Is there anything that can be done to speed up Backup's verification?

PS: I usually backup from an internal SSD to an external USB3 that can reach 100 MB/s write speed, yet verification lives in the 40 MB/s realm.
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

admin
Site Admin
Posts: 60567
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Faster verification in Backup, like in Edit | Compare

Post by admin »

Both Backup and Compare use Byte-To-Byte verification (if Backup is configured like this) with almost exactly the same code. The only differences:

a) Backup allows for more interception (DoEvents), so that you can freely drag the progress dialog or abort by ESC with immediate effect.
b) Backup's progress status is maybe a bit more work .

I think a) has to be the cause of the speed difference.

admin
Site Admin
Posts: 60567
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Faster verification in Backup, like in Edit | Compare

Post by admin »

I made some experiments: I was wrong. That's not the cause of it. It's interesting. Still searching... :maf: :cup:

admin
Site Admin
Posts: 60567
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Faster verification in Backup, like in Edit | Compare

Post by admin »

Got it: The target file (that has just been written a millisecond ago) has to opened with FILE_FLAG_NO_BUFFERING to ensure all buffers are flashed to disk. This makes it slower. No way around this.

For max speed possible in verification ensure these tweaks are both zero:

Code: Select all

CustomCopyFlags=0                                       (or more exactly: ensure it does not have bit COPY_FILE_NO_BUFFERING (4096))
CustomCopyNoBufferingFromBytes=0

Marco
Posts: 2347
Joined: 27 Jun 2011 15:20

Re: Faster verification in Backup, like in Edit | Compare

Post by Marco »

I see... Those tweaks are already both to zero.
I did some more tests and yes, I can see that backup verification speed is consisten, while sometimes Compare is fast because it uses some cache. This probably tricked me.
Anyway, just to have some numbers, I did a comparison between two files, and crunched 536,188,936 bytes in 7.75 seconds = 65.98 MB/s. About 65% of the read speed of the slower drive. Do you get similar results?
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

admin
Site Admin
Posts: 60567
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Faster verification in Backup, like in Edit | Compare

Post by admin »

Slower. :)

Post Reply