Custom Copy ("Backup To...") showing useless "time remaining"

[Creat]

Hello,

it's been a while since I've posted any bugs or complaints, since this is seriously one of the best pieces of software I know (and use constantly).

But recently I've come to use the "Backup to..." function a bit more. I assume that just means the "custom copy" in general, but unfortunately the estimated time is generally completely useless. I most cases it starts with a few seconds/minutes and counts UP at various speeds, even though the job is ~2TB in size. It might be because it counts a (somewhat significant with 800 GB) amount of files that are skipped for the purposes of calculating the time.
The first few times I've checked the time remaining said something in the mid 20 minute range, counting upwards quite quickly. Now it's showing mit 40 minutes, and holding somewhat steady (occasionally rising and falling). It should be noted that even at the maximum theoretical speed the job will take at least another 1.5 hours.

As an additional wish I'd love to be able to see the current transfer speed in addition to the total average that is already shown. This would mean that for larger jobs like this I actually get a feeling if it's progressing slow or fast, not just by seeing the rate go slightly up or down. I imagine a 10s window or something would give a not too hectic 'live' value.

Otherwise keep up the great work and thank you for a just generally awesome file manager.

TheCreat

[admin]

The formula for estimating the remaining time is very simple. Say you have copied 200 MB of 800 MB total, and this took 1 second, then the whole job is estimated to take 4 seconds, so the remaining time is estimated to be 3 seconds.

What you are seeing has to be effects of optimizations on various levels (and probably an Intel chip bug ). I can't see a way to improve my formula.

[DmFedorov]

I can't see a way to improve my formula.

The formula must take into account the number of files.
For example: I took the folder with 2000 files (the total size is 32MB)
Made a copy. Copying lasted 9.86 seconds.
With the help of Npp I deleted the contents of files. (the total size is 0 bytes)
Made a copy. Copying lasted 8.09 seconds.

[admin]

OK, I will try to smarten it up a bit.

[Filehero]

I can't see a way to improve my formula.

For hard drives: include angular velocity.

[admin]

[DmFedorov]

in v16.80.0008 is better, but in the initial period (after counting the number of files and folders) remaining time shows unrealistically large values, after it this time starts to approach the real result.
And even by counting the copying of the same (but empty) files, the remaining time in the initial period displays unrealistically large values.
Up to v16.80.0008 the remaining time was always less than the real remaining time.

The scan was made by copying a folder consisting of 500 subfolders and 4000 files.
(I think that the time to create a folder is counted and equated to copying a file of zero size)

I hope that you will make one more attempt.

[admin]

I will.

[TheTodd]

It seems to depend on the size of the files.
Copying 200 files totaling 500MB, seems much quicker than 4000 files totaling 500MB.
so that can really throw off the math when using linear calculations.

This can be an issue with Windows copy times too.

[admin]

Yes.

I'm already taking that into account with v18.60.0009. Is it still that bad in v18.60.0009?

[DmFedorov]

Is it still that bad in v18.60.0009?

bad. (Sorry, but noticed the new version too late.)

25% is too much. It will not suit anyone. I am personally ready to wait 10-20 seconds (as it makes Windows), not more.
In this 10-20 seconds Windows first counts remaining time (sufficiently accurately) and i see "Estimate". You can check.
And in XY, 25% is added to the counting time of files and folders.

Why we need to know remaining time? To not wait. Do something else at this remaining time.
By calculating remaining time, you can, for example, wait max 10 second, than use the ± symbol and indicate the degree of inaccuracy in minutes, hours, percentages, and after 25% you can not use ±.

[admin]

I don't want to make it too complicated. Windows copying is a complex thing with lots of factors affecting the speed. I think most users will know from experience that "Time remaining" is just a rough guess.

Next version the "Time remaining" is shown earlier. You have to wait at most 10 seconds for it.

[Creat]

Sorry for the late reply, somehow I missed the reply-notification mail.

The formula for estimating the remaining time is very simple. Say you have copied 200 MB of 800 MB total, and this took 1 second, then the whole job is estimated to take 4 seconds, so the remaining time is estimated to be 3 seconds.

The problem with this was (or still is?) that it also includes files that weren't actually copied. Imagine you have a large folder that you are using "backup to..." with (say 200 files, 100s of MB each), and you cancel it at after 100 files. Then you restart it, it means the first 100 files are skipped. It then thinks copying 100 files took ~1 second, and that's obviously nonsense. I would propose: whenever you encounter a file that is skipped, don't just mark it as done, but also remove it from the number of files to be copied. So now it's about to process files 1/100.

A completely different approach to the problem would be to use the current transfer rate to estimate the remaining time. This is gonna be a bit off in most cases (copying small files vs. large files), but should provide a reasonable approximation for homogeneous copies (mostly small or mostly large files). It might be even possible to get an approximation of the cost for creating the node entry (I mean the overhead per

[admin]

Sorry for the late reply, somehow I missed the reply-notification mail.

The formula for estimating the remaining time is very simple. Say you have copied 200 MB of 800 MB total, and this took 1 second, then the whole job is estimated to take 4 seconds, so the remaining time is estimated to be 3 seconds.

The problem with this was (or still is?) that it also includes files that weren't actually copied. Imagine you have a large folder that you are using "backup to..." with (say 200 files, 100s of MB each), and you cancel it at after 100 files. Then you restart it, it means the first 100 files are skipped. It then thinks copying 100 files took ~1 second, and that's obviously nonsense. I would propose: whenever you encounter a file that is skipped, don't just mark it as done, but also remove it from the number of files to be copied. So now it's about to process files 1/100.

For this scenario it is true. But how likely is this? I think a general approach should assume constant skip rate (skipped bytes per processed bytes), and then using the processed bytes (= copied and skipped bytes, as opposed to just the copied bytes) is a good strategy.

A completely different approach to the problem would be to use the current transfer rate to estimate the remaining time. This is gonna be a bit off in most cases (copying small files vs. large files), but should provide a reasonable approximation for homogeneous copies (mostly small or mostly large files). It might be even possible to get an approximation of the cost for creating the node entry (I mean the overhead per

I will make some tests... What you mean by "current transfer rate"? The actual transfer rate in every moment is changing all the time.