Smarter TimeStamp Detection

Features wanted...
Post Reply
Stef123

Smarter TimeStamp Detection

Post by Stef123 »

The added support for portable devices raises my hopes again to "fix" an old issue. To reproduce, please plug in any kind of device or storage that does not equal the file system of your local machine (mostly NTFS I assume). Let's take an external drive formatted to FAT32.

Now copy a few hundred pics onto that drive - from your local NTFS to FAT32.
Now run SyncSelect > Select Different. This will yield about 50% of files as different, even though XY itself has just copied them to the other side.

This results in tons of unnecessary copy operations and backups which take much longer than with other commanders, which are smart enough to handle this without redundant wear and tear.

Portable devices are slow to start with. And they come in a variety of formats other than NTFS. Good reasons to tackle this issue, or so I hope :roll:

admin
Site Admin
Posts: 50916
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: Smarter TimeStamp Detection

Post by admin »

OK, try Sync Select in the next beta and see if it's fixed.
FAQ | XY News RSS | XY Twitter | Stay home! 🎸 Or wear a mask! 😷

Stef123

Re: Smarter TimeStamp Detection

Post by Stef123 »

Hm, repeated my quick and dirty test from above with 14.80.0215
Copied a folder from local NTFS to external FAT32 USB stick.
Ran SyncSelect > Select Different
Again, it flags roughly 50% as different :cry:

admin
Site Admin
Posts: 50916
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: Smarter TimeStamp Detection

Post by admin »

OK, I had a wrong conception of the nature of the problem. Now I see there are small (and random) differences in the timestamps of the copied files. So what we need is a little tolerance when comparing the timestamps. From my tests it seems that *2 seconds* would be enough. So two timestamps that differ by 2 seconds would be seen as equal. Okay?
FAQ | XY News RSS | XY Twitter | Stay home! 🎸 Or wear a mask! 😷

Stef123

Re: Smarter TimeStamp Detection

Post by Stef123 »

Yep. That should do the trick. I've seen similar techniques elsewhere, like FreeCommander.

Marco
Posts: 2309
Joined: 27 Jun 2011 15:20

Re: Smarter TimeStamp Detection

Post by Marco »

Stef123 wrote:Yep. That should do the trick. I've seen similar techniques elsewhere, like FreeCommander.
I confirm. Most backup softwares have this feature where they allow a 2-second difference between source and destination.
Check also robocopy and its /fft switch ( http://ss64.com/nt/robocopy.html )
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

Enternal
Posts: 1174
Joined: 10 Jan 2012 18:26

Re: Smarter TimeStamp Detection

Post by Enternal »

Do you think it would be good if the user can modify that 2 second value? It's better than revisiting this thing again in the future. xplorer2 did something like that too to avoid it happening. Obviously default is 2 seconds.

admin
Site Admin
Posts: 50916
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: Smarter TimeStamp Detection

Post by admin »

Enternal wrote:Do you think it would be good if the user can modify that 2 second value? It's better than revisiting this thing again in the future. xplorer2 did something like that too to avoid it happening. Obviously default is 2 seconds.
I thought about this, but then I thought: One tweak less. It's easy to add this later if really necessary.

PS: Thought again and changed my mind. Tweak comes.
FAQ | XY News RSS | XY Twitter | Stay home! 🎸 Or wear a mask! 😷

Stefan
Posts: 1360
Joined: 18 Nov 2008 21:47
Location: Germany, EU

Re: Smarter TimeStamp Detection

Post by Stefan »

admin wrote:
Enternal wrote:Do you think it would be good if the user can modify that 2 second value? It's better than revisiting this thing again in the future. xplorer2 did something like that too to avoid it happening. Obviously default is 2 seconds.
I thought about this, but then I thought: One tweak less. It's easy to add this later if really necessary.

PS: Thought again and changed my mind. Tweak comes.

Fine! :tup:

Then that should work for 'daylight saving time' issue also ( 18:12:34 <> 17:12:34 )


 

admin
Site Admin
Posts: 50916
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: Smarter TimeStamp Detection

Post by admin »

Stefan wrote:Then that should work for 'daylight saving time' issue also ( 18:12:34 <> 17:12:34 ) 
Not necessary. This issue has been handled as well v14.80.0217. Hasn't it?
FAQ | XY News RSS | XY Twitter | Stay home! 🎸 Or wear a mask! 😷

Stefan
Posts: 1360
Joined: 18 Nov 2008 21:47
Location: Germany, EU

Re: Smarter TimeStamp Detection

Post by Stefan »

admin wrote:Not necessary.
Ohh sorry, I just thought, simple something like "TimestampDiff=60" would be good for that issue.
This issue has been handled as well v14.80.0217. Hasn't it?
Ahh, I see now there is already something going on. Not tested yet, but *searchingformythumpdrive*.... later on.



 

Stef123

Re: Smarter TimeStamp Detection

Post by Stef123 »

v14.80.0217 - again the same test as above.
Now it flags everything as different :veryconfused: seems like it works the other way round, at least it's not the 50% anymore :titter: we're getting closer ...

admin
Site Admin
Posts: 50916
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: Smarter TimeStamp Detection

Post by admin »

Whoops, that was only close to perfection. :whistle: Upload later...
FAQ | XY News RSS | XY Twitter | Stay home! 🎸 Or wear a mask! 😷

Stef123

Re: Smarter TimeStamp Detection

Post by Stef123 »

Thanks. Just ran my first backup with real data, and sure enough, the time savings are SIGNIFICANT. This might even solve some performance complaints of other users. Can't wait to try it on network drives next week. :tup:

Post Reply