Synchronize 2 folders

[admin]

Yes, I understand, but it's not possible.

As a sign of good will I reduced one click when showing the report.

[admin]

If Configuration | Tags | Copy tags on backup operations is ON (it also controls Sync; next version it's called "Copy tags on backup and sync operations") then

a) Should tags be updated only for those files that have been copied or overwritten? If yes, what about tagged folders?

OR

b) Should tags be updated for all items in the target, so that you get freshly synced tags even if the items themselves were up-to-date?

[Filehero]

Just "installed" my scripted backup synchronizer on the 2nd machine - works great.

Definitions

Code: Select all

bak_src               = M:\Backup
bak_tgt               = X:\Backup
bak_dirs              = Graphics|Infos_Notes|Installers|MySettings|Partitions\Windows 10|Personal|Temp|Tracking|VirtualMachines

script

Code: Select all

include "libs\fh_lib.xyi";
include "libs\local_lib.xyi";

  $BACK_DIRS = getLocalConfig("bak_dirs");
  $SRCBASE   = getLocalConfig("bak_src");
  $TGTBASE   = getLocalConfig("bak_tgt");

  foreach($dir, $BACK_DIRS) {
    $target = "$TGTBASE\$dir";
    assertItemExists($target, "dir");
    $logfile = "$target\synclog_" . recase(gpc($target, "component", -1, 0), "l") . "_<date yyyymmdd>.log";
    sync("$SRCBASE\$dir", $target, 1, 0, 4, , $logfile);
  }

The only change required for another system is the bak_folders enumeration.

What a time saver.

/e: as for the tags

1. I think both a) and b) are valid. If a tag/label is just another attribute of a file, then it is rather a). If the tags encode a major piece of information (take an raw image file, which will usually never change again once it has been copied, but will accumulate a lot of keywords over time) I would hope for b).

2. What about making the differentation on the SC level (SC setting), where you can offer three options a), b) and c) "no tags will be copied/synchronized at all"?

FH

[admin]

/e: as for the tags

1. I think both a) and b) are valid. If a tag/label is just another attribute of a file, then it is rather a). If the tags encode a major piece of information (take an raw image file, which will usually never change again once it has been copied, but will accumulate a lot of keywords over time) I would hope for b).

2. What about making the differentation on the SC level (SC setting), where you can offer three options a), b) and c) "no tags will be copied/synchronized at all"?

FH

Yes, thanks. I will need some more time for this, it's probably something for phase 2...

[Filehero]

The speed update is

Q (SC sync): from the progress dialog it seems the "skip" decision is done only when the file item becomes the current on in the processing loop. Well, that's ok because this is how iterations and effective coding works.

However, in specific situations (little but huge files) this is somewhat irritating because at the beginning it always appears as if all files are going to be synced again (though only one has been added since the last sync run, for instance).


For example: 10 files, 9 already synced (~140 GB), one new with ~30 GB

Now, when the new file is synced I see for quite some time ...

Code: Select all

Files processed: 0 of 10
Bytes copied: xx bytes of 170,21 GB  // meaning (0 skipped)
remaining time 1:2x

... where I always think there must be something wrong with my script setup, it's "just one file to be synced!"

Of course, with lots of files (and same total size) this isn't any problem because the skipped file indicator (and remaining time) is converging on the expected values quickly.

Maybe there is a sweet spot (total size / file item number?) where it makes sense to determine the number of skipped files (and size, time) in advance*?

*: accepting, that not-yet-processed files could change afterwards.

[admin]

Maybe it's the wording "Files processed:" that irritates you?

Frankly I don't think I should add another layer to this. The code behind all this is already very complex (not because of the things to be done, but because of the things that can go wrong -- it's always like this in programming). And whose sweet spot would that be? An element of taste at this point is just calling for trouble (and for support ).

[Filehero]

Maybe it's the wording "Files processed:" that irritates you?

No, rather that during the initial phase it seems that the sources will be synced completely every time.

Frankly I don't think I should add another layer to this. The code behind all this is already very complex (not because of the things to be done, but because of the things that can go wrong -- it's always like this in programming). And whose sweet spot would that be? An element of taste at this point is just calling for trouble (and for support ).

This I fully understand, that's not worth the trouble.

[Filehero]

Don, have you changed something with the progress numbers?

Today (before update to 18.30.0021) I saw a negative number for the "bytes copied" count.

[admin]

No.

Negative? How negative? Reproducible?

[Filehero]

Negative? How negative?

Something like "-1.234.765 bytes of 176 GB", the number was going up (towards 0) with every byte copied.

Reproducible?

Maybe. I think if it is reproducible it is related to the job size in bytes. If I find some time the next days (weekend) I will make some experiments with my sample library. It's not a biggie anyway.

Btw, SC sync in loop is incredibly fast now.

[admin]

Reproducible?

Maybe. I think if it is reproducible it is related to the job size in bytes. If I find some time the next days (weekend) I will make some experiments with my sample library. It's not a biggie anyway.

Well, looks like a biggie to me. I mean, negative file sizes! Hola! I need to know all about this.

[Filehero]

Well, looks like a biggie to me. I mean, negative file sizes! Hola! I need to know all about this.

Does any existing code path could explain it? A hint could help me with finding a reproducible test case.

[admin]

No. I just looked again, and made some tests with huge files. Cannot see how this is possible.

[Filehero]

Ok, I'll keep an eye on it, and with the slightest hint I'll start some tests.

[Filehero]

Peng!

The progress view

sync_neg-bytes.png (13.57 KiB) Viewed 1682 times

The source folder

sync_neg-byte_sources.png (11.25 KiB) Viewed 1682 times

Note: this job is the last one in the SC sync job loop.