Page 9 of 12
Re: Synchronize 2 folders
Posted: 11 Sep 2017 20:20
by admin
Yes, I understand, but it's not possible.
As a sign of good will I reduced one click when showing the report.
Re: Synchronize 2 folders
Posted: 12 Sep 2017 16:15
by admin
If
Configuration | Tags | Copy tags on backup operations is ON (it also controls Sync; next version it's called "Copy tags on backup and sync operations") then
a) Should tags be updated only for those files that have been copied or overwritten? If yes, what about tagged folders?
OR
b) Should tags be updated for all items in the target, so that you get freshly synced tags even if the items themselves were up-to-date?
Re: Synchronize 2 folders
Posted: 12 Sep 2017 21:34
by Filehero
Just "installed" my scripted backup synchronizer on the 2nd machine - works great.
Definitions
Code: Select all
bak_src = M:\Backup
bak_tgt = X:\Backup
bak_dirs = Graphics|Infos_Notes|Installers|MySettings|Partitions\Windows 10|Personal|Temp|Tracking|VirtualMachines
script
Code: Select all
include "libs\fh_lib.xyi";
include "libs\local_lib.xyi";
$BACK_DIRS = getLocalConfig("bak_dirs");
$SRCBASE = getLocalConfig("bak_src");
$TGTBASE = getLocalConfig("bak_tgt");
foreach($dir, $BACK_DIRS) {
$target = "$TGTBASE\$dir";
assertItemExists($target, "dir");
$logfile = "$target\synclog_" . recase(gpc($target, "component", -1, 0), "l") . "_<date yyyymmdd>.log";
sync("$SRCBASE\$dir", $target, 1, 0, 4, , $logfile);
}
The only change required for another system is the
bak_folders enumeration.
What a time saver.
/e: as for the tags
1. I think both a) and b) are valid. If a tag/label is just another attribute of a file, then it is rather a). If the tags encode a major piece of information (take an raw image file, which will usually never change again once it has been copied, but will accumulate a lot of keywords over time) I would hope for b).
2. What about making the differentation on the SC level (SC setting), where you can offer three options a), b) and c) "no tags will be copied/synchronized at all"?
FH
Re: Synchronize 2 folders
Posted: 13 Sep 2017 16:34
by admin
Filehero wrote:/e: as for the tags
1. I think both a) and b) are valid. If a tag/label is just another attribute of a file, then it is rather a). If the tags encode a major piece of information (take an raw image file, which will usually never change again once it has been copied, but will accumulate a lot of keywords over time) I would hope for b).
2. What about making the differentation on the SC level (SC setting), where you can offer three options a), b) and c) "no tags will be copied/synchronized at all"?
FH
Yes, thanks. I will need some more time for this, it's probably something for phase 2...
Re: Synchronize 2 folders
Posted: 15 Sep 2017 22:50
by Filehero
The speed update is
Q (SC sync): from the progress dialog it seems the "skip" decision is done only when the file item becomes the current on in the processing loop. Well, that's ok because this is how iterations and effective coding works.
However, in specific situations (little but huge files) this is somewhat irritating because at the beginning it always appears as if all files are going to be synced again (though only one has been added since the last sync run, for instance).
For example: 10 files, 9 already synced (~140 GB), one new with ~30 GB
Now, when the new file is synced I see for quite some time ...
Code: Select all
Files processed: 0 of 10
Bytes copied: xx bytes of 170,21 GB // meaning (0 skipped)
remaining time 1:2x
... where I always think there must be something wrong with my script setup, it's "just one file to be synced!"
Of course, with lots of files (and same total size) this isn't any problem because the skipped file indicator (and remaining time) is converging on the expected values quickly.
Maybe there is a sweet spot (total size / file item number?) where it makes sense to determine the number of skipped files (and size, time) in advance*?
*: accepting, that not-yet-processed files could change afterwards.
Re: Synchronize 2 folders
Posted: 16 Sep 2017 10:31
by admin
Maybe it's the wording "Files processed:" that irritates you?
Frankly I don't think I should add another layer to this. The code behind all this is already very complex (not because of the things to be done, but because of the things that can go wrong -- it's always like this in programming). And whose sweet spot would that be? An element of taste at this point is just calling for trouble (and for support
).
Re: Synchronize 2 folders
Posted: 16 Sep 2017 10:55
by Filehero
admin wrote:Maybe it's the wording "Files processed:" that irritates you?
No, rather that during the initial phase it seems that the sources will be synced completely every time.
admin wrote:Frankly I don't think I should add another layer to this. The code behind all this is already very complex (not because of the things to be done, but because of the things that can go wrong -- it's always like this in programming). And whose sweet spot would that be? An element of taste at this point is just calling for trouble (and for support
).
This I fully understand, that's not worth the trouble.
Re: Synchronize 2 folders
Posted: 18 Sep 2017 20:36
by Filehero
Don, have you changed something with the progress numbers?
Today (before update to 18.30.0021) I saw a negative number for the "bytes copied" count.
Re: Synchronize 2 folders
Posted: 18 Sep 2017 20:41
by admin
No.
Negative? How negative? Reproducible?
Re: Synchronize 2 folders
Posted: 18 Sep 2017 20:54
by Filehero
admin wrote:Negative? How negative?
Something like "-1.234.765 bytes of 176 GB", the number was going up (towards 0) with every byte copied.
admin wrote:Reproducible?
Maybe. I think if it is reproducible it is related to the job size in bytes. If I find some time the next days (weekend) I will make some experiments with my sample library. It's not a biggie anyway.
Btw, SC sync in loop is incredibly fast now.
Re: Synchronize 2 folders
Posted: 18 Sep 2017 21:22
by admin
Filehero wrote:admin wrote:Reproducible?
Maybe. I think if it is reproducible it is related to the job size in bytes. If I find some time the next days (weekend) I will make some experiments with my sample library. It's not a biggie anyway.
Well, looks like a biggie to me. I mean, negative file sizes! Hola! I need to know all about this.
Re: Synchronize 2 folders
Posted: 18 Sep 2017 21:42
by Filehero
admin wrote:Well, looks like a biggie to me. I mean, negative file sizes! Hola! I need to know all about this.
Does any existing code path could explain it? A hint could help me with finding a reproducible test case.
Re: Synchronize 2 folders
Posted: 18 Sep 2017 21:52
by admin
No. I just looked again, and made some tests with huge files. Cannot see how this is possible.
Re: Synchronize 2 folders
Posted: 18 Sep 2017 21:57
by Filehero
Ok, I'll keep an eye on it, and with the slightest hint I'll start some tests.
Re: Synchronize 2 folders
Posted: 19 Sep 2017 21:19
by Filehero
Peng!
The progress view
- sync_neg-bytes.png (13.57 KiB) Viewed 1675 times
The source folder
- sync_neg-byte_sources.png (11.25 KiB) Viewed 1675 times
Note: this job is the last one in the SC sync job loop.