Special Sync and Update

[bjarne.dein]

Hi,

I'm very interesting in creating a script that can update a destination Folder according a Source folder, and a little more than that...

The background here is all my Photos organized in folders like this: [YYYY]\[MM]\[YYYY_MM_DD] - [Event name], where all source files are arriving on my PC, and then backup til my two NAS drives.
When making the first backup to the NAS not all source folders might have got a [Even Name] yet, and not even all photos from all cameras for that day yet, which result in a structure difference as on the two screen dumps attached here

2016-11-17_00-41-29 - Source folder.jpg

and

2016-11-17_00-41-47 - Destination Folder.jpg

.
As you can see, I've now updated all Source folders with [Event Name], and the Destination only some are updated, and some still only holds the raw Date name.

I'd like to get a script that will compare the Source and Dist folder names, and update the Dist folders with correct Source [Event name], and accordingly add eventual new files from Source.

An additional challenge is also, that sometime in the analyze phrase of the raw source folder, I have to split the folder into two or more folders with same date, but different [Event name], and the raw folder has already been backup. So now for the update process to be complete, the according extra source folders will have to be created on the destination, and the files in the Dist folder split into the correct [Event name] folder, according to the source.

I think you get the picture

Who is the one that has the script that can make this happen

Many thanks in advance.

Best Regards,

Bjarne

[highend]

To get this right...

Let's say the source contains:
2016_05_01
This one is backuped once

Now you split it in three folders in the source:
2016_05_01 - Event name 1
2016_05_01 - Event name 2
2016_05_01 - Event name 3

and now the script should create these three folders in the destination
and then move all files in "2016_05_01" into:
2016_05_01 - Event name 1
2016_05_01 - Event name 2
2016_05_01 - Event name 3

and finally remove "2016_05_01"?

[highend]

This should do it (use it with testfolders first!):

Only change the first three variables...
$dstFolders can contain both of your NAS drives, separated with a "|".
If you want to use it only on one, remove the "|" and the second path

Code: Select all

    $srcFolder  = "R:\a";
    $dstFolders = "R:\b|T:\b";

    // If you want to delete subfolders in the destination that DO NOT have
    // a (date-matched) folder in the source, set the next variable to "true"
    $deleteDstFolder = false;


    // =======================================================================
    setting "BackgroundFileOps", 0;
    $srcSubFolders = listfolder($srcFolder, , 2+4, <crlf>);

    foreach($dstFolder, $dstFolders, , "e") {
        if (exists($dstFolder) != 2) { continue; }

        $dstSubFolders = listfolder($dstFolder, , 2+4, <crlf>);

        // Loop over all subfolders inside the destination folder
        foreach($folder, $dstSubFolders, <crlf>) {
            // Source subfolder already exists, look if destination has missing files...
            if (exists($srcFolder . "\" . $folder) == 2) {
                $srcFiles   = listfolder($srcFolder . "\" . $folder, , 1+4, <crlf>);
                $dstFiles   = listfolder($dstFolder . "\" . $folder, , 1+4, <crlf>);
                $dstPattern = regexreplace($dstFiles, "(#|\[)", "[$1]");

                $missing = formatlist($srcFiles, "ef", <crlf>, "!$dstPattern");
                if ($missing) {
                    $missing = formatlist(regexreplace($missing, "(?:^|\r?\n)(.{1})", "|$srcFolder\$folder\$1"), "e");
                    copyto $dstFolder . "\" . $folder, $missing;
                }

            // Source subfolder doesn't exist, look for folders with the same leading date
            } else {
                $dstDate = gettoken($folder, 1, " - ");
                $srcSameDate = formatlist($srcSubFolders, "ef", <crlf>, "$dstDate - *");
                if ($srcSameDate) {
                    foreach($item, $srcSameDate, <crlf>) {
                        $dstFiles   = listfolder($dstFolder . "\" . $folder, , 1+4, <crlf>);
                        $dstPattern = regexreplace($dstFiles, "(#|\[)", "[$1]");

                        $srcFiles = listfolder($srcFolder . "\" . $item, , 1+4, <crlf>);
                        $existing = formatlist($srcFiles, "ef", <crlf>, "$dstPattern");
                        $existing = formatlist(regexreplace($existing, "(?:^|\r?\n)(.{1})", "|$dstFolder\$folder\$1"), "e");
                        $missing  = formatlist($srcFiles, "ef", <crlf>, "!$dstPattern");
                        $missing  = formatlist(regexreplace($missing, "(?:^|\r?\n)(.{1})", "|$srcFolder\$item\$1"), "e");

                        moveto $dstFolder . "\" . $item, $existing, , 2;
                        copyto $dstFolder . "\" . $item, $missing, , 2;
                    }
                    delete 1, 0, $dstFolder . "\" . $folder;

                // Delete dst subfolder because no date-matching src subfolder exists (optional)
                } else {
                    if ($deleteDstFolder) { delete 1, 0, $dstFolder . "\" . $folder; }
                }
            }
        }
    }

[bjarne.dein]

Thanks Highend (What a name )

You got it almost right:-)

The structure is right, but a minor principal need to be straighten out.

Under most circumstances it will be one 2 one relation, which just means rename of the existing destination folder, and then update it with eventual new files.

So to follow that principal on one source folder split into three:
* The existing destination folder will be renamed to first of source folder (2016_05_01 - Event name 1)
* The two others will be created (2016_05_01 - Event name 2 and 2016_05_01 - Event name 3)
* Move files from destination "2016_05_01 - Event name 1" to the respective "2016_05_01 - Event name 2" and "2016_05_01 - Event name 3"
* Add eventual new files from Source to Destination folders

[bjarne.dein]

Hi Highend,

In regards to the scripts, which looks promising:-)

Is it possible to add such script as a menu(and shortcut) as Select Sync, and in a way that is automatically picks up the current source and destination folders from active windows?

If so, how?

Beat Regards ,

Bjarne

[highend]

So to follow that principal on one source folder split into three:
* The existing destination folder will be renamed to first of source folder (2016_05_01 - Event name 1)
* The two others will be created (2016_05_01 - Event name 2 and 2016_05_01 - Event name 3)
* Move files from destination "2016_05_01 - Event name 1" to the respective "2016_05_01 - Event name 2" and "2016_05_01 - Event name 3"
* Add eventual new files from Source to Destination folders

Is there any important reason to do it this way?
Because it would make the script imho unnecessarily complicated. The current design only creates one additional folder instead of renaming the old one. There is one additional file movement involved but moving files is nearly instant (only the master file table entries are changed)

If so, how?

That depends.

E.g. you've opened the NAS 1 destination folder in the right / lower pane and in the active (left / upper) the source folder than instead of using:

Code: Select all

    $srcFolder  = "R:\a";
    $dstFolders = "R:\b|T:\b";

You could use instead:

Code: Select all

    $srcFolder  = <curpath>;
    $dstFolders = get("path", "i");

But be careful, always remember that the active pane point to the source folder!
If you've accidentally activated the pane that points to the destination folder you'll be rather screwed...

[bjarne.dein]

Is there any important reason to do it this way?

As of now, I don't foresee any important.
Thanks a lot so far, I'll give it a try, and see how it goes - and I'll let you know:-)