[Script] XYZback - Dir Mirror/Sync Tool 1.0 - March 06, 2017

[Enternal]

Update: Version 1.0 - March 06, 2017
* Recompiled with the newest version of Lazarus (Lazarus 1.6.4) with better Unicode support. Compiled a x64 executable for those who prefer it. Included the necessary XYplorer script to use XYplorer in conjunction with XYZback. Very minor changes in code. Version bump to final 1.0 as it's stable. This may be the last update since the developer of XYplorer is planning on adding syncing capabilities to XYplorer in the near future. Read 'Readme.md' for further details. Greatly updated 'Readme.md' for clarity, other notes, and how to build 'XYZback'.

XYplorer does not really have a built in directory synchronization or mirroring so just for fun, I wrote a little tool that makes it work with Zback by Davor Zorc. I chose Zback mainly because it's small (main standalone executable is 460 KB) and portable yet also powerful. Zback supports scripting, command line inputs, lots of options, and filtering by file types, dates, and more. Hopefully it's somewhat useful...

Zback by Davor Zorc's Website
http://titan.fsb.hr/~dzorc/zback.html

Basically the idea is you have two directory on the left and on the right pane that you want to mirror/sync. You can then run the script which directs the directories and mode of operation to a little helper XYZback (the meat of the functionality). You can then tweak whatever options you want such as inclusion filters or exclusion filters (however syntax useed belongs to Zback therefore filters are separated by commas), recurse sub-directories, and you could even swap the Source and Destination if you want. The resulting Zback scripting parameters are previewed in the Zback Parameters text field which is also updated on the fly every time you edit any of the other options or paths.

Image is large vertically. So depending on which XYplorer's forum theme you choose, you need to scroll down further since you may only see the 1st part of it which gives off the impression that it's the only one.

XYZback.jpg (300.86 KiB) Viewed 5602 times

Setup
Download the script XYZback.xys and copy it into <xyscripts>. Download XYZback_0.5.zip and unload the directory Zback into the <xyscripts> directory. XYZback.exe is inside the Zback folder already. The Source folder will be the pane on the left and the Destination folder will be on the right. After setting up the pane, run the script and choose whether you want to Mirror or Sync. The Preview button will open up Zback which will preview its actions which is recommended. The Run button will have Zback open up and run right away.

Zback, the software itself by Davor Zor, is also already included in the XYZback_0.5.zip with permission from him. Two things to note, Zback itself is portable and save all settings into Zback.ini. However I changed two settings that I think are good to mention. By default, Zback does not copy the files' creation times. It does copy the modification date by default though. Since I was thinking that when you sync, you want it to be an exact copy, therefore I changed the option so that it also copies over the creation date as well. You can change this setting if you want by running Zback.exe, go to the Option tab, and uncheck Set NT Creation dates, and then Save Options. Or open Zback.ini and change SetNTCreationDate to 0. The other change was changing the font to Segoe UI size 9. I'm just lazy to change that back to the default Courier New font so do whatever you want

Another thing to note. By default Zback strictly required administrative privileges which really limits the usefulness of Zback on machine that you don't have full access to. I asked the author himself about this and he provided for me a tool that can turn on/off this requirement called zb_uac. Also already included in Zback folder in XYZback_0.5.zip. Just extract it from zb_uac.zip into the same folder as Zback.exe, run it, and follow directions. I have actually already used it on the Zback.exe so you don't have to do anything unless you really want strict requirement of administrative privileges. And as per his request to give the original link, the actual download link to zb_uac is hidden from his site but can be accessed here http://titan.fsb.hr/~dzorc/zb_uac.zip.

Also, I have setup a new repository for XYZback. Currently very empty with the readme and the likes... I will update that soon... too tired now.
https://github.com/Silvernine0S/XYZback/

Anyway thank you very much to Zback's author Davor Zorc and Don for both of these wonderful tools! Also incoming rant...

XYZback 1.0.zip

XYZback 1.0 (Includes Source & XYS Script)

(2.29 MiB) Downloaded 406 times

XYZback 1.0 x64.zip

XYZback 1.0 x64 Executable

(1.06 MiB) Downloaded 375 times

Old Version Below

[Enternal]

My rant...

So I actually wanted to learn Lazarus/FPC since I heard good things about it. Thought it was a good idea to try it out. Supposedly FPC (the actual compiler) 3.0 and Lazarus (IDE) have good support for Unicode and UTF-8 so I manually compiled it on Windows since it's not offered in binary form yet.

So if you look at my XYS script, you will notice that it sends a couple parameters to XYZback.exe. In particularly:

Code: Select all

XYZback.exe mirror|sync altmethod

Mirror and Sync is obviously which sets the mode of operation Zback will in at the very end. Altmethod on the other hand...

XYZback actually support parameters in this form:

Code: Select all

XYZback.exe mirror|sync "C:\Source" "C:\Destination"

which is self explanatory and basically passes all these parameters directly into XYZback. However due to many issues, I ended up adding an alternative method. What this method does is I would tell XYZback the mode of operation I want, write a UTF-16 LE file with the source and destination path on each line, and then have XYZback read the file. XYZback would then generate another file in UTF-8 with BOM (I know I know... but there is a reason for that and that is my skills lol) in which Zback would then read it and perform the actions. So why even though XYZback could simply then passed the parameters into Zback since Zback supports most of the functions through the command line parameters?

Unicode. Darn Unicode.

Initially XYZback fully support taking in parameters, you set the options, it send out the new parameters into Zback. Not anymore.

FPC, the compiler apparently supports Unicode. Lazarus, the IDE, however has abysmal support for Unicode. Internally it does things in UTF-8... at least it tries too since ANSI errors and weird things pop up at the oddest times. Not all standard libraries fully support UTF-8 or even libraries that were supposed to support UTF-8 borked at the odd things. A library for calling processes using UTF-8 drove me nuts for a day until I found out through a senior member on the forum that what I was going through (another user actually), was a known problem. So a UTF-8 library that sort of works and sort of don't. *sigh*. Anyway I was successfully in getting XYZback in reading in parameters and sending out parameters and internally it was all in UTF-16. It worked great with all the non-English directories I tried until I stumbled upon a directory that has a Chinese character in it and also Japanese characters [活动]. That did not work since it seems passing parameters with these characters just won't work at all. Spend a day and a half and finally decided to generate a UTF-16 LE file with the paths instead. Found out the hard way that the standard libraries just gives me hell trying to write to a UTF-16 LE file. I then decided to rewrite it all to UTF-8 internally which works fine and output the text file to UTF-8 without BOM which works smoothly. Zback recognize UTF-16 LE files but not UTF-8 without BOM. So now I had XYZback output the text file in UTF-8 with BOM to have Zback actually read it correctly

So that should be smooth right? Well apparently those Chinese & Japanese characters are just bad for passing as parameters so that means I can't pass them as parameters into XYZback as well. Darn. All that work is now pointless. Programmed XYZback to read in UTF-8 files as that should be the easiest. XYplorer however does not support writing text files in UTF-8 so... back to programming. Reading in a UTF-16 files was also hell since the standard libraries just croak all over the place and does weird things. After another day and a half, thanks to theo's utf8tools that does most of the heavy lifting, finally got XYZback to read in UTF-16 files... but by reading the UTF-16 LE file, convert it to UTF-8 file, and then read the file again to get the paths .

So in the end, XYZback does thing not very elegantly at all. It read in a "parameter file" instead of actual parameters. It outputs a "parameter file" for Zback instead of passing actual parameters. But at the very least that stupid folder with the 活动 works when I sync/mirror it. I'm guessing that most of the time you won't come across stupid folders with a mix of characters that don't play well with each other through the command line but if you do... it should work at least... I think. I hope...

Or maybe I should not be such a perfectionist for those 1% chances of coming across those types of folders and reactivate the function of passing actual parameters into Zback instead of a file. What do you think?

So in conclusion, I suck at coding. If I was a good coder, I could go past this even with all these problems with the tools. And Unicode is a PITA. A REAL PITA. Oh, and Lazarus's abysmal handling of Unicode and even UTF-8 as libraries that sometimes work and sometimes don't when it involves encoding is something that must always be considered. I mean there were times that I was getting results that were vastly different each time I hit a button that was simply doing a function on a constant data for testing purposes . Mostly though, I suck at coding. Thank you Don for solving the problems with Unicode in XYplorer. Though... it would be nice if you can output files in UTF-8 with/without BOM...

[admin]

Ha, I love that rant. Reminds me to the main character in the fabulous book I'm just reading: The Martian. Have not seen the movie (yet) but the book is SUPER!

Conc. UTF8, there is SC utf8encode()... not enough?

[highend]

Have not seen the movie (yet) but the book is SUPER!

The film as well. Best one I've seen this year in the movie...

Sorry, no rants from me, I'm using BeyondCompare / SyncBackPro to do stuff like that (and I need the feature to copy locked files (from SB) which ZBack hasn't)

[Enternal]

Ha, I love that rant. Reminds me to the main character in the fabulous book I'm just reading: The Martian. Have not seen the movie (yet) but the book is SUPER!

Conc. UTF8, there is SC utf8encode()... not enough?

Yeah I heard great things about The Martian as well. I recently got the book and was sad I did not get to watch the movie since on the day it was released in theaters, my friends invited me but I was sick
UTF8Encode... this is what happens when I haven't done any scripting in XYS in a long time and I even forgot that function existed... OK. Back to cleaning up my code...

@highend
Yeah I don't expect it to be helpful to everyone but for quick and dirty stuff, it should be useful. Just like you though, I also have BeyondCompare and FreeFileSync integrated with XYplorer when I need more certain features and options. For extra precaution, I have SyncBackPro 5 (don't care about version 6 at the moment) set up as well. Lately though I have been using Bvckup2 although that's not really a sync tool but rather a backup tool... Love all 4 of them so much which is why I have them all set up even if it's very redundant lol.

[Enternal]

Updated to version v1.0. Nothing really significant and no new feature. Basically recompiled everything with optimization settings, organized all files, update Readme files for more clarity, and more.

Taken directly from the GitHub page:

Recompiled with the newest version of Lazarus (Lazarus 1.6.4) with better Unicode support. Compiled a x64 executable for those who prefer it. Included the necessary XYplorer script to use XYplorer in conjunction with XYZback. Very minor changes in code. Version bump to final 1.0 as it's stable. This may be the last update since the developer of XYplorer is planning on adding syncing capabilities to XYplorer in the near future. Read 'Readme.md' for further details. Greatly updated 'Readme.md' for clarity, other notes, and how to build 'XYZback'.

Check 1st for links to the new version as well as the link to this program's GitHub page.

[bdeshi]

Can relate to your rant. I'd tried out Lazarus too (remember xyskinner v2? ) but the ide felt so very... cobbled together.

Learning experiences are always a good thing though. The end result is great too.

[xyplorerköln]

When running the 1.0 script in XYplorer 18.70.0100, XYplorer reports scripting error:

"Dubious syntax: tu"

in line:

WriteFile("<xyscripts>\Zback\XYZbackStart.txt", "<path 1><crlf><path 2><crlf>",o,tu);

[jupe]

You should be able to change it to this, it looks like just the quotes are missing:

Code: Select all

WriteFile("<xyscripts>\Zback\XYZbackStart.txt", "<path 1><crlf><path 2><crlf>", "o", "tu");

I haven't tested the entire script though...