Backup To - missing feature

[jacky]

Okay so you know how when the source for a Backup operation is the Catalog, then is states "multiple sources locations" and recreate the full folder structure on destination ?

Now that's a pretty cool thing for sure, but here is the problem I'm facing today : I want to be able to be in List, select a bunch of files, and then start a backup for those select files only.

But the thing is, I need the backup to be done with that same structure preservation, so if destination is D:\Backup I don't want my files (say they were in E:\Project\2007\compta) to end up in D:\Backup but still to go in D:\Backup\E\Project\2007\compta !

(And no, I don't want to seach for * from MyComputer every time just to do that, nor do I have the time to create a temporary Catalog item and start the backup from there, etc)

Basically I wish there was an option to turn this source structure preservation on manually, regardless of whether or not I selected multiple items from different locations, etc

Now that would allow for some real, on-demand backup!

The easiest way I see this would be for an option to be added in UDC|Backup To, so it behaves as it does now when off, and recreates the full source structure when turned on; or some new variable <source> that would be the structure, allowing even destination such as "D:\Backup\ <source>\<dyyyy-mm-dd>\" as well as rending this feature usable from the other interfaces (window, menu Edit|M/C/B To) of the Backup feature!

Could that be possible, please ?

[admin]

<source>...

So in a multi-location context <source> would be resolved on a per-file basis:
File: E:\Project\2007\compta\file.txt
<source>: E\Project\2007\compta\ (dropped :, backslashed)

In a sinlge-location context <source> would be resolved based on current path:
Curpath: E:\Project\2007\compta\
<source>: E\Project\2007\compta\ (dropped :, backslashed)

Okay, I like it. But I'm not sure if I like "<source>". Although I don't have a better idea.

[jacky]

yep, exactly!

Well, I used <source> because it relates to the source structure (since either way it's always actually the source path - I mean on single location current path == file path), but maybe to avoid confusion a <scepath> might be better... and keeps somewhat consistent with <curpath> too.

[admin]

yep, exactly!

Well, I used <source> because it relates to the source structure (since either way it's always actually the source path - I mean on single location current path == file path), but maybe to avoid confusion a <scepath> might be better... and keeps somewhat consistent with <curpath> too.

<srcpath> would seem a more common abbreviation to me. But <source> is one letter less... I'll meditate about it...

[admin]

Actually "D:\Backup\<srcpath>\<dyyyy-mm-dd>\" is brilliant. It's *exactly* how I organize my own backups since years, and the system is great! Now with the <srcpath> variable, the system will be perfect like a diamond!

Before I add it, it would be a good time to decide how to format the source path. The goals:
(1) Have a good chance to implement an automatic "Restore" command.
(2) Make it visually clear, what part of the whole backup path is the source path.
(3) Distinguish drive letters from (UNC) servers with one-letter names (if that's allowed, I don't know) to avoid merging.
(4) Sort all servers together after all drives.

I quote my own solution to all above goals from http://www.xyplorer.com/xyfc/viewtopic. ... 6722#16722 :

So currently I see this as a working solution (d_ for drive, s_ for server):

Source: C:\MyData\
Target: D:\backup\
Result: D:\backup\d_C\MyData\

Source: \\Computer\MyData\
Target: D:\backup\
Result: D:\backup\s_Computer\MyData\

[admin]

Actually "D:\Backup\<srcpath>\<dyyyy-mm-dd>" is brilliant. It's *exactly* how I organize my own backups since years, and the system is great! Now with the <srcpath> variable, the system will be perfect like a diamond!

Or not??

Actually, "exactly" was exaggerated. My daily backup system looks like this:

Code: Select all

...
F:\Backup\XY-code\20071209\class\Cremoved\
F:\Backup\XY-code\20071209\class\Cremoved\file.cls
F:\Backup\XY-code\20071209\class\old\...
                           mod\blah\...
                           mid\...
F:\Backup\XY-code\20071210\class\Cremoved\
F:\Backup\XY-code\20071211\class\Cremoved\
...

Fixed part\date part\source part (relative to some source path). The source part contains all folders and files of the source in their original structure and naming. So there's only one date part per backup. OTOH, with "D:\Backup\<srcpath>\<dyyyy-mm-dd>" the date part would be in many places (although only once per path). You would have "date-forking" (my term, sorry) in many places. The data would be very dismembered... I have to go now, no time to find the right words...

[jacky]

I quote my own solution to all above goals from http://www.xyplorer.com/xyfc/viewtopic. ... 6722#16722 :

Yes, I think this sounds all good. As I said on that topic, there will always be "confusion" possible, as one may already have a folder called "d_D" or "Drive D" or whatever could possibly be used, but in such a case XY would simply ask which one of the different possibilities to be used, it's as simple as that I woud imagine.

Oh, another post....

hmm... well, you could go for "F:\Backup\XY-code\<dyyyymmdd>\<srcpath>" then, which could look like this: (assuming the source files on in E:\XYplorer\current)

F:\Backup\XY-code\20071209\d_E\XYplorer\current\class\Cremoved\
F:\Backup\XY-code\20071209\d_E\XYplorer\current\class\Cremoved\file.cls

So that's already better (date-forking-wise) than before. To be perfect and avoid too long paths, it would required another option, to define the starting point of the source path to recreate.

Eg, with "F:\Backup\XY-code\<dyyyymmdd>\<srcpath:E:\XYplorer\current\>" you woud then get :
F:\Backup\XY-code\20071209\class\Cremoved\
F:\Backup\XY-code\20071209\class\Cremoved\file.cls

...which sounds perfect (it does, right?)

Of course, this causes the issue of when the path specified on the variable isn't actually found on the files' path, but then XY could simply create the full path, or ask the user what to do : go on (full path created) or cancel ?

[graham]

Admin wrote:

Before I add it, it would be a good time to decide how to format the source path. The goals:
(1) Have a good chance to implement an automatic "Restore" command.
(2) Make it visually clear, what part of the whole backup path is the source path.
(3) Distinguish drive letters from (UNC) servers with one-letter names (if that's allowed, I don't know) to avoid merging.
(4) Sort all servers together after all drives.

Just to add a few thoughts here.

The point of taking a backup is that you may need to use it either by restoring to the original location or to another so I think it important that each file has the scrpath 'embedded', however it is not, I think, necessary for that path to be highly visible. Thus to use a file(s) backed up one would either restore (use original srcpath) or enter a new destination. The srcpath could become visible as part of the restore/copy operation to aid the user as to the desired destination. The current method of reproducing the srcpath structure is not easy for rapid finding of individual files. If embedding the srcpath to a file was not possible then could this not be maintained in a history file by actual file name so a lookup process would be used to find the source. This approach would enable many backups to exist and quick searching of files backed up.

As we discussed before, it could also be advantageous to have the backup location either as previous or new. Thus some list management of previous backup history in the case of using catalog.

For user selected files not contained in a catalog then the user would supply the destination.

[admin]

Admin wrote:

Before I add it, it would be a good time to decide how to format the source path. The goals:
(1) Have a good chance to implement an automatic "Restore" command.
(2) Make it visually clear, what part of the whole backup path is the source path.
(3) Distinguish drive letters from (UNC) servers with one-letter names (if that's allowed, I don't know) to avoid merging.
(4) Sort all servers together after all drives.

Just to add a few thoughts here.

The point of taking a backup is that you may need to use it either by restoring to the original location or to another so I think it important that each file has the scrpath 'embedded', however it is not, I think, necessary for that path to be highly visible. Thus to use a file(s) backed up one would either restore (use original srcpath) or enter a new destination. The srcpath could become visible as part of the restore/copy operation to aid the user as to the desired destination. The current method of reproducing the srcpath structure is not easy for rapid finding of individual files. If embedding the srcpath to a file was not possible then could this not be maintained in a history file by actual file name so a lookup process would be used to find the source. This approach would enable many backups to exist and quick searching of files backed up.

As we discussed before, it could also be advantageous to have the backup location either as previous or new. Thus some list management of previous backup history in the case of using catalog.

For user selected files not contained in a catalog then the user would supply the destination.

I want to have it maximally transparent, foolproof, and robust. I don't want to depend on a corruptible system of log files, data bases, or worst of all: zips (you did not mention this, I know). A zipped backup is like dancing on the vulcano. A single defect byte/sector can make a gigabyte of data worthless. Long live redundancy! (and rock'n'roll!).

Second, XYplorer is not attempting to be a backup software. All I want is an quick and easy backup feature that serves my own personal needs. Daily source codes backups means: you have small amout of data (around 6 MB) and small number of files (around 300); they are in a folder structure not very deep (3 levels); and most important: they totally depend on each other, which means: for me it's not interesting to restore single files. The restore must always do all files: the complete thing. Sometimes I have to go back to an earlier version of the code to try something right in the backup location (yep, I do this because I have 3 identical backups in different locations, so I allow me to fool around with one of them): so the folder names within the code branch must be correct. If there were dates appended to each path (like generated from <dyyyy-mm-dd>) it would not work for me. So I need the date in front of my backed up branch: BackupRootPath\Date\BackuppedData.

[jacky]

Agreed, XY's goal isn't to be a fuly featured backup solution after all!

So I need the date in front of my backed up branch: BackupRootPath\Date\BackuppedData.

Well, I think the "srcpath with argument" (as I described above) would get you exactly what you need, wouldn't it?

[graham]

admin wrote:

Second, XYplorer is not attempting to be a backup software. All I want is an quick and easy backup feature that serves my own personal needs. Razz

OK, I give way to your rock solid and reliable solution and agree links are vulnerable but surely you want to satisfy the needs of others if at all possible wrt simple routine backup.

In my case I have a number of un-conected folders in a backup catalog, Shift +B and all done, backed up and secure. What I would like is the ability to select any single item (usually a folder in my case) from within the Backup Catalog and have the ability to copy or restore it. To do this directly using the catalog item would be very convenient and save a lot of time. Possible? Likely?

[admin]

admin wrote:

Second, XYplorer is not attempting to be a backup software. All I want is an quick and easy backup feature that serves my own personal needs. Razz

OK, I give way to your rock solid and reliable solution and agree links are vulnerable but surely you want to satisfy the needs of others if at all possible wrt simple routine backup.

In my case I have a number of un-conected folders in a backup catalog, Shift +B and all done, backed up and secure. What I would like is the ability to select any single item (usually a folder in my case) from within the Backup Catalog and have the ability to copy or restore it. To do this directly using the catalog item would be very convenient and save a lot of time. Possible? Likely?

Just to state the obvious: If you always backup to the same place, overwriting older backups, then the restore could be done by a complementary reverse-direction backup (with one difference of course: older is allowed to overwrite younger). But I can't handle this for you, since I don't know your target location.

A solution could come with scripting, e.g.:
backup source target /prompt
would prompt for "backup or restore?", similar to the Windows System Restore Wizard.

[admin]

Agreed, XY's goal isn't to be a fuly featured backup solution after all!

So I need the date in front of my backed up branch: BackupRootPath\Date\BackuppedData.

Well, I think the "srcpath with argument" (as I described above) would get you exactly what you need, wouldn't it?

Yes, I just did not have the time to answer your mail. It would be a possibility but it lacks flexibility. I would like to have one formula for all my backups (I have also other things to backup, not only XY sources). But I see that it looks impossible to find a general solution.

My own backup location should (and does) looks like this:

Root\Topic\Date\Data

Root: just one, say D:\Backup\
Topic: my various projects: XY-code, XY-webpage, Fudoku-Generator, etc.
Date: the date of the backup
Data: the backup data, usually files within a folder structure

That way I can easily retrieve older versions of my projects.

BTW, this is, of course, a typical backup context for stuff that evolves, that has versions. Another backup type would be my foto collection, which grows and grows. Here I need no date part, I just add and add and add...

[jacky]

Root: just one, say D:\Backup\
Topic: my various projects: XY-code, XY-webpage, Fudoku-Generator, etc.
Date: the date of the backup
Data: the backup data, usually files within a folder structure

I hear you, but I don't think there can be a real solution there. If all your projects happened to be under the same folder (ie. same parent for XY-code, XY-webpage, etc) then it would work with no problem.

If not, you then would need one backup destination per project, yes. But I'm not sure I can see a way out of this, especially since the source might be on different drives, at different level within their folder structure, etc

But, you could just define one UDC-Backup for each of your projects, and whenever you want to backup you hit that one KS that pops up the menu with the list of projects, and choose the one appropriate.

---

Well, the only way I see for you to have one single Backup destination that works all the time would be to have another list of "backup sources", either a list of path to "strip" from <srcpath> (or else the full path is recreated), or with a list of source and associated destination as well.
So either just:
E:\Dev\VB\XY-code\
E:\Dev\www\XY-webpage\
F:\Some\Other\Place\Fudoku-Generator\

Or something like:
E:\Dev\VB\XY-code\>D:\Backup\XY-code\<dyyyymmdd>\
E:\Dev\www\XY-webpage\>D:\Backup\XY-webpage\<dyyyymmdd>\
F:\Some\Other\Place\Fudoku-Generator\>G:\Backup\Fudoku\Generator\<dyyyymmdd>\

[admin]

Or something like:
E:\Dev\VB\XY-code\>D:\Backup\XY-code\<dyyyymmdd>\
E:\Dev\www\XY-webpage\>D:\Backup\XY-webpage\<dyyyymmdd>\
F:\Some\Other\Place\Fudoku-Generator\>G:\Backup\Fudoku\Generator\<dyyyymmdd>\

That's exactly how I do it now.



Okay, I should not do this, but I can't resist: I do my backups with a hidden non-documented backup interface in XYplorer!

Ever saw the key BackupBatches in the INI...? It makes the difference. But not all the difference. There also must be a file present that contains lines of data very similar to the format you have chosen in the quote above!

This module is there since years. But heck, I never had the time to clean it and document it! It's just too raw for you.