[SOLVED] Global v.s. local tag storage, and some suggestions

[sj515064]

Currently XYplorer stores all tags in one file "tag.dat". Developer may have his own considerations, but this does cause some trouble to moving tags together with the tagged files around. I give two examples and suggest three options for solution. If it is too long to read, please read Solution #3 at the end directly

Suppose that my friend and I are both users of XYplorer, and cooperate on a project, files of which are stored in a folder.
He adds some comments to files within the folder, and sends it to me. I open the folder within XYplorer, but fail to see the comments in the column.
This is not surprising -- the path of the folder on different computers may be different.

If discussions above are imaginary, I do come across actual problems in my daily use. Part of my job is to do experiments in the lab, where I can't use my own laptop. Thanks to the portability of XYplorer, I'm able to run it from the USB stick, though. I use XYplorer to comment the measure documents (*.mdf) the time they are generated, in case that I would forget the details upon departure. Of course, the mdf files are saved on the stick, and the tag storage is set to "relative to the tag database". Back to my desktop I would like to copy the mdf files to my working folder, and the trouble comes -- I lose the comments on my desktop-version XYplorer. Again, the problem is due to different paths.

Similar discussions have been raised before:
viewtopic.php?f=5&t=19249&p=161019&hili ... ge#p161019

And workaround is provided here:
viewtopic.php?f=7&t=8587&p=76510#p76510
and here:
download/file.php?id=11716
However, these workarounds are not official.

Suggestions:
Solution #1 (the least possible at the moment, maybe in future releases):
As has been mentioned by other user, the radical solution is to add support for storing tags locally, within the parent folder of the tagged items. That way, the tags can easily go around with the files. However, even this sounds quite easy at first, lots of debugging and tests may be required before the functionality matures.

Solution #2 (a bit more practical now):
Provide import and export GUI tool for tags and integrate it in official release.

Solution #3 (the most practical, in my own opinion):
Actually, I think this problem can be solved simply (maybe not perfectly) by providing another option for tag database storage: relative to parent folder! That means, if I comment "File1" within "Folder1", an entry like "Folder1\File1 MyComment" will be added to "tag.dat". Whenever I want to import or export tag information, I simply copy and paste these entries from one "tag.dat" to another. The current used global tag storage scheme stays untouched.
This approach is with a potential problem though. That is, there may be two folders both named "Folder1", and both containing the file "File1" on the same disk! However, such circumstances are rare, and they are more likely that one is the backup of the other. In this case, displaying the same comments for files on both sides is proper, because the contents are the same.

Conclusion: please provide an option for storing the tags relative to the parent folder!

[bdeshi]

Solution #3 (the most practical, in my own opinion):
Actually, I think this problem can be solved simply (maybe not perfectly) by providing another option for tag database storage: relative to parent folder! That means, if I comment "File1" within "Folder1", an entry like "Folder1\File1 MyComment" will be added to "tag.dat".

Configuration > Information > Tags > Storage: Relative to tag.dat can help, at least when all tagged items are on the same drive as the tag db.

This approach is with a potential problem though. That is, there may be two folders both named "Folder1", and both containing the file "File1" on the same disk! However, such circumstances are rare, and they are more likely that one is the backup of the other.
(emphasis mine)

This is not a strong assumption.

[sj515064]

Configuration > Information > Tags > Storage: Relative to tag.dat can help, at least when all tagged items are on the same drive as the tag

That does help to some extent. However, the relativity of "tag.dat" and the tagged files should be reserved, otherwise tags, labels, comments would disappear.

This is not a strong assumption.

Yes you are right

I finally realized that the current scheme of tags database storage is dedicated for handling certain category of files, like photos, music, etc. In those cases, using a centralized database is reasonable, because this enables fast search with tags. However, it sacrifices portability.

Tags and labels need not be stored locally, because they are mainly for self ease of managing a group of private files, which usually need not be exchanged with others. However, comments, on the other hand, are meant for carrying information around between different machines sometimes, and they would be better stored locally. That way, they can travel with the folder.

[bdeshi]

I see how your ideas can be useful, but I can't see how paths in tag db can remain relative in the case where the tagged file and the database reside in separate drives.

Regarding solution #1, iirc, Don has spoken of his disinterest in littering the file system with disparate files, like folder-specific tag databases, and I agree with him.
As an alternative to this (and s#2), may I suggest you take a look at this script I'd written long ago, see if it something like can help: viewtopic.php?f=7&t=14288&p=126044&hilit=retag#p125724
I wrote this when I myself was in a similar situation, but being an old script, It needs to be updated. I can give this a facelift if you think this can solve your troubles for now. ... I should give this a facelift anyway.

A built in gui for exporting importing tags per location will be nice still, +1!

[sj515064]

Dear Sammay,

Brilliant ReTag.xys! Will try it later.

Wishes again for the developer to incorporate such feature (importing/exporting tags per folder) into release

[bdeshi]

Don't use that retag script. Improvements to Xy scripting since have made that obsolete, and the script's operation was quite unconventional as well.

Instead use this script.

Code: Select all

// import/export tag backups for a location as a *.xytagsbackup file.
// provides a process for sharing tags for a folder within the folder between XYplorer profiles.

"Import Tags For this Location|:tagsfind"
  $rootpath = <curpath>;
  $tagsfile = "";
  // find importable tagsfiles: try selected .xytagbackup, else search <dirname>.xytagbackup, going up folders. 
  if (<curext> == 'xytagbackup') {
    $tagsfile = <curitem>;
  } else {
    $rootpath .= '\'; // make parent traversal include current path to search for *.xytags
    $parent_count = gettoken($rootpath, 'count', '\');
    while ($parent_count > 0) && !($tagsfile) {
      $rootpath = gettoken($rootpath, -2, '\',, 1);
      $found_tagsfile = "$rootpath\" . gpc($rootpath, 'file') . '.xytagbackup';
      if (exists($found_tagsfile) == 1) {
        $tagsfile = $found_tagsfile;
        $answer = confirm("Tags file found under $rootpath:<crlf 2>$tagsfile<crlf 2>Merge?",,, 3);
        // Yes/No/Cancel; returns 1/0/2
        switch ($answer) {
          case 0: $tagsfile = ""; break;
          case 2: end 1,, 1; break;
        }
      }
      $parent_count--;
    }
  }
  if ($tagsfile) {
    tagload($tagsfile, 1);
    if confirm("Tags under $rootpath imported.<crlf 2>Delete " . gpc($tagsfile, 'file') . "?") {
      delete 0, 0, $tagsfile;
    }
  } else {
    msg "No importable tags backup found.<crlf 2>Tags backup for a location must be saved as <foldername>.xytagbackup.";
  }

"Export Tags Under this Location|:tagsadd"
  $rootpath = <curpath>;
  $tagsfile = "$rootpath\" . gpc($rootpath, 'file') . ".xytagbackup";
  if (exists($tagsfile) == 1) {
    end !(confirm("Overwrite existing tags backup?<crlf 2>$tagsfile")),, 1;
  }
  $tagsfile = tagexport($tagsfile, $rootpath, 3);
  if ($tagsfile) {
    msg "Tags under $rootpath have been exported to<crlf 2>$tagsfile<crlf 2>";
  }

I think it will work best as a custom button script.
The script shows two menu items for exporting and importing tag backups.

Exporting creates a xytagsbackup file in the root of a folder that contains tags applied to any item inside the folder. Then you can copy this folder along with that file to the tag recipient system, and import the backup using the import option of the script.
hope this helps.

[sj515064]

Works perfectly This script should be included in official release.

Thanks!

[admin]

Solution #3 (the most practical, in my own opinion):
Actually, I think this problem can be solved simply (maybe not perfectly) by providing another option for tag database storage: relative to parent folder! That means, if I comment "File1" within "Folder1", an entry like "Folder1\File1 MyComment" will be added to "tag.dat". Whenever I want to import or export tag information, I simply copy and paste these entries from one "tag.dat" to another. The current used global tag storage scheme stays untouched.
This approach is with a potential problem though. That is, there may be two folders both named "Folder1", and both containing the file "File1" on the same disk! However, such circumstances are rare, and they are more likely that one is the backup of the other. In this case, displaying the same comments for files on both sides is proper, because the contents are the same.

Conclusion: please provide an option for storing the tags relative to the parent folder!

I tried but failed to understand this solution. Can you give an example with full paths where storing just the parent folder would do any good? Maybe I did not have enough coffee but...

PS: Your timing is good! I'm just deep into this area...

[sj515064]

I tried but failed to understand this solution. Can you give an example with full paths where storing just the parent folder would do any good? Maybe I did not have enough coffee but...

Forget about Solution #3 If you are curious about the reason, read the following paragraph; otherwise skip to "Updated thoughts".

The initial idea is based on the assumption that two files/folders with the same name and the same parent folder are likely to be with the same contents, and therefore displaying the same tags is appropriate.
However, just as Sammay has pointed out, this assumption is not strong enough. For example, people who are not using version management tools like SVN may have duplicates of project folders with slightly differences between each of them. Say, I am working on a project, and suddenly come up with an idea and would like to try it out. I don't know whether the idea works and would like to keep the original project folder untouched in case of chaos. I will then make a copy of my original project folder to somewhere, and test my idea on it. This results in files and subfolders with both the same name and the same parent folder on my disk! If Solution #3 is used, I will not be able to tag the files and subfolders within these two project folders separately.
So conclusion: Solution #3 is not applicable.

Updated thoughts:
So the candidates are only Solution #1 (storing tags database locally) and Solution #2 (providing GUI tools, maybe commands in Menu | Favorites, for tag import/export per location, in official releases).
1. For Solution #1, in short, it enhances portability, but sacrifices fast search. Why? (If you are busy skip to Solution #2)
What I exactly mean:
Suppose there is a parent folder with a file named "File1" and a subfolder in it. The subfolder contains a file named "File2". When I tag "File1" or the subfolder, a tags database is generated and stored in the parent folder, automatically set to "hidden" so that contents in the folder won't be cluttered.
If I instead tag "File2", then it is stored in the subfolder. In other words, the tags database is always stored in the parent folder of the tagged items. Each time I open a folder, XY looks for a tags database, and if there is any, parse it and grab the tag information. In this way, the tags can travel with the folder.
However, this makes the fast search with tags impossible!

2. For Solution #2, it seems to be the most practical at the moment! The script command "tagexport" and "tagload" are already there. Sammay has written a script (see the code posted above in this topic) that works, simply provide it as a clickable command, maybe in Menu | Favorites, and it's done! Every time I want to share a folder with tagged items in it, I navigate into the folder within XY, execute the locally export command, and it will generate a file storing the tags (for the items contained by this folder only). I then send the folder to my friend who is also using XY, he executes the import command and the tags are merged into his "tag.dat" within the XY data folder.

So the final conclusion: please provide a GUI tool for exporting/importing tags per location/folder!

PS: Your timing is good! I'm just deep into this area...

Glad to hear that!

[admin]

OK. But how? GUI commands should to be simple and not ask too many questions. Those scripting commands, however, offer a couple of parameters.

So what about this? We could used a fixed name, e.g. "XYplorerTag.dat", and then do it like this (GUI commands in bold, corresponding scripts in fixed font):

Export Tags To Local Database = tagexport("XYplorerTag.dat", , 3, 1); (= store relative to tags database; remove from main DB)
Import Tags From Local Database = tagload("XYplorerTag.dat", 1); (= permanently merge into current DB; but the local DB survives (or should it be killed?))

Hm?

[bdeshi]

I'd like:
Export Tags To Local Database = tagexport("XYplorerTag.dat", , 3); (= store relative to tags database; don't remove from main DB)
Import Tags From Local Database = tagload("XYplorerTag.dat", 1); (= permanently merge into current DB)

After Import, the local DB survives (or should it be killed?): Prompt?

[admin]

Maybe the captions should be simpler:
Export Local Tags
Import Local Tags

[admin]

I'd like:
Export Tags To Local Database = tagexport("XYplorerTag.dat", , 3); (= store relative to tags database; don't remove from main DB)
Import Tags From Local Database = tagload("XYplorerTag.dat", 1); (= permanently merge into current DB)

After Import, the local DB survives (or should it be killed?): Prompt?

I like that.

[bdeshi]

Maybe the captions should be simpler:
Export Local Tags
Import Local Tags

Sure, but if these options are going to be in the Favorites menu, it should be made clear that this will export tags under current location only. Maybe "Export tags in this location"

[admin]

That's what I meant by "Local Tags".

What's actually done is: "Export Local Tags to Local Database" but that's clumsy...