Collection Manager [v2.5]

Discuss and share scripts and script files...
FluxTorpedoe
Posts: 723
Joined: 05 Oct 2011 13:15

Collection Manager [v2.5]

Post by FluxTorpedoe » 20 Sep 2013 16:11

-------------------------------------
Download latest version here: Collection Manager v2.5
-------------------------------------


Hi'
This thread is dedicated to talks about the "Collection Manager" script.
One note: it's muuuch easier to use it than to read or write about it! 8)

There will be a proper thread as soon as I have the time to write a clean presentation. In the meantime, I'll post pre-releases of the script here, so those interested can talk about it to help polish it before it reaches "mass release"!

I'll start with a few screenshots. The first one is taken from the concise help included in the script, so I don't have to explain the basics... ;)
CM_Help.png
CM_Help.png (43.54 KiB) Viewed 7522 times
Here's my current Menu (when I right-click on CM button) with five collections. I've changed the icon of the first four collections.
CM_Menu.png
CM_Menu.png (11.46 KiB) Viewed 7525 times
Here's the "Manage Collections" submenu. You can see that "Music" is a [Folders] collection (because I don't want each file to be searched), and that "Recipes" are just [Data] (because they're word, pdf or html for which I haven't made covers). You can also see that I have two "monitored" collections (stressed by the • as in the main menu).
CM_Menu_Sub.png
CM_Menu_Sub.png (13.79 KiB) Viewed 7525 times
Now here's the Finder I got when clicking on "Find eBooks" in the main menu. Filter lists are auto-populated when a collection is first created (or updated) based on filenames and folder names (for rating only). Data is stored in an INI file. Note that the first element of each "double - filter" has been (automatically) indented so it's easier to identify and can also be searched.
CM_EbooksFinder.png
CM_EbooksFinder.png (18.07 KiB) Viewed 7525 times
And here's the Finder I got when clicking on "Find Movies". Since it's the finder of the first collection, it's also what is run when left-clicking the Collection Manager button. (Filters —and full movie data— had been auto-assigned to covers from IMDB data.)
CM_MoviesFinder.png
CM_MoviesFinder.png (17.26 KiB) Viewed 7525 times
In the last (semi-fictitious) example, let's say I wanted to watch a favorite drama movie. So I would have checked "Old", "Rating: A", and "Drama", then pressed OK, getting a new maximized tab with big thumbnails showing the results. At this point, since this collection is "monitored" (confirmed by the • after its name in the tab results), I would only have to double-click on the first poster to launch the movie Aguirre in my default media player!
CM_MoviesFound.jpg
CM_MoviesFound.jpg (155.92 KiB) Viewed 7525 times
~ ~ ~ ~ ~ ~ ~ ~ ~

Well, enough talk for now!
I'll try and post the script tomorrow...

Have a nice day, 8)
Flux
Last edited by FluxTorpedoe on 15 Mar 2017 15:00, edited 9 times in total.

binocular222
Posts: 1410
Joined: 04 Nov 2008 05:35
Location: Hanoi, Vietnam

Re: Collection Manager [Lounge]

Post by binocular222 » 20 Sep 2013 20:20

Your system is built around a "tag" concept ! This is what I'm long dreaming of. My "tag" concept is:
- Files are physically stored chaosly in HDD and Primary function of a File Manager is to help Human search for files
- There're only 2 ways to search: Search-by-Content and "Tag" (I only concern the latter and it's defined below)
- Each file is assigned with a piece of info which describe the content of the file (people call it metadata - info about data, I just call it "tag")
- Tag can be stored:
(i) in an external file (XYplorer's tag.dat; NTFS's MTF, or in an .ini file in your case)
(ii) using the file's name (you chose this)
(iii) inside the file (I currently add \_:Tag1|Tag2|Tag3:_/ at the end of file)
- Folders is simply case (i) above, thus Tag has a broader meaning than Folder
- Folders structure is good way to demonstrate relationship between tags (parent-child). However, this structure face a severe limitation: One file cannot appear in different folders, which means, we cannot freely add tag to file. Example: How do you save Friedman's article about economic crisis if you have these folders:
D:\Economic\case study\
D:\Friedman\
D:\Havard\school ebook\


- Tag concept tries to achieve 4 things:
(1) Add tag to files freely
(2) Define relationship between tags, not just Parent-Child but also peer-to-peer
(3) Utilize a file manager to "browse by tag" instead of browse by folders
(4) Above all, tags must be flexible, consistent, allow mass-update, adhere to file

Hope that I can borrow some solutions from you.
Last edited by binocular222 on 21 Sep 2013 01:47, edited 1 time in total.
I'm a casual coder using AHK language. All of my xys scripts:
http://www.xyplorer.com/xyfc/viewtopic. ... 243#p82488

admin
Site Admin
Posts: 47075
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: Collection Manager [Lounge]

Post by admin » 20 Sep 2013 20:32

binocular222 wrote:(3) Utilize a file manager to "browse by tag" instead of browse by folders
Man, this is on my table since long! It's part of my concept of "tag-based file management". A tree where the nodes are labels, or tags, or boolean combinations of tags. But I never got it 100% right in my head. I keep returning to it in my meditations. One day...

Filehero
Posts: 2283
Joined: 27 Feb 2012 18:50
Location: Windows 10 Pro x64

Re: Collection Manager [Lounge]

Post by Filehero » 21 Sep 2013 06:48

Hi Flux,

no matter what the actual outcome will be: all my respect for starting this can-of-worms endeavour using SC! :appl:

I will surely give it a try some day I find some time to setup a basic set of tagged files/folders in XY (haven't done so yet because for everything the basic folder-based file organisation approach isn't really suited I use specialized tools like Lightroom or Windows Media Center/iTunes etc.).

admin wrote:
binocular222 wrote:(3) Utilize a file manager to "browse by tag" instead of browse by folders
Man, this is on my table since long! It's part of my concept of "tag-based file management". A tree where the nodes are labels, or tags, or boolean combinations of tags. But I never got it 100% right in my head. I keep returning to it in my meditations. One day...
I don't know all media asset management systems, but I still think Adobe's Lightroom is one of the best in this realm (at least amongst the desktop systems).

You get
- attributes (flags, ratings, colors)
- standard metadata (IPTC, camera settings)
- hierarchical user metadata (keywords) with a keyword tree
- static collections (where you can set an active one an image will be added just by hitting a shortcut key)
- dynamic collections (all shoots done in 2013)

Though they're symantically and under the hood technically different all those metadata are nicely integrated and presented on the gui.


Cheers,
Filehero

FluxTorpedoe
Posts: 723
Joined: 05 Oct 2011 13:15

Re: Collection Manager [Lounge]

Post by FluxTorpedoe » 21 Sep 2013 12:28

admin wrote:
binocular222 wrote:(3) Utilize a file manager to "browse by tag" instead of browse by folders
Man, this is on my table since long! It's part of my concept of "tag-based file management". A tree where the nodes are labels, or tags, or boolean combinations of tags. But I never got it 100% right in my head. I keep returning to it in my meditations. One day...
Well, that's the kind of navigation I wanted with this script...(while waiting for an integrated "tag-based file management" ;) ).

Folders are a good storage component but they're useless (or even a nuisance) when it comes to managing Digital Assets. BTW, I definitely wanted to mention Lightroom, but Filehero beat me to it! ;)


~ ~ ~ ~ ~ ~ ~ ~ ~ A few considerations about Digital Assets Management / XYplorer / Collection Manager

First, let's consider objectively that for dedicated work (images, music), nothing beats a dedicated soft. So here we're talking in a broader way — or when you want a quick and efficient access to your data without running those softs (which are usually quite slow to start-up).

Ideally, we'd like an easy way to find items based on what I'd called filters: (user)rating, (user)labels, (user)tags, (user)classification, (file)metadata,... e.g.
- I want to find a comedy movie that I haven't seen (don't care on which drive it is)
- I want to find that good ebook about Design that I'd already started to read (don't remember where I put it).


BUT usually we'd like to refine the search —or at least identify differences between found items. It's in this results display that things get complicated...
- Among the comedies I now see, how do I know which one is a Family Comedy and not a Drama?
- What better way to identify this book among the other Design books than by its cover?


If Lightroom/Aperture/Picasa..., "work" so well, it's because we see the results of our searches. And WindowsMediaCenter/iTunes/MediaMonkey... are great 1. because we're able to see the metadata (e.g. rating, year), and 2. especially when we also see album covers.

So basically, my main approach was that we must see what we're looking for! (though that shouldn't be an obligation — see conclusion).
- See the items (through covers).
- See the "filters" (especially when covers aren't available).

Right now in XY, only filenames and labels are visible. There's no way to see tags written inside files, and for XY tags, you'd have to be in "Details" or "Details with (tiny) thumbnails".
For that reason, I chose to get the "genre" filters from filenames and not from tags (at least until "Large Tiles" becomes customizable ;) ), the "status" (seeing/seen/priority/etc.) from labels, and with an exception for the ratings I reluctantly take from folders (IMHO, Ratings should be XYplorer's next best thing).

:arrow: This allows you to effectively and efficiently perform the previous searches in Collection Manager in less than 3 seconds! :D

To conclude for the moment:
:!: Covers and filters are not mandatory. One can define a "Movies" or "Music" collection that just points to folders on different drives, with no covers and no tags, no renaming of filenames (note that with covers, item filenames can be left as-is with only cover filenames named comprehensively). It would still provide a very fast way to search for a movie or music by name — much faster than starting a dedicated soft, or opening the several folders to perform several searches.
:!: I tried to make Collection Manager "modular", that is, structured in different functions. e.g. For each collection, the indexed list of filters is built by a dedicated function which currently extracts most filters from filenames, but which can easily be changed to extract them from tags, metadata, (or even from inside a file), etc.
The search "algorithm" already uses a combination of different types of filters (from folder names for rating, from labels, and from filename filters), so it can easily be changed to match any new type of filters.

~ ~ ~ ~ ~ ~ ~ ~ ~

Geez, now I've talked too much again, and don't have time to post the script... Tomorrow then! :cup:
8)

binocular222
Posts: 1410
Joined: 04 Nov 2008 05:35
Location: Hanoi, Vietnam

Re: Collection Manager [Lounge]

Post by binocular222 » 22 Sep 2013 06:35

While waiting for the script, I'm thinking of a solution:
- Both Folder name & Filename are considered "tag".
i.e: "D:\Movie\Titanic [Leonardo DiCaprio 1997 Drama].avi" would represent 6 tags: Movie,Titanic,Leonardo,DiCaprio,1997,Drama

- Use Everything to search (with match folder path enabled)
i.e in the search field of Everything: D:\ movie drama leonardo
(It's easy to write an xys script feeding current folder path & filename to Everything)

- To search by thumbnail/cover art:
After searching by Everthing mentioned above, Everything can export the result to an .efu file (simple csv formatted with file name, date modified, size...)
Then feed this file list to XYplorer. This requires Virtual Folder, which Enternal and Flux mentioned here http://www.xyplorer.com/xyfc/viewtopic. ... 484#p84599
Of course a native support for Virtual Folder from XYplorer would be better (or if XYplorer support MFT search would be an uber killer feature: no more need for Everything)


Possible drawback:
- Filename can be overly long, which make Name column in XYplorer ugly.
Maybe an option to hide a portion of filename would help, i.e: Hide everything inside []

- Tags between foldername and filename can be duplicated. i.e: You created a new folder "Drama" and move the above Titanic file inside, then "Drama" appear in both folder and file name

- Not utilize metadata inside media files (music, movie, picture, office document), thus a dedicated software is better for these file type

- Not ulitize XY Tag/Label
Only solved if XYplorer support MFT search, thus combine Everything & XYplorer search into 1
I'm a casual coder using AHK language. All of my xys scripts:
http://www.xyplorer.com/xyfc/viewtopic. ... 243#p82488

FluxTorpedoe
Posts: 723
Joined: 05 Oct 2011 13:15

Re: Collection Manager [Lounge]

Post by FluxTorpedoe » 23 Sep 2013 16:15

Damn, bad timing, something came up... :(
I won't be able to be active online for the next few days or more (or only sparsely). Anyway, I'll read everything as soon as I can and will catch up with you all.

@binocular222
There's interesting things to discuss in what you said but I don't have time right now, will come back to it later...

@all
But since I already promised to release the script, here it is.
No time to add much explanation though, but there's always the integrated help...

Hope some of you like it!
Have a nice day, 8)
Flux

--------------------------
################
### Installation ###
################


To use this script, first, unzip the following file,
Data.zip
(27.67 KiB) Downloaded 185 times
and copy its content: the folder "Icons" (19 .ico from FatCow) and the folder "Scripts" (1 .xys), in your "Xyplorer\Data" folder. Then:

1. Use the following snippet (with Don's litteral instructions):
-a) Paste ::snippet; into the address bar and press ENTER. A text box will open to enter snippet.
-b) And here comes the copy+paste ready snippet. It will create a new button at the right end of your toolbar.

Code: Select all

Snip: CTB 1
  XYplorer 12.90.0, 09.21.2013 12:00:00
Action
  NewUserButton
Name
  Collection Manager
Icon
  <xyicons>\CM_Collection.ico
ScriptL
  load "CollectionManager.xys", "_Finder";
ScriptR
  load "CollectionManager.xys";

FireClick
  0
[/size]
Note that:
- load "CollectionManager.xys", "_Finder"; will launch the Finder of the first collection
- load "CollectionManager.xys"; will open Collection Manager main (dynamic) menu — so if you want to have it directly visible (i.e. included) as RightClick, you must have the tweak CTBNoRClickDefaultCommands=1)


2. For Cover Launcher (see Help & screenshots), add this entry somewhere on top of your PFA (Menu "Tools" | "Customize File Associations...").

Code: Select all

*.jpg;*.png>load "CollectionManager", "_FolderMonitoring";
It will only "catch" the jpg and png that belong to folders you've set as "monitored" in Collection Manager.

That's all! Now you can enjoy it right away.
Try it first with your Music or Movies folders for example...

PS: "Movie Posters Grabber" is a different script not provided here right now.

binocular222
Posts: 1410
Joined: 04 Nov 2008 05:35
Location: Hanoi, Vietnam

Re: Collection Manager [Lounge]

Post by binocular222 » 23 Sep 2013 20:16

Your script is quite easy to use :appl:
a minor wish would be an option to include subfolder.

P.S, now, I would continue to hijack your thread with my fantasy :mrgreen: , don't want to start a new thread to keep the flow of thought.

This is a perfect scenario:
There's a separate file listing "pairs of peers" relationship between tags.
Tags are folder name and the portion of filename between []. Example:
D:\Asia\flower\rose.jpg
D:\Asia\animal\dog.jpg
D:\Asia\animal\birds\parrot.jpg
D:\Europe\animal\reptiles\crocodile [amphibious].jpg
would have 6 pairs of tags: Asia|flower, Asia|animal; animal|birds; Europe|animal; animal|reptiles; reptiles|amphibious
a pair means: 2 members are related but they're equal, no parent-child relationship.
Step 1: When user choose Asia, a popup menu (or addressbar) appear with 2 options in gray: flower, animal (left click to add to boolean, right click to subtract).
Step 2: Continue to left click on animal:
.... Address bar would show Asia,animal (with green color - indicated that 2 tags are boolen-enabled), bird in gray
.... List would show dog.jpg
Step 3: Continue to right click to disable Asia:
.... Address bar would show animal (green); Asia (red), Europe, birds, reptiles (gray) <= notice we can navigate both back and forth
.... List would show dog.jpg

The advantage is:
- Super easy to filter "animal" in Asia only, in Europe only or in both
- Navigation is very similar to current folder structure (in fact the address bar's UI would look similar to Windows Explorer now with breadcrum). Notice that in step 1, Asia is enabled but "birds" does not appear although being a child-child folder of Asia => very much similar to how all file manager handle folders now.
- Still respect current folder structure
- Can utilize metadata. This is HUGE advantage. Example: if dog.jpg above has author=Mai, we simply consider Mai as a tag and add a pair of "animal|Mai". now, select Asia > animal > Mai would list out all Mai's animals.

How realistic is it?
- on first run, we need to scan through all computer and list out all folders structure & File properties, time consuming but very easy to coding
- Next, we need real-time monitor of folder/file creation/rename. I don't know about C+ or VB but AHK can monitor directory quite easy.
- The .txt file can be large, is it a problem? I'm not a true programmer, so cannot say

P.S, just found a true tag-based file management, still far from practical but still note worthy:
http://silkwoodsoftware.com/screenshots.html
I'm a casual coder using AHK language. All of my xys scripts:
http://www.xyplorer.com/xyfc/viewtopic. ... 243#p82488

stanmarsh
Posts: 85
Joined: 10 Mar 2009 07:43

Re: Collection Manager [Lounge]

Post by stanmarsh » 31 Oct 2013 02:46

hello all,

this great thread will make xyplorer much more powerful!

1. i like to share some good DAM and file management articles i found.

http://www.asianefficiency.com/organiza ... documents/

http://www.extensis.com/downloads/docum ... ces-guide/

http://www.realstorygroup.com/Feature/124-DAM-vs.-DM

2. few apps that we can get some inspiration

the first app is taggedfrog,
it has a cool tag cloud

http://lunarfrog.com/taggedfrog/

another is together, its a mac os app,
if you look at the left side of the app, i think this can be created easily in xyplorer's catalog

http://reinventedsoftware.com/together/

3. have a helper app using external programs that communicates with xyplorer using xyplorer messenger. using this we can combine different apps into one. this will supplement xyplorer file management capabilities. this has now been made a reality by FolderMenuEx by Enternal!

link to FolderMenuEx

http://www.xyplorer.com/xyfc/viewtopic. ... 14f#p95737

e.g. book management

you could use calibre for book management, but instead of using the full calibre app, you can get specific parts of calibre like fetching metadata, covers and converting books ( i think you can use the calibre cmds for these) that you need, and from FolderMenuEx, you can use these functions in xyplorer.

4. and lastly some tags we can use,

http://en.wikipedia.org/wiki/List_of_en ... _knowledge
http://en.wikipedia.org/wiki/Outline_of ... lds_differ
http://en.wikipedia.org/wiki/File:The_S ... iverse.png


merci!

FluxTorpedoe
Posts: 723
Joined: 05 Oct 2011 13:15

Re: Collection Manager [Lounge]

Post by FluxTorpedoe » 16 Jan 2014 01:04

Hi' back everyone! :biggrin:

@binocular222
Thanks for your input... even if I'm more than late. :wink:
Well, looks like you've already found yourself a nice tag-based manager, but to answer your main points:
- Unfortunately, I found that using folder names as tags is not flexible enough, and that always raises problems eventually.
- Using a tree structure or a menu structure would be a dream indeed! :) But for the former, I don't know a way of scripting that in XY, and for the latter, we'd have to click several time to view each submenu (we don't have the "dropdown" effect when pointing the mouse over scripted menu entries —yet :mrgreen: ).

Anyway, I haven't had the opportunity to use my script much since October, and I have to get up to speed with the latest tricks of XY!
And maybe you've found new ideas or tools in the meantime...

@stanmarsh
Some softs look interesting, I'll definitely have a look at them!

FluxTorpedoe
Posts: 723
Joined: 05 Oct 2011 13:15

Re: Collection Manager [Lounge]

Post by FluxTorpedoe » 24 Jan 2014 00:22

Hi'

First, my apologies for the previously rushed "preview" release, it may not have worked under certain configs.
This one is much stronger — let's call it a beta! ;)

Also in this release, now, for your "Covers" collections (collections based on pictures), you can also enable folder monitoring to associate pictures with folders, i.e. when double-clicking on a pic, it will open its parent folder in the other pane.

For newcomers, please keep an open mind: this script is much easier to use than to read about or setup — and if you have a need for it, it's incredibly powerful! :)

In any case, if you have any kind of remark or question, don't hesitate !

Have a nice day, 8)
Flux

- - - - - - - - -
Collection Manager v0.b7 - Changelog
* Fixed several issues with different XY settings
* Optimized folder monitoring detection (which launches items on image double-click)
+ Added new folder monitoring function: open parent folder on image double-click
* Changed folder monitoring to launch items through Windows or PowerLauncher script
+ Added tweak for thumbnail size (see header of script)
Data.zip
(28.45 KiB) Downloaded 122 times
Install instructions are here.

:!: Note to existing users: CollectionManager.ini had to be modified (this shouldn't happen again).
For all SearchPath entries, paths are now separated by "|" instead of ";". So you have to update CollectionManager.ini, either by replacing ";" with "|", or by deleting it and recreating the collections.

autocart
Posts: 758
Joined: 26 Sep 2013 15:22

Re: Collection Manager [v0.b7]

Post by autocart » 28 Jan 2014 10:39

This seems to me to be kind of a safed search management with its own implementation and UI.
Very nice!!! So far the new version works well for me.
Thank you very much!!!

[EDIT:] oops, spoken too soon as usual.
When I add a second folder something is not working right anymore.
For some reason the collection manager comes up with this name search string:
M:\path1\;M:\path2\?>("\s\[.*\.bmp" | "\s\[.*\.doc.*" | "\s\[.*\.dwg" | "\s\[.*\.gif" | "\s\[.*\.jp.*g" | "\s\[.*\.odt" | "\s\[.*\.pdf" | "\s\[.*\.png" | "\s\[.*\.xcf" | "\s\[.*\.xls.*") /fr
I dunno what the "\s\[" is actually doing there.
customInlineRenameKeys ("OldStyle", "Mouse++" and "Ultimate"), clipboardHelper_and_XYEscToList

FluxTorpedoe
Posts: 723
Joined: 05 Oct 2011 13:15

Re: Collection Manager [v0.b7]

Post by FluxTorpedoe » 03 Feb 2014 00:24

Hi autocart,

The presence of "\s\[" in the search string means that Collection Manager will only search "filtered" files. This happens when it has detected more than 5 different "filters" for a specific collection, a filter being the presence of [] in the filename, preceded by space (hence the \s in regex). I had implemented this to filter only "useful" files (e.g. "Cover [Punk, Rock].jpg" but not "Back.jpg").
So:
— CM will only search files "/fr" with your defined extensions "*\.ext" : bmp, doc, dwg, etc.
— CM will only search files with these extensions AND containing filters "\s\[", e.g.: [Romance, Comedy], [Science]

:arrow: The script runs: Path?>("\s\[.*\.ext") /fr

It may seem convoluted, but I hope this clarifies things. :)

- - - - - - - - -

Now to the problem itself: adding another folder shouldn't have an impact.
What do you mean by "not working right anymore"? Nothing is found, only some files,...?

If the problem comes from the forced "filtering", try raising the "5" value in script (e.g. to 100) :

Code: Select all

$FilterSearch = (gettoken($FindFilters, "count", "|") >= 5)? '"\s\[.*\.' : '"\.';
Someday I'll have to implement an easy way to set this.

Well, keep me updated about your problem.

Have a nice day, 8)
Flux

stanmarsh
Posts: 85
Joined: 10 Mar 2009 07:43

Re: Collection Manager [v0.b7]

Post by stanmarsh » 04 Feb 2014 04:53

hello FluxTorpedoe very glad to see your back and with a new version! :)

i will test your new script version!

thanks!

autocart
Posts: 758
Joined: 26 Sep 2013 15:22

Re: Collection Manager [v0.b7]

Post by autocart » 04 Feb 2014 08:26

FluxTorpedoe wrote: a filter being the presence of [] in the filename, preceded by space
This is your own definition of a filter, right? Or did I overlook something in the XY-help file?
[EDIT:] Ok, the info is in the Collection Manager-help file. It would be nice, if there was an option to turn the use of these filters in CM on/off. I totally do not need these kind of filters at all. Therefore also the dialog that pops up whenever I open a collection is kinda annoying.
FluxTorpedoe wrote:Now to the problem itself: adding another folder shouldn't have an impact.
What do you mean by "not working right anymore"? Nothing is found, only some files,...?
Address bar with 1 folder:
"M:\path1\?>("\.bmp" | "\.doc.*" | "\.dwg" | "\.gif" | "\.jp.*g" | "\.odt" | "\.pdf" | "\.png" | "\.xcf" | "\.xls.*") /fr"

Address bar with 2 folders:
M:\path1\;M:\path2\?>("\s\[.*\.bmp" | "\s\[.*\.doc.*" | "\s\[.*\.dwg" | "\s\[.*\.gif" | "\s\[.*\.jp.*g" | "\s\[.*\.odt" | "\s\[.*\.pdf" | "\s\[.*\.png" | "\s\[.*\.xcf" | "\s\[.*\.xls.*") /fr
customInlineRenameKeys ("OldStyle", "Mouse++" and "Ultimate"), clipboardHelper_and_XYEscToList

Post Reply