XYplorer Beta Club

Hi, i have a huge (100tb) of smb shared library (on a server-client basis) and i was wondering if its possible to group them based on genre, year and alphabet using xyplorer on client side.
Each movie already has movie.nfo, but it only has 2 relevant properties, imdb id and year.
So i was wondering if its somehow possible to query genre from tmdb/imdb using script somehow,
(But this could be done another way, such as with filebot script)

And more importantly, would it be possible to auto-tag each new items based on genre, alphabet and year?

Once that part is completed, on client side im thinking of grouping same tags together somehow, e.g. B alphabet, comedy genre, 2016 movies, etc..

That way client could somehow getting organized view based on those 3 categories..
I hope it makes sense.. sorry if its a little complicated

While getting the genre for each movie from imdb via xy scripting is possible, I wouldn't do that (no threading, html isn't meant to be parsed via regex)
An alternative way:
https://www.imdb.com/interfaces/
https://datasets.imdbws.com/title.basics.tsv.gz
Parse that one, much faster to get information from this source...

And more importantly, would it be possible to auto-tag each new items based on genre, alphabet and year?

Auto tag? No, but a script could easily do that...

highend wrote:While getting the genre for each movie from imdb via xy scripting, I wouldn't do that (no threading, html isn't meant to be parsed via regex)
An alternative way:
https://www.imdb.com/interfaces/
https://datasets.imdbws.com/title.basics.tsv.gz
Parse that one, much faster to get information from this source...

And more importantly, would it be possible to auto-tag each new items based on genre, alphabet and year?

Auto tag? No, but a script could easily do that...

Thanks for the tips, i will check it later.
I've been playing around with the latest version,
i will see if it could cover what i need..

Got a question,
Assuming the tagging is completed,
is there anyway to open an instance of xyplorer with specific tag as startup path?
e.g comedy genre tab would open all folders tagged as comedy

Sure, e.g. something like this would work:

Hi’
Indeed, html isn’t ideal to be parsed, plus we’re dependent on the updates IMDB makes to their pages.

That being said, it’s doable. Especially if you already have the IMDB ID for each file (which removes the trickiest part: finding the right movie page).
I have a script that does that (find the right page, gather data, find external poster, save poster named with parsed data like tags).
I’d thought of sharing it for a while, but I don’t have time to clean it up, so I purged it to post only some excerpts.

Select some files (whose name are correct movie names) and run the script. This version will process each selected file, find the IMDB page, retrieve the data and output it to an "_IMDB_.txt" file, one movie per line.
// Edit: This script is ideal as a "live" tool when you only want to process dozens of items, not several hundreds at once.

• For each movie, the data currently being extracted is:
 — Title
 — Year
 — Genre
 — Director (up to 3)
 — Actors/Actresses (first 3)

• Notes:
 — Edit the bottom of the code to finetune the data to your liking
 — Leave a sufficient "wait" between each request (especially if searching with google), else you’ll be blocked for a few minutes
 — The data parsing will work until IMDB decides to change their pages’ code (but usually less than once per year)
 — If you have the IMDB ID (which will be much faster and accurate), edit code accordingly

Good luck,
Flux

100 TB of data. Let's say, each movie has only 5 GB. That would make 20k movies. How long does it take to query the imdb site and gather the data from it...
Export the title.basics.tsv e.g. into an sqlite database and write a search tool to query the genre from one file which contains just all of your imdb ids and the whole thing is done in under 3-4 minutes...

highend wrote:100 TB of data. Let's say, each movie has only 5 GB. That would make 20k movies.

Wooops, you’re right! I failed to take into account the context… Not used to such numbers!
I’ve used my script extensively for several years but rarely for more than a hundred of items at once (even did it from a pile of movie tickets!); I edited my post accordingly.

Tested the whole thing a bit...

Tsv to sqlite.exe takes about 50 seconds to read the ~440MB title.basics.tsv file and export the tt-id + genre into a ~154MB sqlite
database

The query tool takes a file with 20000 tt-ids and spits out the belonging genres in about 25s.

Overall the whole thing wouldn't need much more than one and a half minute