Alternative, non-locking Thumbnail generation Method

Features wanted...
ThomasHelzle
Posts: 39
Joined: 21 Dec 2016 15:08
Contact:

Alternative, non-locking Thumbnail generation Method

Post by ThomasHelzle »

Hi,

I'm currently working on 3D renderings using the high-rez Arroway Wood Textures and in the process stumbling over something I wanted to ask for a very long time:
ATM - correct me if I'm wrong - there are basically two methods to create thumbnails for a folder, either they are generated on the fly while one scrolls through the folder, generating only thumbs for the visible ones, or we can set in preferences that XY should generate all thumbnails at once.

Now I'm sure these both work well with web sized images or very fast drives/machines, but I regularly work with large textures, like 14500x1900px or 5500x8500px and the folder for the abovementioned wood textures has 675 of them, including bump maps, roughness maps and different styles and sizes, roughly 14GB in all - on a fast but "normal" rotary HD.
I use 320x320px thumbnails with a special shortcut so I can actually see some detail.

The problem is: The "on the fly rendering" of the thumbnails is very slow to browse these kind of large textures, but it allows me to relatively quickly right-click blowup check out the textures 1:1, since the process stops when all images on the page are "thumbed".
The "create all thumbnails at once" on the other hand makes more sense in theory, since as soon as all the thumbs are rendered, I can very quickly browse for textures that look like they could work.
But this thumbnail-creating locks up XYplorer until it is finished! I can not check out individual thumbs while it is running.
Generating all thumbs in this folder takes something like 10 minutes or more, so not something you want to wait for.
(And yes, I do save the thumbnails).

So my request would be the following:
I would love to have a kind of hybrid. All thumbnails should be created when entering a folder, but somehow in a non-blocking background thread, that allows me to already work in the folder and for instance look at textures with right-mouse-blowup etc. So that background thread should also run with low priority, so it doesn't block other things.

Right now I work around this by letting thumbs be generated when I don't work on the machine, but if you get a whole load of new textures or photos etc. this doesn't work if you have to dive in right away.

As a little extra, the thumbnail generation thread could also focus on the currently visible files first, before continuing with the ones outside the view window, spreading maybe up and down equally in both directions, centered around the current view, so that no matter in which direction you scroll, you should always have thumbs waiting already... :-)

I hope this makes sense, otherwise I'm happy to clarify!
It really is my number one wish for XY :-)

Thank you for this fantastic tool!!!!!

Cheers,

Tom
"Out beyond the ideas of wrongdoing and rightdoing, there is a field. I’ll meet you there." - Rumi
ScreenDream|Thomas Helzle 8) Mastodon

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Alternative, non-locking Thumbnail generation Method

Post by admin »

FYI, background threading in the strict sense is not possible for XYplorer. Only background processing. For the user it's the same, for me it's harder. :) I might add it one day.

But even now all is not lost. Why don't you use a parallel second XYplorer instance just for creating thumbnails while you continue working with your main instance?! There should be a possibility via scripts to open this instance directly from the main instance for the current or any folder and to create these thumbnails and write them to the same cache that the main instance will use later. Not 100% sure (I'm not a scripter), but if it's not possible yet, I will make it possible.

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Alternative, non-locking Thumbnail generation Method

Post by highend »

There should be a possibility via scripts to open this instance directly from the main instance for the current or any folder and to create these thumbnails and write them to the same cache that the main instance will use later
viewtopic.php?t=12257
but if it's not possible yet, I will make it possible
You could easy the whole process by adding a new script command that creates thumbnails for selectable size(s) and a specific path. As a function with a returncode if possible :party:
One of my scripts helped you out? Please donate via Paypal

ThomasHelzle
Posts: 39
Joined: 21 Dec 2016 15:08
Contact:

Re: Alternative, non-locking Thumbnail generation Method

Post by ThomasHelzle »

Well, generating the thumbs in a second instance doesn't really solve the workflow issue that is at the center of this.

I often get new textures or photos or other libraries with many large files and I need to directly check them out.
Or I browse an old backup drive with dozens of folders, looking for some specific file.
And the direct, immediate access is the issue here.
If I have time and nothing else to do, sure I can run the thumb generation manually, but if I do this in the main instance or a secondary one, doesn't really matter much in that regard - I simply can't instantly and fluidly evaluate large collections of images with XYplorer.

I can do it in other software like XNview, it allows exactly the kind of background generation and previewing I hope to one day see in XY, but this means I have to switch tools and have even more applications running instead of doing almost everything in XY. And it doesn't have the right-mouse-blowup which helps a lot to quickly see the nitty gritty of huge textures... ;-)

Another case where I prefer using XY is when looking for older 3D project files. I don't remember the name, but I go through my folders looking for renderings and from there can directly see the accompanying 3D scene file - which image viewers don't show normally.
And again, the files are huge EXR or TIF files that take their jolly time to generate thumbs for.

So: If you should come up with a non-blocking form of thumbnail generation that does allow me to still scroll fluidly through a folder (right now the thumb-generation blocks any attempt at scrolling around and makes it totally hit and miss) and have rollover-previews as well as right-mouse-blowup working at the same time, I'd be ever so grateful!!! :-) :tup: :party:

Cheers,

Tom

Edit: Thanks a lot highend for the link, but like I wrote above, my issue is a bit different, it's more about immediate, non-stuttering and fluid access to folders, seeing the images and files as fast as possible while the thumbs are generated and still being able to move around.
Last edited by ThomasHelzle on 10 Nov 2021 13:19, edited 1 time in total.
"Out beyond the ideas of wrongdoing and rightdoing, there is a field. I’ll meet you there." - Rumi
ScreenDream|Thomas Helzle 8) Mastodon

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Alternative, non-locking Thumbnail generation Method

Post by admin »

one day... :)

ThomasHelzle
Posts: 39
Joined: 21 Dec 2016 15:08
Contact:

Re: Alternative, non-locking Thumbnail generation Method

Post by ThomasHelzle »

Ok, will try to survive with XNview until then... ;-)
"Out beyond the ideas of wrongdoing and rightdoing, there is a field. I’ll meet you there." - Rumi
ScreenDream|Thomas Helzle 8) Mastodon

klownboy
Posts: 4109
Joined: 28 Feb 2012 19:27

Re: Alternative, non-locking Thumbnail generation Method

Post by klownboy »

highend wrote: 10 Nov 2021 13:08 You could easy the whole process by adding a new script command that creates thumbnails for selectable size(s) and a specific path. As a function with a returncode if possible
Yes, the major problem with XY thumbnail generation is having to be in the current folder. I had another revision to the Thumbnail Maintenance script v1.20, that I've been sitting on, that allows the user to select a folder, select the sizes of thumbnails, and build them in the background, but it still wouldn't help in ThomasHelzle situation. Essentially he wants to be able to view thumbnails properly and have thumbnail generation continue.

When your Folder View or tab setting is set such that you are in a thumbnail view obviously XY is going to start generating thumb (unless you have "Show cached thumbnails only checked"). We can build thumbs in a another "background" instance of XY, but you really can't be viewing thumbnails in the same folder as the background operations is generating the thumbs since your foreground instance will also want to do the same when in a thumbnail view. You can be in a list view in the foreground and properly generate thumbnails in the background instance.

That referenced script will build thumbnails in the background while you 're doing something else in the foreground instance or getting some coffee. XY internally controlling the generation of thumbnails in a specified path and thumb size and accomplish that in a temporary 2nd instance would certainly help.
Windows 11, 22H2 Build 22621.1555 at 100% 2560x1440

autocart
Posts: 1246
Joined: 26 Sep 2013 15:22

Re: Alternative, non-locking Thumbnail generation Method

Post by autocart »

BTW, for whom it concerns,
unless you have "Show cached thumbnails only checked"
Well, then just have it checked.
And with copydata you are able to run a script in another XY instance. So, then you should be able to refresh thumbnail view in the 1st XY instance from a script running in the 2nd XY instance.
So, the script updating the thumbs in the 2nd instance could call a refresh command in the first instance in certain intervals.
Then it would come close to what the OP wants, I think.

Or did I overlook something?

ThomasHelzle
Posts: 39
Joined: 21 Dec 2016 15:08
Contact:

Re: Alternative, non-locking Thumbnail generation Method

Post by ThomasHelzle »

Thanks guys, this is quite interesting.
But I fear it lacks the fluidity of what I can achieve for instance with XNview?
It sounds rather complicated?

And with only cached thumbnails visible, I would have to generate and save ALL thumbnails (I have terrabytes of images here...)?

And how would that refresh work, would it be "invisible" or would it re-sort/relist everything constantly?

Somehow it sounds a bit crazy...? :-)

I still think I have to live with using XNview for the time being.
Or would a cooperation with it's dev help? He really has very fast thumb-creation and fluid interaction down...

Cheers,

Tom
"Out beyond the ideas of wrongdoing and rightdoing, there is a field. I’ll meet you there." - Rumi
ScreenDream|Thomas Helzle 8) Mastodon

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Alternative, non-locking Thumbnail generation Method

Post by highend »

Somehow it sounds a bit crazy...?
It could disturb whatever you're currently doing in the main instance if the slave sends some kind of refresh at any given interval. You'd better avoid that
One of my scripts helped you out? Please donate via Paypal

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Alternative, non-locking Thumbnail generation Method

Post by admin »

The slave could quietly fill the cache for the given folder, and tell the master when it's ready. That would be one message, one refresh (if the master is in that folder at that point).

ThomasHelzle
Posts: 39
Joined: 21 Dec 2016 15:08
Contact:

Re: Alternative, non-locking Thumbnail generation Method

Post by ThomasHelzle »

Yeah, but I don't think this is what we want.
If creating the thumbs takes 10 minutes, I'm no better off than at the beginning, or am I?

Cheers,

Tom
"Out beyond the ideas of wrongdoing and rightdoing, there is a field. I’ll meet you there." - Rumi
ScreenDream|Thomas Helzle 8) Mastodon

autocart
Posts: 1246
Joined: 26 Sep 2013 15:22

Re: Alternative, non-locking Thumbnail generation Method

Post by autocart »

ThomasHelzle wrote: 10 Nov 2021 19:36 Yeah, but I don't think this is what we want.
If creating the thumbs takes 10 minutes, I'm no better off than at the beginning, or am I?

Cheers,

Tom
Well, of course, the 2nd instance could not really react to the current scrolling position in the 1st instance.

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Alternative, non-locking Thumbnail generation Method

Post by highend »

It could, but still...
One of my scripts helped you out? Please donate via Paypal

autocart
Posts: 1246
Joined: 26 Sep 2013 15:22

Re: Alternative, non-locking Thumbnail generation Method

Post by autocart »

highend wrote: 10 Nov 2021 19:51 It could, but still...
It could? react to a change in the scrolling position in the other instance as well? Maybe it could tell the other instance to send the scrolling position back to the one instance. It would be better, if Don just did it build-in.

Post Reply