Folder size computations take a very long time

Please check the FAQ (https://www.xyplorer.com/faq.php) before posting a question...
lumi
Posts: 3
Joined: 05 Dec 2021 14:01

Folder size computations take a very long time

Post by lumi »

I typically work in a two-pane setup and frequently switch between large folders with sometimes 100,000+ files and display of folder sizes turned on. I use XYplorer portable version on Win10. Up until version 22.40 it would take a few seconds for folder sizes to refresh when switching to even the biggest folders, but since I updated to 22.50.0200 the updates take minutes. I tried restarting XYplorer, but it still takes very long for the sizes to be calculated. Of course I can bail by pressing ESC, but I wonder what might have changed in 22.50 for this time to be so much longer.

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Folder size computations take a very long time

Post by admin »

Is this ticked?

Configuration | General | Tree and List | List | Cache folder sizes

lumi
Posts: 3
Joined: 05 Dec 2021 14:01

Re: Folder size computations take a very long time

Post by lumi »

Yes, it is ticked. For reference, a roughly 7GB folder with 220,000 files and 35,000 subfolders takes about 5 minutes to have its size calculated. My computer is not especially slow (i5 processor, 16GB RAM, 2TB SSD).

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Folder size computations take a very long time

Post by admin »

There have been some minor changes in folder size calculation recently after 22.40.0000 but they all made it faster, not slower (of course).

Could you verify that 22.30.0200 really was faster than 22.50.0200? You can get it here: https://www.xyplorer.com/freezer.php?ver=22.30.0200

lumi
Posts: 3
Joined: 05 Dec 2021 14:01

Re: Folder size computations take a very long time

Post by lumi »

Well, I downgraded to 22.30.0200 and now it’s not calculating folder sizes on some drives – it puts a tilde (~) in front of the numbers – but it is on others. Auto-refresh is checked on Views. Clicking on Refresh (F5) doesn’t make the tilde go away once it’s there. Clicking on Calculate Folder Sizes does cause them to be calculated and the tilde goes away, which happens in a reasonable time (couple seconds). The difference in 22.50.0200 is that when I click on Calculate Folder Sizes a bar at the bottom says “getting folder size for …” whichever folder or drive, files and folders flash by, and it takes minutes to finish calculating.

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Folder size computations take a very long time

Post by admin »

Okay, I think I see what's causing it. Try again with the next beta.

WirlyWirly
Posts: 195
Joined: 21 Oct 2020 23:33
Location: Through the Looking-Glass

Re: Folder size computations take a very long time

Post by WirlyWirly »

I'm having a similar issue but on portable drives. Folder sizes aren't being cached and it takes very long time to calculate them. When I try to run a script while folder sizes are being calculated the script hangs until the process is done (SessionManager).

I'm wondering if there's a way to skip folder size generation for portable drives or why they might not be getting cached in the first place.

I do have this ticked: Configuration|General|Tree and List|Cache folder sizes

That setting almost works fine for folders on internal drives (It doesn't automatically save updated/new folder sizes unless I force File|Save Settings) but for portable drives it doesn't seem to do anything.

Any tips?

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Folder size computations take a very long time

Post by admin »

Not sure what you mean. Folder size calculation is not supported at all for portable devices. Did you have other experiences?

WirlyWirly
Posts: 195
Joined: 21 Oct 2020 23:33
Location: Through the Looking-Glass

Re: Folder size computations take a very long time

Post by WirlyWirly »

Yes, XY calculates folder sizes for my usb drive. I assumed this was normal. Tried it on my pc and laptop and it does it on both.

Folders with a few files calculate like normal, but I have folders with entire Django projects that take very long, as I would expect since they're full of small files.

If it matters, they're formatted as exFat.

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Folder size computations take a very long time

Post by admin »

Ah, USB! When you said "portable drives" I understood you meant "portable devices" (phones etc).

Well, without looking into the code, USBs are removable drives. So there doesn't seem to be much point in caching the folder sizes. If the folder structure is similar on two devices, the cached values are most likely incorrect.

Change log:

Code: Select all

v16.80.0009 - 2016-05-27 11:56
    * Folder Size Caching: Now removable drives are excluded from Folder Size 
      Caching. The variable drive letters don't play well with the current way 
      of caching.

WirlyWirly
Posts: 195
Joined: 21 Oct 2020 23:33
Location: Through the Looking-Glass

Re: Folder size computations take a very long time

Post by WirlyWirly »

I agree, but since they're not cached, XY tries to calculate them everytime I access the drive, which could take minutes with large folders.

Is there a option/tweak to disable folder size calculation for usb drives?

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Folder size computations take a very long time

Post by highend »

Try this one (not tested!):
ShowFolderSizesExclude=C:\|T:\|U:\
One of my scripts helped you out? Please donate via Paypal

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Folder size computations take a very long time

Post by admin »

Yep, that should work. I added the tweak anyway, it was kind of missing.

WirlyWirly
Posts: 195
Joined: 21 Oct 2020 23:33
Location: Through the Looking-Glass

Re: Folder size computations take a very long time

Post by WirlyWirly »

Thanks @Highend, that tweak will actually help speed up some virtual drives I use!

@Admin, I updated and added the tweak but the behavior remains unchanged; Folder sizes are still calculated for usb drives and not cached.

Code: Select all

ShowFolderSizeInListOnRemovables=0
I'm using detailed view with the size column enabled.

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Folder size computations take a very long time

Post by admin »

Sure you tweaked it right? Works fine here.

Post Reply