Folder size caching

[n2k]

Hey, since I always have my folder sizes on, it can slow down browsing when navigating quickly through folders, would it be possible to cache folder sizes?
I guess the biggest problem is keeping the cache up to date, but I suppose this can be done in a smart way. For me personally, I wouldn't need kilobyte accurate folder sizes when viewing C:\ or D:\
So don't do deep level searches above a certain number, for it to be quick when re-caching is required.

[admin]

Tip: With the new columns you could build your own cache.

[LittleBiG]

I can give you a possible solution.
1. Choose an Extra column to keep the size of the folders.
2. Create a toolbar button to fill that Extra column with folder size and and to clean it. My extra column is ex4.

Code: Select all

"Refresh size"
  $token = "";
  foreach($token,formatlist(folderreport("dirs:{fullname},{size RAW}|","r",,,,"|"),"e")) {
    tag(replace(gettoken($token,2,",")," "), gettoken($token,1,","), ex4, 1);
  }
  #485;
  status "Saved size refreshed.";
"Remove size"
  sel a;
  tag(,, ex4, 2);
  sel;
  #485;
  status "Saved size removed.";

This script works on the actual folder.

3. Create a new custom column with this script:

Code: Select all

return report("{Dir {Extra 4}|{size RAW}|}","<cc_item>");

4. Show the custom column (custom column 1-5 or as a user column). The name of my column is "Size (keep folder)".

[n2k]

Thanks LittleBiG for the example script, but I was more thinking of a automatic cached folder size, without using any toolbar buttons.
I've tinkered a bit with scripting, but it's not really fast yet when retrieving folder sizes for the first time, also I feel like I'm missing key scripting components for this approach to be any good.

The script below is what I currently have and has the following shortcomings:

• doesn't trigger when list is changed (not hooked to auto refresh)
• I have to disable actual folder size calculation for this or I don't have the speed boost of the cache (Tools -> Configuration -> Tree and List -> List -> Show folder sizes = off) and therefor the followings things do not work either:
  • doesn't use correct color codings when a folder is empty
  • doesn't support status bar total selected size when folders are selected

The following list are things that could be optimized with better scripting (I would need better understanding of what is possible / most efficient):

• instead of using folderreport() to recursively get folder sizes I should first get all files in the folder, and then check if there are any sub-folders, for which the same code would apply
• while calculating sub-folder sizes it would be nice to the ui/column update periodically/asynchronously after each sub-folder's size has been calculated
• while in a loop calculating sub-folders sizes there should be a limit to how deep it should go
• I would like to write a function which is reusable at different parts in the script, with the current state of if checks is pretty limited without copy/pasting the same code

possible scripting bugs/unexpected behaviour:

• when variables are null/do not exist/have no value the variables name is outputted as text instead of "" (nothing)

My current custom column definition script (applied to files and folders as right-aligned text), this script will use the column Extra 1 as a cache to store the total size (in RAW format) and the last modified folder date separated by a comma.

Code: Select all

if (<cc_isfolder>) {
  $cache = report("{extra 1}", "<cc_item>");
  $cSize = gettoken($cache, 1, ",");
  $cDate = gettoken($cache, 2, ",");
  $mDate = report("{Modified yyyy-mm-dd hh:nn:ss.fff}", "<cc_item>");
  
  if (!$cSize) {
    $cSize = "0";
  }
  
  if (!$cDate) {
    $cDate = "1970-01-01 00:00:00.000";
  }
  
  if (datediff($cDate, $mDate, "ms") == 0) {
      return formatbytes($cSize, "KBR");
  }
  else {
    // find a way to recursively grab folder sizes, meaning when done once at the highest level it doesn't
    // need to redo it for all the subfolders when browsing to one, folder report does this behind the scenes
    $itemSizes = folderreport("items:{Size RAW}", "r", "<cc_item>");
    
    foreach($itemSize, $itemSizes, <crlf>, , "") {
      $totalSize = $totalSize + $itemSize;
    }
    
    tag($totalSize . "," . $mDate, "<cc_item>", ex1, 1);
    
    return formatbytes($totalSize, "KBR");
  }
}
else {
  return formatbytes(filesize("<cc_item>"), "KBR");
}

[LittleBiG]

Just a remark, the column "Modified" is not a trustworthy way to decide if the folder size was changed. Anyway, good luck with your approach, I hope you can produce something good in the end. But at a point you should decide if the pain isn't more than the gain.

From my point of view caching is enough for the biggest folders (which are needed to be cleaned sometimes) and only when I do the actual cleaning. The main goal is not to lose the folder size when I enter a subfolder, then go back. Any other calculation is just a waste of time. I keep it that simple.

[bzowk]

Hey Guys -

I know this is an older thread, but just added a column using the script posted in LittleBiG's post which uses a button to refresh and/or remove sizes. What a big help! As there are many folders in the paths I'm using it in, I much prefer it to an automatic refreshing one due to the time it takes to populate all of them. I added it my XYPlorer and it works great!

I have just one question, please. The "total size" values which are populated in the column are in bytes. Therefore, a folder whose total displays as 12.7gb in a standard Windows property view is displayed as "13730367925" in the custom column. Another example is a folder which is 367mb is displayed as "385863791." Even though there is a big difference in size between the two, it's difficult to distinguish even though the data is right there in front of you. The below screenshot shows what I'm talking about...


My Question
Is it possible to edit the script so that the returned values are displayed in a more presentable fashion? I'd think the easiest approach would be to have each display in gigabytes making the first example above appear as "12.7" and the second one "0.37".

I tried playing around with the script, but have had no luck

Thanks Guys!

[highend]

Just use this script for the custom column:

Code: Select all

return formatbytes(report("{Dir {Extra 4}|{size RAW}|}", "<cc_item>"), "GB", 2);

[bzowk]

Thanks!

Unfortunately, I haven't figured out how to use it yet. I see the script contains "Extra 4" so assume it is to be used with the matching column. However, when I try the script, nothing appears in the column. I also tried creating a "Custom column" and using the script. I got values reported this way, but they weren't accurate and most were "0.00gb"

Can you give me a hint? Sorry to be such a newb

[highend]

I see the script contains "Extra 4" so assume it is to be used with the matching column

No, this is about where the values are stored in the tag.dat file, not in a visible column in the XY ui

I got values reported this way, but they weren't accurate and most were "0.00gb"

When you have values you have setup the cc probably correct.
I can't comment on if they are accurate...

Show a screenshot that contains the original size column + your cc and let the original size column calculate the size before you make the screenshot (Menu - View - Calculate Folder Sizes)

My values are correct here...

[bzowk]

OK, nevermind - you're right

I must had done something incorrectly. I started from scratch using a new column and got it to work. For future reference for anyone who may wish to use this in the future (and are on my level), here's what i did:
1. Went to Tools / Configuration and choose "Custom Columns"
2. Selected an "Undefined" column and edited it
3. Entered a name then choose the type "Script"
4. Pasted the script (line) that highend so graciously posted

Code: Select all

return formatbytes(report("{Dir {Extra 4}|{size RAW}|}", "<cc_item>"), "GB", 2);

5. Important: Choose the item type "Files and Folders"
6. Saved it
7. Back in main window, right clicked on empty column area and choose "Add column"
8. Right clicked newly added column, selected "Select custom column", then choose the one create in step 4

Bingo - Thanks again highend...

[admin]

Just a little follow up: v16.80 comes with Folder Size Caching built in.

[roxyplo]

Thanks admin ! This is a HUGE leap for mankind ! Had to turn of FS just because it was recalculating each time !

I take that back...the checkbox is disabled for no apparent reason