Uncleaned reports needing regex

[PeterB]

I have been rather astonished that the Reports function doesn't offer a neat dump (if I didn't overlook this of course). It's Info Panel, then Report Tab, then Current Folder or Current List, and then "classic directory dump", and others that are not that useful.

This Classic directory dump is cluttered with too much information, I'm afraid of. I have a directory with downloads, with subfolder structure. The files are in the format

Name, then a dot, then 3 suffix characters, then at least 2 spaces (but within the name, it's possible there might be 2 spaces in a row, too, so searching just for 2 spaces can produce bad results), then there is the size in bytes, the date and the time, within that dump.

Which means it doesn't seem to be possible to have just the file names, together with their suffixes.
(@Admin: Other file managers I then tried produce even worse results here.)

So I tried my chanced with an editor, which means I put the dumb into an editor.

Here, I've been trying for 2 hours to create a correct find/replace thing, "regex" checked of course, and then search for:

Dot, 3times a-z, 2 or more spaces, then anything, and all this at the end of the line,

which would make something like

\.[a-z]{3} *$

The dollar sign was not accepted, and the "*" did not make all those endless digits (size, date, time) selected; it seems a "*" has nothing to do with regex where it would be a dot instead, followed by a number, but even if I search for

\.[a-z]{3} [0-9 :.]{100}

the selection comprises only the dot, the suffix and any spaces and stops before the first number (size, date, time).

I have similar problems with the replace then, since a formula like

\.[a-z]{3}

gives catastrophic results, the dot and suffix are replaced literally by a string "\.[a-z]{3}" which is of course not what I wanted: My aim was to preserve dot and suffix, of course. So the problem here is that you must search for dot and suffix, together with the unwanted characters to the right, but they mustn't be replaced at all, and it's nowhere I can find the necessary code in order to assure that those searched-for characters are left unchanged by the replace (Perl style btw, and of course the help file of my editor isn't of any help for non-programmers not knowing the secrets of regex already).

I would not post this problem here if it were external to XYplorer but it seems to me that I've bumped into a common problem for many XYplorer users here since I could imagine that many people using a file manager do want to have a print-out or a list, a dump, of a certain tree hierarchy, which means important folders, but only for the purpose of knowing what's in these folders, in the neatest possible way, for example, you want to store such a list for an external drive that you only connect here and then, sporadically, and here, the unwanted spaces, sizes, dates and times make you can't get a quick look at your list, a list that you then have to manually indent anyway if you want it neat. Explanation: If all this crap is deleted, then the much shorter lines could be printed in 2 or 3 columns, which would produce not only too much paper waste, but also incredibly ugly printed pages as it is now, where any visual search would be very unpleasant.

So the "reports" function of XYplorer is much better than that of some other programs, but it requires you to use regex replace within an external editor, which for non-programmers is not evident.

I, with all my efforts, can not do anything here for cleaning up the dump I'm so in need of. So I hope for some better search, and especially, better replace code, in order to try again.

Btw: In the total absence of the appropriate regex codes, alternatively it would be possible to have an editor just search for dot suffix 2 spaces, line per line, then have it do 2 backspaces, then have it delete the rest of the line (shift-end, then delete), then search the next line in the same way. Of course, this would be needed to be programmed, too...

[nas8e9]

Assuming you have a spreadsheet program (Excel, LibreOffice, etc.), I'd do a Basic info to CSV, to clipboard, and paste it in a spreadsheet. Any unneeded column can be easily deleted in the spreadsheet; afterwards, copy, save or print as needed.

Edited to add: Alternatively, you could script a solution by using the report command. The help file has more information in the Scripting Commands Reference-topic.

[FluxTorpedoe]

Hi'

• I'm not sure I really understand what kind of "neat dump" you want.
Isn't that clear enough - or too much? :
File | To Clipboard | Item Name(s)

• But if you need more-but-specific info, yes, folderreport() is the way to go, with a lot of possibilities to display the things you want the way you want.

• Alternately, just to answer your regex question. If you only want to strip filenames of their end-of-line, then after having made your dump to clipboard:

Code: Select all

$dump = <clipboard>;
  $dump = regexreplace(<clipboard>, "(\.\w{2,6})\s\s.*?$", "$1");
  text $dump;

Note:
- the regex searches for extensions backwards starting from the end of line (in case you have dots in your filenames).
- it searches for extensions that may be 2 to 6 'word' characters (because you can have digits in extensions, and they can be shorter or longer). But you can adjust this if you know what files you have.
- This regex wouldn't work in other editors without editing.

Hope this helps
Flux

[PeterB]

@nas8e9:
Thank you for your spreadsheet idea which I hadn't thought of.

Pasting the classic directory dump into cell A1 in MS Works Spreadsheet (WSS), and into Excel (E) is possible, but in both cases, any line is within the first cell of that line, so no separation. The same with "paste special" which is not even possible for WSS.

The same for the basic csv: Not possible in both programs, the clipboard content is not loaded. But you can put the basic csv into a file which you then try to open within WSS or Excel (no pasting, but open file). WSS refuses the format, but in Excel, you can set up some settings, especially ";" as the divider character, and it imports without fault, so the file names only (with path and suffix) are in the first column and can than be exported into other programs.

That seems very good to start with, but you don't have any tree structure, not even the folder names in an extra line each, above the corresponding files, but as said, every file name is with the full path instead, so this is not what I want: I want the folder names, then the corresponding files, and this for every subfolder, and if I can have it, I also would like to have some indentation or such, let's say a blank line above each such folder name line - and such a (real or approximative) tree structure would imply that the file names would be without the full path of course which takes the whole line if printed in 3 columns, the important file names being below. All this is very unsatisfactory.

[PeterB]

@FluxTorpedoe:
Thank you for your answer trying to answer my question by several angles. As for the regex, it's too difficult for me especially since there are dots within the date and time, and I don't know if I can enter 3 lines into the command line of XYplorer, perhaps by separating them with the ";".

File - To Clipboard - Item Names(s) would give the item names that are selected, but you cannot select a whole tree, with subfolders, anywhere, and I cannot manually open 100 subtrees one by one, and process then manually, for this command to be helpful for me.

I do not doubt that folderreport() can do anything I want, but the report I have access to by the gui of XYplorer does not do much. There are several formats, and several targets, but there is no possibility to option for more or less details for each file, as explained above, and frankly spoken, I had hoped that such functionality, and perhaps something like a tree structure - the current output is very Dos-like, with lines before and after each folder name line you would not want when a single blank line before would be much more eye-friendly - could be had, too (perhaps with indentation by option, but that would be rather minor since in the end, if you print in several columns, there is no real space for indentation anyway).

So at this moment I'm looking for a free or commercial folder structure dump utility, perhaps Directory List & Print, Directory Lister Pro, or other. Up to now I had been wondering why such programs exist and thought that by buying an expensive file manager, I'd have that functionality anyway, but that, as I have seen, is a mistake. So I'm really astonished that an expensive file manager cannot put out your directory structure in the simple form

folder (bold)
.subfolder (bold)
onlyfilename.suffix
onlyfilename.suffix
etc.suffix

.subfolder (bold)
..subfolder (bold)
filename.suffix
etc.suffix

..anothersubfolderhere (bold)
filename.suffix

.andanothersubfolderherebutonelevelupagain (bold
..subfolder (bold)
...
...

or something like that. I had really thought that went without saying if you pay a good price.

[RalphM]

@nas8e9:
...Pasting the classic directory dump into cell A1 in MS Works Spreadsheet (WSS), and into Excel (E) is possible, but in both cases, any line is within the first cell of that line, so no separation. The same with "paste special" which is not even possible for WSS.

In order to get the fields separated in Excel you have to split the data in column A based on the separator with "Data/Text to columns" which allows you to choose the separator to use for the splitting, i.e. ";".

[highend]

Just to show you how this could be done:

Select a folder from which you want to have a report (incl. all subfolders and their files):

Code: Select all

$subFolderList = "";
	foreach($selFolder, <get selecteditemspathnames |>) {
		if(exists($selFolder) == 2) { // Process only folders
			$subFolderReport = folderreport("dirs", "r", $selFolder, "r", , "<crlf>"); // Get a list of all subfolders
			if($subFolderReport == "") { break; }

 			$htmlOutput = "";
			foreach($folder, $subFolderReport, "<crlf>") {
				if($folder == "") { break; }
				$folderHeader = "<strong>$folder</strong></br>";
				$filesPerSubFolder = replace(folderreport("files", "r", $folder, , , "<crlf>"), "$folder\", ""); // Get a list of files in subfolder
				$filesPerSubFolder = replace($filesPerSubFolder, "<crlf>", "</br>");
					if($filesPerSubFolder == "") {
					$filesPerSubFolder = "</br>";
				} else {
					$filesPerSubFolder = $filesPerSubFolder . "</br></br>";
				}
				$htmlOutput = $htmlOutput . $folderHeader . $filesPerSubFolder;
			}
		}
	}
	html($htmlOutput, 800, 600);

[PeterB]

@RalphM
Thank you for mentioning further processing within Excel, and indeed that should be possible but is a lot of fuss.

As said, I looked up some third party tools, and I have discovered that most paying tools

Directory Report, 25 dollar, also does renaming and search for double files
Directory Lister Pro, 29 dollar, "special offer" by Bits for about 20 dollar
Print Directory, 1 (!) comp 20 dollar, 3 home-used (!) comps 40 dollar
and more

have lots of bells and whistles, for MORE info, but don't seem to be really good for LESS info - no screenshot showing "simple and neat" whatsoever for these three, at least.

But there is Directory List & Print, free or 20 dollar, which for 20 dollar, gives more details (which I don't want) and allows dump into a file (which is not really needed since the free version, too, allows for direct printing and for dumping into the clipboard, from which you then can do anything you want with it).

What is extraordinary here, even the free version does EXACTLY what I want and what I had imagined, less the formatting (of course) and less the indentation - can live with that since I have no superfluous lines (like in XYplorer, and they are really ugly there) but blank lines before any folder, then the filenames.suffix, nothing else. For really big folder structures, it takes a while to build them up and even asks, on 2,500, 5,000 and 10,000 lines if you want it to continue, but after some minutes (that's the price to pay and would be the same for the paid version I suppose), you'll have your structure with 20,000 files.

Even the free version lets you choose, with or without subfolders, ditto for files, ditto for size, ditto for date, ditto for time (which means you not only can opt for them as a group, but for each, individually), ditto for path in the file name or in an extra column, ditto for attributes, and for the suffix (each, individually, with or without). As said, output, in the free version, to printer or to file. The blank line before folders seems to be forced upon you, but since it's exactly what I want, and this program leaves out any unnecessary **** or ==== line or such, as can amply be found within the output of XYplorer. Attention, it's not a real tree, which means, for any folder, it lists first its subfolders, and then, those subfolders again, together with the corresponding files in them (or even when they are empty). So, any subfolder is given two times, first as "child" of its "parent", then as "parent" of its "children" - but I can live with that.

There is no trial version of the paid version, which means you cannot try to automatically format the folders - which would be worth the 20 dollars for me. So I have to find a way to format, to bold, any line that is directly under a blank line, or any line that has got a ":" or a "\" in it, since my file names don't contain either.

Since for the "price" - free - this is an incredible good (even if rather slow) program that provides real neat output, I think that sharing my find is useful for any non-programming user of XYplorer.

Source is http://www.infonautics.ch/directorylistprint/

[PeterB]

I've been reprimanded for multiplying posts here, instead of just editing them. So I edited the post above three times, but got aware that the "posting time" within the list page of this forum, for such updated posts, remains the original posting time. So, for other users, without manually checking your "old" post, it's not possible to become aware you've posted anew, since that outdated original posting time, just by updates. So I made a new post, in order to overcome this problem.


@highend: I just see your incredible script. So XYplorer is a programmer's file manager since its functionality, gui-wise, is far inferior to what it can do when you write scripts? Ok, ok, let's put it the other way round: Its gui-wise functionality is standard, but its in-built script language sets quite other standards then...for people who know to program, and that's the problem with this approach for everyone else.

@highend: I just run your incredible script, it even includes formatting of the folders, so it invalidates my Swiss find. But as I ran it, I got a message that the web page I wanted to obtain (!) wasn't available! (I ran it within the window "run scripts".) I must look again into that...

@highend: Yes, it says "Navigation to the webpage was cancelled". Seems to have to do with the html output (which in itself is a very good thing since html is much neater than .rtf is). I suppose this script takes some long seconds, on a big folder (with many subfolders I mean), and something in the script is running thru and not waiting for that "tree" to be built up, so there isn't any output in html format yet when the script wants to show the html file. That's just guessing, of course.


Another add-on: It seems that many such programs are 1-user (ok) 1-pc (not ok) only, incl. the one I described in detail - says cnet.com. I didn't check really, but you should, before buying anything.

[highend]

I just ran the script on a folder with 6766 subfolders, containing 17758 files. 10,4 GB overall.
Are you using an up to date (beta) version?


No error messages. Ofc this takes a while. Maybe it's less time consuming to do just one folderreport for files / subdirs and and strip unnecessary content via a few regexes. Haven't tried that though.

[nas8e9]

I just ran the script on a folder with 6766 subfolders, containing 17758 files. 10,4 GB overall.
Are you using an up to date (beta) version?


No error messages. Ofc this takes a while. Maybe it's less time consuming to do just one folderreport for files / subdirs and and strip unnecessary content via a few regexes. Haven't tried that though.

I'm probably having the same problem as PeterB, in that I get a "Navigation to the webpage was cancelled"-error.

When trying the script, I'm seeing something odd: after executing line 4, $subFolderReport is empty?

[highend]

I guess you both didn't select a folder before (but a file instead)?

It was just a small example to show that it's possible to do such a thing with scripting, no major error checking included.

[nas8e9]

I guess you both didn't select a folder before (but a file instead)?

It was just a small example to show that it's possible to do such a thing with scripting, no major error checking included.

I get no result with both a single and multiple folders selected.

Even without a stake in all this, thank you very much for your effort. Your ability to quickly script a solution (whether complete or to illustrate something) is, at least to this non-programmer, amazing.

[admin]

Even without a stake in all this, thank you very much for your effort. Your ability to quickly script a solution (whether complete or to illustrate something) is, at least to this non-programmer, amazing.

Agreed. And he has a very clean way of coding (exactly as I would do it ).

[highend]

Even without a stake in all this, thank you very much for your effort. Your ability to quickly script a solution (whether complete or to illustrate something) is, at least to this non-programmer, amazing.

Agreed. And he has a very clean way of coding (exactly as I would do it ).

You are kidding, right? oO

Btw, here is a different approach.

Should be 5x times faster than the previous one... (on large folders)

Code: Select all

end exists("<curitem>") != 2, "No folder selected, aborted!";
	$list = folderreport("dump", "r", "<curitem>", "r", , "<crlf>");
	$list = regexreplace($list, "-{10,}|={10,}|^\d+\sfile\(s\).*?$", ""); // Remove comments (- and =) and footers
	$list = regexreplace($list, "^.*<DIR>", ""); // Remove unnecessary directories
	$list = regexreplace($list, "([A-Z]:\\.*$)", "</br></br><strong>$1</strong>"); // Tag existing directories
	$list = regexreplace($list, "\s{2,}\d{1,}.*?$", ""); // Cleanup files size, date, etc.


	$list = formatlist($list, "e", "<crlf>"); // Remove all empty lines
	$list = regexreplace($list, "\r\n", "</br>"); // Replace line endings with </br>

 	html($list, 800, 600);