[β] Everything in XYplorer — Rev. 0.96 / 2014/07/24

Discuss and share scripts and script files...
Post Reply
Marco
Posts: 2223
Joined: 27 Jun 2011 15:20

[β] Everything in XYplorer — Rev. 0.96 / 2014/07/24

Post by Marco » 01 Jul 2014 12:16

[This first post gets updated from time to time, particularly when a new revision is released. You are strongly advised to read this post carefully!]


The bridge between the fastest MFT search engine - David Carpenter's Everything - and the most versatile file manager - Donald Lessau's XYplorer - on the planet.


Latest release notes

Code: Select all

Rev. 0.96 / 2014/07/24
 * Now $paper is defined as a base file name, i.e. without extension: ".txt"
   is added silently by the script when needed.

 * Use the Like operator instead of regxmatches() during IPC testing: it is
   now clearer and should be faster too.

 * Tweaked the console commands that load Everything service and client so
   that no invisible cmd.exe processes remain in the background when no
   needed.

 * Tweaked the timing of events during the IPC testing to cope with UAC.

 * Added a fancy status bar indication of how much time you have to wait for
   the script to load Everything service and client.

 * Tweaked the validation of the current path: it is now clearer and should be
   faster too.

 * A previous paper folder could overwrite a new paper folder under certain
   circumstances. Fixed, now paper folders are opened with the same unified
   code, no matter how their results were obtained.

 * Shortened paths in the form "paper:<xypaper>\$paper" to "paper:$paper" for
   better clarity and less (useless) redundancy.

 * Search time is now given with a resolution of 0.1 ms (for the time being…).
   Reason: datediff() doesn't work well with nanoseconds and sometimes gives
   negative results. Also, time is split into two contributes: parsing time
   (within XYplorer environment) and "data retrieval" time (outside XYplorer).

 * Other minor cosmetic syntax and comment changes.

 * Remarks: this version should feel more responsive than the previous one.

Rev. 0.95 / 2014/07/23
 * Added a new customizable parameter: $ipc. If set to TRUE the availability
   of Everything via IPC is tested, i.e. the script checks whether es.exe can
   successfully communicate with Everything and obtain search results, and
   starts Everything service and client if not.

 * Added a new customizable parameter: $diacritics. If set to TRUE (default)
   it matches diacritical marks, therefore a ≠ à. This behaviour can be
   temporarily overridden with the custom switch "-d".

 * Added a new customizable parameter: $loc_aware. If set to TRUE (default) it
   limits the scope of search to the current location and all its subfolders.
   This behaviour can be temporarily overridden with the custom switch "-l".

 * Note: the custom switches above must be placed at the very start of the
         query in order to be recognized, and the last letter must be followed
         by a blank space. They can be compounded and order doesn't matter, so
         "-dl " = "-ld ". However each letter must not be repeated, so "-ll "
         is not valid syntax.

 * Added a new customizable parameter: $unicode. If TRUE it tells es.exe to
   always output the search results to an output file. If set to FALSE instead
   (default), the script will try to read the search results from memory,
   first, but if it finds question marks (the sign of the presence of
   unrecognized Unicode characters) it will trigger the search a second time,
   effectively falling back to a TRUE status of the parameter and roughly
   doubling the execution time. Useful for people who rarely deal with CJK
   characters and can thus gain some speed, especially on low-end machines.

 * $query is now a global variable, making possible to call the script with an
   alias.

 * Added a safety belt: if you pass an empty query you are prompted again to
   enter a search term. Reason: an empty search query tells Everything to list
   all the items in its database, usually tens of thousands of items, making
   subsequent parsing and listing by XYplorer very long.

 * As a consequence of the point above, leading and ending spaces in the "raw"
   $query, i.e. without considering the custom switches, are trimmed. Also,
   leading and ending spaces in the query given through the input() interface
   are trimmed, in order to be consistent with the processing made by XYplorer
   on a search query inserted via an alias.

 * Added search time in the status bar, with a 100-nanosecond resolution. Time
   is measured from the moment the validation of $query starts (line after)
   till XYplorer opens the paper folder tab (line before).

 * Addendum: this is my first script where I use a XOR operation, I feel pro!
Download script
Everything in XYplorer.xys
Everything in XYplorer — Rev. 0.96 / 2014/07/24
(29.86 KiB) Downloaded 711 times
Everything in XYplorer.xys
Everything in XYplorer — Rev. 0.95 / 2014/07/23
(27.81 KiB) Downloaded 190 times
Everything in XYplorer.xys
Everything in XYplorer — Rev. 0.94 / 2014/07/09
(12.62 KiB) Downloaded 223 times

Screenshots

Image


Features

✔ Fully Unicode: want to listen to some Пётр Ильи́ч Чайко́вский but can't find the tracks on your drives? You're a fan of the グランツーリスモ saga but can't remember where you saved the tutorials? If Everything can find those files, Everything in XYplorer will bring them in your favourite file manager!
✔ Advanced syntax support™: you can pass whatever query Everything supports, with no need to escape any meta-character. Everything in XYplorer takes care of that for you.
✔ Advanced syntax support 2.0™: as an addition to the point above, you can still enjoy hassle/escaping-free searches and yet use es.exe advanced switches to refine your queries.
✔ Advanced syntax support 3.0™: custom shorthands to confine your search to the current path or to match diacritics are available.


Setup & Usage

In order for Everything in XYplorer to work, Everything itself must already be up and running. Even if this isn't the place to discuss about Everything set-ups, I'll describe what my configuration is like, so that if you can't get Everything in XYplorer to work you can follow the steps below. I use Everything as a limited user and in a portable way. That means:
- first loading the Everything service with the following (a UAC prompt will appear)

Code: Select all

C:\path\everything.exe -svc
- then loading the Everything client in background with

Code: Select all

C:\path\everything.exe -startup
Everything is now fully functional (here you can find more info about the service-client architecture of Everything, http://forum.voidtools.com/viewtopic.ph ... 72c80c5a9b).

es.exe is the required and necessary bridge between Everything and XYplorer. Download it from http://www.voidtools.com/download/ and put it in the folder where Everything.exe is located. This isn't strictly necessary, because you can put it anywhere, but a little guidance won't hurt.

Almost done. Now download this script and put it in <xyscripts> (again, it's a "suggestion"). Open it with a text editor and tweak the $path variable according to your system configuration. You have to enter there the path of the folder containing es.exe, as explained in the comments. Save the changes, and now you're really ready. Load the script as you do with any other script file and do your searches :D Really, there's not much to say: a prompt appears, you type your search query, then you hit Enter. The results will appear instantly. It just works :wink:

Want more info? RTFC, which means the same thing of RTFM and C stands for "Comments" :wink:


Tips

A nice place for this script is an user button in the toolbar. Here are the steps to create and configure one:
- download Everything in XYplorer, put it in <xyscripts> and tweak it as described above;
- type in the Address Bar

Code: Select all

::snippet
- in the window that appears paste

Code: Select all

Snip: CTB 1
Action
  NewUserButton
Name
  Everything in XYplorer
Icon
  :find
ScriptL
   load "Everything in XYplorer";
FireClick
  0
- click OK and you're done! A regular left click will show you the prompt where you can enter your search query.

However a keyboard shortcut is the fastest way to invoke Everything in XYplorer. Based on TheQwerty's guide, you should:
- download Everything in XYplorer, put it in <xyscripts> and tweak it as described above;
- ensure you have User-Defined Commands and Custom Keyboard Shortcuts enabled as features (F9 > Other - Features);
- go to User > Manage Commands…;
- in the Category section on the left select the last item, Load Script File;
- on the top right click New…, and then Add New Command;
- now type "Everything in XYplorer" (without quotes) both in the Caption and Script File boxes;
- finally, click Assign Keyboard Shortcut…: a window will all the available shortcuts will appear. I choose Alt+X on my system because I like it, so I press Alt+X and then Enter;
- now click OK and you're done! Hitting Alt+X will bring the Everything in XYplorer prompt.

FluxTorpedoe suggests an alias for Everything in XYplorer, and there truly is nothing simpler, you just need to
- download Everything in XYplorer, put it in <xyscripts> and tweak it as described above;
- type in the Address Bar

Code: Select all

@?=global $query = "<@1>"; load "Everything in XYplorer";
- then press Enter. From now on, whenever you type @? in the Address Bar, followed by a blank space and a search term, Everything in XYplorer will take care of finding that for you.


Tested on

* Microsoft® Windows® 8.1 with Update 1 Enterprise x64 ITA
* Microsoft® Windows® XP


Requirements

+ XYplorer (version 14.20.0305 or greater) with
- scripting enabled
+ Everything 1.3.4.670b (or greater) (available at http://www.voidtools.com/download/)
+ Everything Command-line Interface (available at http://www.voidtools.com/download/)


Notes

:!: :!: :!: I absolutely take no responsibility for whatever damage, direct or indirect, the usage of this script may cause. This code is provided as-is, use at your own risk. You are advised.

:!: :!: :!: I can't see all my files! Everything works by scanning/monitoring the MFT, so it sees (and displays) literally everything on an NTFS volume - even something tricky like the content of System Volum Information, which are protected by restrictive ACLs. Accessing those results is something entirely different. When opening a paper folder, XYplorer checks the existence/accessibility of each item before displaying it on the list. If you can't access it you won't see it. However, you can tick the option "Allow Zombies" to have XYplorer list every item that Everything found to overcome this "limitation".

:!: :!: :!: This script is huge and bloated. In the era of TB drives I would think thrice before calling a 13-KB text file "huge". Especially since it is UTF-16 LE encoded, which is expected to take up twice the space of a "regular" Windows 1252 encoded text file, but with the benefit of representing every known character. But yes, it is "bloated". With comments. Because they can be helpful. If you really need to save some KBs on your drive you can do this:
-select the script in XYplorer;
-type this in the Address Bar and hit Enter

Code: Select all

writefile("<curitem>", regexreplace(readfile("<curitem>"), "^/\*[\s\S]*?\*/\r\n\r\n|/\*[\s\S]*?\*/|\t.*|^\r\n| ツ"));
-now the script is reduced to a mere 774 bytes, as of Rev. 0.94. Are you happy?

:?: :?: :?: Will Everything in Xyplorer ever use an external configuration file? Will it ever be multilingual? The strength of Everything is its speed. Its live filter capability is amazing. I can do nothing to simulate this, all I can do is making the transfer of the results from Everything to XYplorer as fast as possible. Which means reducing the operations to a bare minimum. No security/validity checks, nothing. Reading a configuration or language file would introduce delay which, no matter how small, is the exact opposite of the goal of this project. So the answer is pretty much a "no". If you think that the amount of comments can impact negatively on the parsing times then read the point above.


Possible future updates

es.exe communicates with Everything using WM_COPYDATA messages, the same kind of messages natively supported by XYplorer. It might be possible, in the future, to entirely bypass es.exe and have a script that works out of the box, with no need for additional downloads.


:D :D :D Special thanks to…

* TheQwerty, for its guide and actually explaining me how to assign keyboard shortcuts to scripts
* FluxTorpedoe, for many suggestions


Previous releases notes

Code: Select all

Rev. 0.94 / 2014/07/09
 * Fixed a bug introduced with Rev. 0.93 due to the recent change in the name
   of a variable.

Rev. 0.93 / 2014/07/09
 * Back to a monolithic structure: entirely removed the Purge function. Please
   modify the CTB as per first post of the Everything in XYplorer thread.

 * No longer creates a unique paper folder for each search.

 * No longer attaches the query to the paper folder as a comment.

 * Search results are now opened in a new tab only if you're not currently
   viewing a paper folder created by this script.

Rev. 0.92 / 2014/07/08
 * Still beta status, but not for long.

 * Changed the default base name to "Everything in XYplorer".

 * Optimized the regular expression used to escape the minus signs in Smart
   mode:
   - dropped support for -h and --help, because they don't make any sense in a
   search, so their minus signs are ALWAYS escaped;
   - -n is recognized as a switch ONLY WHEN followed by a blank and one or
   more digits.

 * Swapped the if-then clauses regarding the minus escaping. Since the default
   Smart mode is more likely to be used, having it first in the if block
   should save some microseconds during execution.

 * Added display of the search query in the status bar after the paper folder
   is opened.

 * Search query is now attached to the paper folder as a comment, for future
   reference.

Rev. 0.91 / 2014/07/07
 * Still beta status.

 * Added (lots of) comments.

 * Added "two plus one" modes of escaping the minus sign (-).

 * Escaping of special characters no longer uses regexreplace() but
   replacelist() instead. Easier to read, shorter (saves one line), and maybe
   is faster too.

 * The resulting paper folder is now opened in a new tab, so the current tab
   is no longer lost.

 * Added subscript to delete all the paper folders generated by Everything in
   Xyplorer.

 * Due to the addition above, the script is no longer monolithic but formed by
   two subscripts instead.

Rev. 0.90 / 2014/07/01
 * The first version, a public beta.
[/size]

Additional downloads

This section intentionally left blank.
Last edited by Marco on 24 Jul 2014 21:08, edited 16 times in total.
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

admin
Site Admin
Posts: 45219
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: EveryXYing

Post by admin » 01 Jul 2014 14:50

Great! ET seems to be very popular, just like XY, so EveryXYing should get popular squared. :appl:

(I did not try the script, too busy...)

binocular222
Posts: 1409
Joined: 04 Nov 2008 05:35
Location: Hanoi, Vietnam

Re: EveryXYing

Post by binocular222 » 01 Jul 2014 15:16

Awesome! I have been using Everything for a loooong time but never notice it has a separate command line es.exe
No need for "Everything.exe -svc"
I'm a casual coder using AHK language. All of my xys scripts:
http://www.xyplorer.com/xyfc/viewtopic. ... 243#p82488

admin
Site Admin
Posts: 45219
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: EveryXYing

Post by admin » 01 Jul 2014 15:21

I now begin to see that the combo runret + paperfolder is true gold. Looking forward to what will be discovered...

Marco
Posts: 2223
Joined: 27 Jun 2011 15:20

Re: EveryXYing

Post by Marco » 01 Jul 2014 15:24

binocular222 wrote:No need for "Everything.exe -svc"
As long as you're already working as admin, for ET requires admin rights to access the MFT.
Otherwise you need to start the ET service (elevated), and then the ET "client" (non-elevated).

I'm going to ask void (the developer) if he/she can fix the Unicode thing.
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

Marco
Posts: 2223
Joined: 27 Jun 2011 15:20

Re: EveryXYing

Post by Marco » 01 Jul 2014 15:29

admin wrote:I now begin to see that the combo runret + paperfolder is true gold. Looking forward to what will be discovered...
Unfortunately any passage involving the console destroys Unicode characters. I'll see what can be done.

EDIT: I almost solved it :D
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

binocular222
Posts: 1409
Joined: 04 Nov 2008 05:35
Location: Hanoi, Vietnam

Re: EveryXYing

Post by binocular222 » 01 Jul 2014 16:08

Marco wrote:I'm going to ask void (the developer) if he/she can fix the Unicode thing.
He is David Carpenter (Help > About)
I'm a casual coder using AHK language. All of my xys scripts:
http://www.xyplorer.com/xyfc/viewtopic. ... 243#p82488

Marco
Posts: 2223
Joined: 27 Jun 2011 15:20

Re: EveryXYing

Post by Marco » 01 Jul 2014 17:08

It's still pretty short :mrgreen:
Tested with Italian, English, Russian and Japanese.
Any test on configurations where paths with spaces are involved is welcome.
I'll try to add some smart handling of queries with meta-characters.

Code: Select all

 $es = "C:\Everything\es.exe";                                  //the full path of es.exe executable

 $paper = "<xydata>\Paper\" . "Results" . ".txt";               //the full path of the paper folder containing the search results,
                                                                //ideally you should change the name only. Will be shorter with the upcoming native <xypaper>

 $query = input("Everything in XYplorer ™", "Type your search query as you would in Everything", "Everything is awesome... [it's a movie quote ツ]", "s");

 $list = runret("cmd /c chcp 65001 && $es $query > $paper");    //the core, now the console handles Unicode but won't show it properly, hence
                                                                //runret() can't be used anymore and the old-school redirection is the only way.
                                                                //Any suggestion/help/improvement with quoting is welcome!

 writefile("$paper", readfile("$paper"));                       //trick to convert the BOM-less UTF-8 file created by the console into a UTF-16 LE one with BOM,
                                                                //which XYplorer will read properly (hopefully this will become unnecessary)

 goto paper:$paper;
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

Marco
Posts: 2223
Joined: 27 Jun 2011 15:20

Re: [Coming soon] Everything in XYplorer — Rev. 1.00 / 2014/

Post by Marco » 02 Jul 2014 12:33

Updated first post with a fully functional beta. OP will be WIP until the weekend.
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

admin
Site Admin
Posts: 45219
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: [Coming soon] Everything in XYplorer — Rev. 1.00 / 2014/

Post by admin » 02 Jul 2014 12:36

I say: WOW! :appl: :cup: :appl: :cup: :appl:

Enternal
Posts: 1171
Joined: 10 Jan 2012 18:26

Re: [Coming soon] Everything in XYplorer — Rev. 1.00 / 2014/

Post by Enternal » 02 Jul 2014 13:13

Sweet! This is getting really really awesome! :biggrin:

Marco
Posts: 2223
Joined: 27 Jun 2011 15:20

Re: [Coming soon] Everything in XYplorer — Rev. 1.00 / 2014/

Post by Marco » 02 Jul 2014 15:47

Thank you guys!
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

admin
Site Admin
Posts: 45219
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: [Coming soon] Everything in XYplorer — Rev. 1.00 / 2014/

Post by admin » 02 Jul 2014 15:53

Which exact version of Everything should I download?

Marco
Posts: 2223
Joined: 27 Jun 2011 15:20

Re: [Coming soon] Everything in XYplorer — Rev. 1.00 / 2014/

Post by Marco » 02 Jul 2014 15:55

I use the portable one, x64. This > http://www.voidtools.com/Everything-1.3.4.670b.x64.zip
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

admin
Site Admin
Posts: 45219
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: [Coming soon] Everything in XYplorer — Rev. 1.00 / 2014/

Post by admin » 02 Jul 2014 16:08

Well, it works! Took me 2 minutes from download to my first ultra-fast system search using ET+XY. Wow.

Post Reply