[GRANTED] SC function wish - Base64 encoding/decoding

Features wanted...
Marco
Posts: 2306
Joined: 27 Jun 2011 15:20

[GRANTED] SC function wish - Base64 encoding/decoding

Post by Marco »

I wonder if there are other users out there who might find the ability to decode base64 strings (and conversely, encode files in base64) useful. One example would be extracting image files from mhtml files.
Last edited by Marco on 20 Feb 2014 11:48, edited 1 time in total.
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

admin
Site Admin
Posts: 50429
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: SC function wish - Base64 encoding/decoding

Post by admin »

Coming, but undocumented and not really optimized for speed:

Code: Select all

base64decode(string)
base64encode(string)
In the next beta please test this (especially if you are in an East-Asian locale):

Code: Select all

echo base64decode(base64encode("Köln")); //should return Köln
FAQ | XY News RSS | XY Twitter | Stay home! 🎸 Or wear a mask! 😷

TheQwerty
Posts: 4353
Joined: 03 Aug 2007 22:30

Re: SC function wish - Base64 encoding/decoding

Post by TheQwerty »

I hate you both! :evil:

I spent a couple days writing a script to do base64 conversions, but for any practical usage it was just too darn slow. Part of the problem is likely that I end up creating a binary string which quickly becomes huge. I hoped adding some caching could improve things until I figured out how to do it in a more buffered manner, but I haven't found the time/desire.

Not to mention I somehow forgot or didn't realize XY had a native HexToDec until more recently so I also wanted to replace my custom sub-script with calls to that instead.

Now that it's pointless I guess I'll just share it in its "too-darn-slow" state for posterity:
BitConverter.lib.xys
(23.71 KiB) Downloaded 114 times
At least I learned more than I ever wanted to know about base64 and dug a bit deeper into the messy character-encoding dung pile.

admin
Site Admin
Posts: 50429
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: SC function wish - Base64 encoding/decoding

Post by admin »

LOL. (sorry)

BTW, revised secret syntax:

Code: Select all

base64decode(string, isunicode)
base64encode(string, isunicode)
Example:

Code: Select all

echo base64decode(base64encode("日本語", 1), 1); //日本語
Why is it secret: because documenting takes longer than writing it!
FAQ | XY News RSS | XY Twitter | Stay home! 🎸 Or wear a mask! 😷

TheQwerty
Posts: 4353
Joined: 03 Aug 2007 22:30

Re: SC function wish - Base64 encoding/decoding

Post by TheQwerty »

admin wrote:Why is it secret: because documenting takes longer than writing it!
This seems to be happening more frequently as of late, and while I understand that it is a time-drain I'm not a fan of the only documentation available for things being forum posts.

What can we can do to help?
Aside from stop finding bugs and asking for changes. :P

admin
Site Admin
Posts: 50429
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: SC function wish - Base64 encoding/decoding

Post by admin »

TheQwerty wrote:Aside from stop finding bugs and asking for changes. :P
That's a great idea for starters.
FAQ | XY News RSS | XY Twitter | Stay home! 🎸 Or wear a mask! 😷

Marco
Posts: 2306
Joined: 27 Jun 2011 15:20

Re: SC function wish - Base64 encoding/decoding

Post by Marco »

admin wrote:That's a great idea for starters.
But we ain't starters :mrgreen:

So let's assume I wanted to write a script that, in order to something peculiar, requires an external executable. In the next beta I could:
0. write my whole script as usual
1. read the executable into a variable
2. base64encode such variable, and also add a CRLF every 70 characters as per common practice
3. embed such variable in my script

And upon unpacking, the script could remove the CRLFs from such variable, base64decode it and write back the executable on disk?
No more zip files but self-sustaining xys scripts? :)
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

admin
Site Admin
Posts: 50429
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: SC function wish - Base64 encoding/decoding

Post by admin »

Not sure about the 70 characters thing. Try it. I really have no time for it.
FAQ | XY News RSS | XY Twitter | Stay home! 🎸 Or wear a mask! 😷

TheQwerty
Posts: 4353
Joined: 03 Aug 2007 22:30

Re: SC function wish - Base64 encoding/decoding

Post by TheQwerty »

admin wrote:Not sure about the 70 characters thing. Try it. I really have no time for it.
Implementations of base64 that follow RFC 4648 should not insert line-feeds into the encoded data. This is something introduced in the MIME and PEM implementations of base64 encoding to get around limitations in SMTP.
Marco wrote:No more zip files but self-sustaining xys scripts? :)
I've been considering ways to package scripts for a while now. This would certainly be interesting depending on the performance hit. :?

admin
Site Admin
Posts: 50429
Joined: 22 May 2004 16:48
Location: Cologne, Win 8.1, Win 10
Contact:

Re: SC function wish - Base64 encoding/decoding

Post by admin »

TheQwerty wrote:
Marco wrote:No more zip files but self-sustaining xys scripts? :)
I've been considering ways to package scripts for a while now. This would certainly be interesting depending on the performance hit. :?
I don't think it's too bad. Try it. v13.70.0139 has it.
FAQ | XY News RSS | XY Twitter | Stay home! 🎸 Or wear a mask! 😷

Marco
Posts: 2306
Joined: 27 Jun 2011 15:20

Re: SC function wish - Base64 encoding/decoding

Post by Marco »

admin wrote:Not sure about the 70 characters thing. Try it. I really have no time for it.
No worries, I didn't mean that the function should do that. With formatlist() and the perforate switch I can achieve the 70-chars thing.
Thanks for the clarification TheQwerty.
Damn, I should be studying but all these shiny new addiction are attracting me much like a moth to the flame :twisted:
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

Marco
Posts: 2306
Joined: 27 Jun 2011 15:20

Re: SC function wish - Base64 encoding/decoding

Post by Marco »

Thanks Don, works perfectly! Honestly I don't care much about performance, I encoded 5 MB in a second (even if better performances are always welcome). Here's the code to embed/store files into a script

Code: Select all

 $path = "C:\file.ext";                                       //full path of the file you want to encode
 $file = readfile($path, "b");                                //reads the file into a variable in binary mode (switch "b")
 $encoding = base64encode($file);                             //base64-encodes the variable above
 $encoding = formatlist($encoding, "p", <crlf>, 76) . <crlf>; //splits the encoding into lines 76 characters long
                                                              //and appends a final newline, as done by this utility, http://www.f2ko.de/programs.php?lang=en&pid=b64
 text $encoding;
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

TheQwerty
Posts: 4353
Joined: 03 Aug 2007 22:30

Re: SC function wish - Base64 encoding/decoding

Post by TheQwerty »

A little fiddle to compare XY's results for encoding/decoding strings with C#'s: http://dotnetfiddle.net/YxJqeZ

Seem's okay so far, but I haven't gotten to play much with using it on binaries.

Marco
Posts: 2306
Joined: 27 Jun 2011 15:20

Re: SC function wish - Base64 encoding/decoding

Post by Marco »

I tried on one binary file (executable) and the result (encoding) was consistent with another command line utility. Decoding back shows no mismatches with the source file, so it's green light for me.

Code: Select all

"ENCODE"

 $path = <curitem>;                                           //full path of the file you want to encode
 $file = readfile($path, "b");                                //reads the file into a variable in binary mode (switch "b")
 $encoding = base64encode($file);                             //base64-encodes the variable above
 $encoding = formatlist($encoding, "p", <crlf>, 76) . <crlf>; //splits the encoding into lines 76 characters long
                                                              //and appends a final newline, as done by this utility, http://www.f2ko.de/programs.php?lang=en&pid=b64
                                                              //it's not strictly necessary, so you can comment it out

 writefile($path . ".base64.txt", $encoding, , "ta");         //writes the encoding on disk, with the same name of the source file, in ASCII mode (switch "ta")

-

"DECODE"

 $path = <curitem>;                                           //full path of the file you want to decode
 $encoding = readfile($path, "t");                            //reads the file into a variable in text mode (switch "t")
 $encoding = regexreplace($encoding, <crlf>);                 //strips any CRLF pair from the encoding
                                                              //only necessary if you perforated the encoding as above
 $file = base64decode($encoding);                             //base64-decodes the variable above
 $path = regexreplace($path, "\.base64\.txt$");               //removes the custom file extension
 writefile($path, $file, , "b");                              //writes the file on disk, with the same name of the source file, in binary mode (switch "b")
EDIT: ok, no problems so far with binary files. Some issues with decoding back text encoded in UTF-8:

Code: Select all

text base64decode("4pa2INChLiDQn9GA0L7QutC+0YTRjNC10LIuINCg0L7QvNC10L4g0Lgg0JTQttGD0LvRjNC10YLRgtCwIC8gUy4gUHJva29maWV2LiBSb21lbyBhbmQgSnVsaWV0IC0gMSAtIFlvdVR1YmU=");
gives

Code: Select all

▶ С. Прокофьев. Ромео и Джульетта / S. Prokofiev. Romeo and Juliet - 1 - YouTube
so you should perform an additional step with

Code: Select all

text utf8decode("▶ С. Прокофьев. Ромео и Джульетта / S. Prokofiev. Romeo and Juliet - 1 - YouTube");
to obtain (playable :D )

Code: Select all

▶ С. Прокофьев. Ромео и Джульетта / S. Prokofiev. Romeo and Juliet - 1 - YouTube
[/url]
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

Marco
Posts: 2306
Joined: 27 Jun 2011 15:20

Re: [GRANTED] SC function wish - Base64 encoding/decoding

Post by Marco »

PS: maybe this code can give ideas for better performance? http://www.di-mgt.com.au/src/basRadix64.txt
Tag Backup - SimpleUpdater - XYplorer Messenger - The Unofficial XYplorer Archive - Everything in XYplorer
Don sees all [cit. from viewtopic.php?p=124094#p124094]

Post Reply