Page 3 of 5
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 01 Jul 2014 15:20
by Marco
Enternal, do you get the gibberish in the final file or in the variable that holds the content of the final file?
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 01 Jul 2014 22:17
by Enternal
Marco wrote:Enternal, do you get the gibberish in the final file or in the variable that holds the content of the final file?
Oh! Never thought of that. I tested it and it's gibberish in the variable that holds the content.
After that, I tried reading that variable that READS the file and that one is gibberish. So it really is weird. I simply used:
Text $File and Text $Encoding
to read display what was read from the file and in both cases for encoding and decoding, it's messed up.
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 01 Jul 2014 23:26
by Marco
You shouldn't worry about the content of the variable(s). Files are read in binary mode, that's why you see gibberish. But that's the point too, as you preserve the very bytes by doing so. That's what matters, because base64 is useful for encoding bytes into text.
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 02 Jul 2014 00:44
by Enternal
But if it supposed to preserve the very bytes, how come the decoding process did not revert it back to the original? So basically I encode "Hello!", I should get "Hello!" back out in the process. And I do get that if it's all Roman letters in a text file. The problem occurs when I used something that's not Roman letters in a text file like: Здравствуйте!
Encoding and decoding it straight using:
Text Base64Decode(Base64Encode("Здравствуйте!", 1), 1);
works perfectly fine but it becomes a problem if I try encoding and decoding a text file.
Basically I made a text file with UTF-8 encoding. Put:
Code: Select all
Hello!
I am crazy!
I love chickens
Encoding and decoding that file comes out perfectly fine. Now if I add a line that does not have Roman letters so that the file becomes:
Code: Select all
Hello!
I am crazy!
I love chickens
Здравствуйте!
Encoding and decoding failed. In this case, all the lines with Roman letters comes out fine but the line with Russian characters become gibberish.
I have a feeling that it's because of system encoding settings again... *sigh
EDIT: AND I was right on the money about system encodings. I tested on a purely English system:
Code: Select all
System Locale ID: 1033 (en-US)
Thread Locale ID: 1033 (en-US)
Default ANSI Code Page: 1252 (ANSI - Latin I)
Active Code Page: 1252 (ANSI - Latin I)
DBCS Code Page: No
and it works as expected. On the problem system:
Code: Select all
System Locale ID: 1041 (ja-JP)
Thread Locale ID: 1033 (en-US)
Default ANSI Code Page: 1252 (ANSI - Latin I)
Active Code Page: 932 (ANSI/OEM - Japanese Shift-JIS)
DBCS Code Page: Yes
The encoding and decoding failed with gibberish.
DON! SAVE ME!! I just love how I always manage to break something whenever it comes to encoding and decoding.
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 02 Jul 2014 07:43
by bdeshi
readfile() has a codepage parameter. passing 1252 as codepage could help.
Code: Select all
$File = readfile($Path, b, , 1252);
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 02 Jul 2014 10:16
by admin
Enternal wrote:DON! SAVE ME!! I just love how I always manage to break something whenever it comes to encoding and decoding.
The next version allows to decode/encode files directly; this should fix the issues. See the new code versions inserted below.
Code: Select all
"ENCODE"
$Path = <curitem>;
$File = ReadFile($Path, "b");
$Encoding = Base64Encode($File, 1);
$Encoding = Formatlist($Encoding, "p", <crlf>, 76) . <crlf>;
WriteFile($Path . ".Base64.txt", $Encoding, , "ta");
"ENCODE 2"
$Path = <curitem>;
$Encoding = Base64Encode($Path, 2);
WriteFile($Path . ".Base64.txt", $Encoding, , "ta");
"DECODE"
$Path = <curitem>;
$Encoding = ReadFile($Path, "t");
$Encoding = Regexreplace($Encoding, <crlf>);
$File = Base64Decode($Encoding, 1);
$Path = RegexReplace($Path, "\.Base64\.txt$");
WriteFile($Path, $File, , "b");
"DECODE 2"
$Path = <curitem>;
$File = Base64Decode($Path, 2);
$Path = RegexReplace($Path, "\.Base64\.txt$");
WriteFile($Path, $File, , "b");
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 02 Jul 2014 10:19
by admin
PS: secret syntax:
Code: Select all
base64decode(string, mode)
base64encode(string, mode)
mode:
1 = string is unicode
2 = string points to file
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 02 Jul 2014 13:11
by Enternal
Awesome! Thanks!
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 03 Jul 2014 09:30
by Enternal
So I tested it and even tested your new code that you provided. Unfortunately the problem is still there with the gibberish.
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 03 Jul 2014 10:30
by admin
For a UTF-16 file that just contains こんにちはI get this:
//5TMJMwazBhMG8w
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 03 Jul 2014 10:53
by Marco
admin wrote:For a UTF-16 file that just contains こんにちはI get this:
//5TMJMwazBhMG8w
Works here. Mind, Enternal: when you decode2 the $File variable will contain gibberish, but once it is written to disk the
Korean Japanese text will appear correctly (at least here on my system).
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 03 Jul 2014 11:06
by Enternal
Actually I was playing with Russian letters. Also, I just realized that I was using UTF-8 text files so when I switched that back to UTF-16, it works perfectly!
On the other hand, encoding a simple file and then decoding it still breaks it on my system. So why would that be the case? That's precisely why I was testing it with a text file because it's easier for me to "see" how a file changed. A text file should simply be text so if it's encoded and then decoded, no matter what it is, it should retain the information right? To note, I wasn't looking at the $File variable anymore since it's not very helpful to me. I look at the final output file and that's where I see the problem.
I might as well as attached an example of an original image (GTvmqWB-Original.jpg), then base64 encoded (GTvmqWB.jpg.Base64.txt), and then base64 decoded back (GTvmqWB-Decoded.jpg). The code used:
Code: Select all
"BASE64 ENCODE|:queue"
// http://www.xyplorer.com/xyfc//viewtopic.php?f=5&t=11016
// http://www.xyplorer.com/xyfc//viewtopic.php?f=5&t=11442&start=15
$Path = <curitem>;
$Encoding = Base64Encode($Path, 2);
WriteFile($Path . ".Base64.txt", $Encoding, , "ta");
"BASE64 DECODE|:queue"
$Path = <curitem>;
$File = Base64Decode($Path, 2);
$Path = RegexReplace($Path, "\.Base64\.txt$");
WriteFile($Path, $File, , "b");
System locale settings:
Code: Select all
System Locale ID: 1041 (ja-JP)
Thread Locale ID: 1033 (en-US)
Default ANSI Code Page: 1252 (ANSI - Latin I)
Active Code Page: 932 (ANSI/OEM - Japanese Shift-JIS)
DBCS Code Page: Yes
As you can see, what went in did came back out the same. I hope I make sense what my concerns were. Or... I am just completely using it wrong and not the way it was intended.
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 03 Jul 2014 11:17
by Marco
Ok, I think that the problems lies in your system locale. In fact, the images you posted are different, but doing the encode-decode passages here generates a bit-perfect copy. Specifically, the decoding passage seems broken on your config, because your encoded file is identical to the encoding that I get here. Let's see what Don can do.
Re text: when you encode-decode a file, it doesn't matter if it's text or binary. It will always be seen as a flow of bytes.
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 03 Jul 2014 11:25
by Enternal
Marco wrote:Ok, I think that the problems lies in your system locale. In fact, the images you posted are different, but doing the encode-decode passages here generates a bit-perfect copy. Specifically, the decoding passage seems broken on your config, because your encoded file is identical to the encoding that I get here. Let's see what Don can do.
Re text: when you encode-decode a file, it doesn't matter if it's text or binary. It will always be seen as a flow of bytes.
Yep! That's exactly what I was trying to get at!
For text, it's just easier for me to check how it changed because I don't have a hex editor on hand all the time. So if it's a text file, whatever I encoded BETTER be the SAME when I decode. Which in my case it did not which is why I questioned what happened. I use text files all the time to do quick checks like this including when I was playing around with encryption software. There was one software that had their encoding done wrong or something but what I encrypted and decrypted were not the same as the original. It's easier than encrypting a file, decrypt the file, and then pull out some hex editing software and compare the values. Too much work (lazy me).
After writing the above, I just realized that XYplorer has a hex viewer! Haha.
Re: [GRANTED] SC function wish - Base64 encoding/decoding
Posted: 03 Jul 2014 11:36
by admin
I had a little idea in the meantime. Try the next beta...