Invisible character when <crlf> present in writefile()

Things you’d like to miss in the future...
Post Reply
WirlyWirly
Posts: 195
Joined: 21 Oct 2020 23:33
Location: Through the Looking-Glass

Invisible character when <crlf> present in writefile()

Post by WirlyWirly »

I'm having trouble writing a file that contains <crlf> and then later pulling key:value pairs from that file using getsectionlist().

It seems that when you use writefile() and your string contains <crlf>, an invisible EOL character is appended everywhere <crlf> was used, which breaks the functionality of getsectionlist(). The function returns no results.

Here's how I reproduce the issue...

First, I use getsectionlist() to get all the key:value pairs in every section of my current configuration.ini (using the default <crlf> as the splitter between items)

Code: Select all

        // Get all current key:value pairs from .ini
        $sectionGeneralINI = getsectionlist("General", $childLauncherINI, "<crlf>");
        $sectionAppsAdditionalINI = getsectionlist("AppsAdditional", $childLauncherINI, "<crlf>");
        $sectionAppsRenameINI = getsectionlist("AppsRename", $childLauncherINI, "<crlf>");
        $sectionAppsIconINI = getsectionlist("AppsIcon", $childLauncherINI, "<crlf>");
        $sectionAppsHiddenINI = getsectionlist("AppsHidden", $childLauncherINI, "<crlf>");
I then read the contents of a template.ini file into a variable, then use regexreplace() to insert each list into that template. The default <crlf> splitter from the last step assures that every item in the list is in a new line, like it should be for a .ini.

Code: Select all

        // Read template.ini and replace placeholders with key:value pairs
        $templateINI = readfile($templateINI);
        $templateINI = regexreplace($templateINI, "---------- GENERAL ----------", $sectionGeneralINI);
        $templateINI = regexreplace($templateINI, "---------- APPSADDITIONAL ----------", $sectionAppsAdditionalINI);
        $templateINI = regexreplace($templateINI, "---------- APPSRENAME ----------", $sectionAppsRenameINI);
        $templateINI = regexreplace($templateINI, "---------- APPSICON ----------", $sectionAppsIconINI);
        $templateINI = regexreplace($templateINI, "---------- APPSHIDDEN ----------", $sectionAppsHiddenINI);
Finally, I overwrite the existing configuration file with the newly built template file.

Code: Select all

        $writeStatus = writefile($childLauncherINI, "$templateINI", "o", "utf8");
When viewing the newly written .ini in my editor, I can see an invisible EOL character where the <crlf> splitter broke each item. When I next run my script and try to use getsectionlist() to retrieve my saved settings, it returns no results. It won't return results until I remove each of the invisible characters from the file.

I've attached a .ini that showcases these invisible characters. It was generated using the commands I outlined above. If you use getsectionlist() on it you'll return with no results, even though that section clearly exists. However, getkey() still functions as normal.

I'm assuming this is either a bug with writefile() or getsectionlist().

If it's not, anyone know how I might use writefile() and not include these invisible line-ending characters? I've tried all the mode options, but I haven't been able to get it working.

*** Update ***
I just tried using setkey and got the same issue; An invisible EOL character is added and using getsectionlist() returns no results. After deleting the EOL character it works as expected.
Attachments
InvisibleCharacters.7z
(1.85 KiB) Downloaded 36 times

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Invisible character when <crlf> present in writefile()

Post by highend »

Without getting the template file I can't say much but apart of that: Your attached .ini file has only LF, not CRLF line endings. That's probably your problem...
One of my scripts helped you out? Please donate via Paypal

WirlyWirly
Posts: 195
Joined: 21 Oct 2020 23:33
Location: Through the Looking-Glass

Re: Invisible character when <crlf> present in writefile()

Post by WirlyWirly »

Hmm, I'm not sure why it would, I believe I'm using <crlf> as intended. I also tried the chr(13) . chr(10) trick you showed me earlier and the result was the same.

I've attached the template.ini, it's pretty normal from what I can tell, no invisible EOL characters. When I call readfile() on it, the non-placeholder parts get written fine and don't have the invisible EOL characters.
template.7z
(610 Bytes) Downloaded 45 times
Also, like I said in the update to the OP post, the invisible EOL character also appears when using setkey, which has no mention of a <crlf>

highend
Posts: 13274
Joined: 06 Feb 2011 00:33

Re: Invisible character when <crlf> present in writefile()

Post by highend »

You have a wild mix regarding line endings in your ChildLauncher.ini file
1.png
1.png (19.08 KiB) Viewed 1612 times
If I use the corrected one (-saved.ini), it works fine
One of my scripts helped you out? Please donate via Paypal

WirlyWirly
Posts: 195
Joined: 21 Oct 2020 23:33
Location: Through the Looking-Glass

Re: Invisible character when <crlf> present in writefile()

Post by WirlyWirly »

It must be my editor then, which is Vim through the GitBash terminal. I believe its using linux EOL, which is why there's a mix.

https://stackoverflow.com/questions/111 ... ine-in-vim

According to a reply on that StackOverflow, Vim is smart enough to adhere to existing EOL types, but a less intelligent editor wouldn't check and just use its default. Kind of like how some programs have a hard-coded call to explorer.exe, instead of asking the file system for its default manager.

I assume that's the case with these XY functions, they don't check to see what EOL characters are being used by the file, they just default to DOS EOL.

I would consider this a bug because the XY functions aren't checking for EOL, which will cause this issue when writing to any file with a Linux origin. However, being a windows program, I understand if it's outside the scope of XY.

I've run into similar problems before, the last time was when I first tried using your column edit script.

Horst
Posts: 1085
Joined: 24 Jan 2021 12:27
Location: Germany

Re: Invisible character when <crlf> present in writefile()

Post by Horst »

Time to change the Editor :)
Windows 11 Home x64 Version 23H2 (OS Build 22631.3374)
Portable XYplorer (actual version, including betas)
Everything 1.5.0.1371a (x64), Everything Toolbar 1.3.2, Listary Pro 6.3.0.69

WirlyWirly
Posts: 195
Joined: 21 Oct 2020 23:33
Location: Through the Looking-Glass

Re: Invisible character when <crlf> present in writefile()

Post by WirlyWirly »

Horst wrote: 21 May 2022 17:33 Time to change the Editor :)
That's a bit drastic for an EOL issue :naughty:

---

For anyone that may run into mixed EOL characters and experience similar issues, this is the Vim ex-command that will let you convert between unix/DOS linebreaks.

Code: Select all

:set fileformat=dos
and the reverse

Code: Select all

:set fileformat=unix
In my case, since I'm using a template.ini file, I converted all the linebreaks in that template file to DOS. Now that they're all DOS, reading/writing against the file using XY functions won't output mixed EOL's.

I still say this is a bug, though I'm sure it's very low-priority. Hopefully this will assist someone in the future who has similar issues :beer:

admin
Site Admin
Posts: 60357
Joined: 22 May 2004 16:48
Location: Win8.1 @100%, Win10 @100%
Contact:

Re: Invisible character when <crlf> present in writefile()

Post by admin »

I don't see a way to handle this special case without slowing down the normal cases which will be 99,9999... %. So I guess you will have to fix your input file.

Found a way, should work better in next beta.

WirlyWirly
Posts: 195
Joined: 21 Oct 2020 23:33
Location: Through the Looking-Glass

Re: Invisible character when <crlf> present in writefile()

Post by WirlyWirly »

Just tried out the fix, it works great. Files with mixed EOL are reading/writing proper.

Thanks a bunch! Like you said, it won't apply to 99.9999% of people, but for those .0001 % of us :party:

Post Reply