Invisible character when <crlf> present in writefile()

[WirlyWirly]

I'm having trouble writing a file that contains <crlf> and then later pulling key:value pairs from that file using getsectionlist().

It seems that when you use writefile() and your string contains <crlf>, an invisible EOL character is appended everywhere <crlf> was used, which breaks the functionality of getsectionlist(). The function returns no results.

Here's how I reproduce the issue...

First, I use getsectionlist() to get all the key:value pairs in every section of my current configuration.ini (using the default <crlf> as the splitter between items)

Code: Select all

        // Get all current key:value pairs from .ini
        $sectionGeneralINI = getsectionlist("General", $childLauncherINI, "<crlf>");
        $sectionAppsAdditionalINI = getsectionlist("AppsAdditional", $childLauncherINI, "<crlf>");
        $sectionAppsRenameINI = getsectionlist("AppsRename", $childLauncherINI, "<crlf>");
        $sectionAppsIconINI = getsectionlist("AppsIcon", $childLauncherINI, "<crlf>");
        $sectionAppsHiddenINI = getsectionlist("AppsHidden", $childLauncherINI, "<crlf>");

I then read the contents of a template.ini file into a variable, then use regexreplace() to insert each list into that template. The default <crlf> splitter from the last step assures that every item in the list is in a new line, like it should be for a .ini.

Code: Select all

        // Read template.ini and replace placeholders with key:value pairs
        $templateINI = readfile($templateINI);
        $templateINI = regexreplace($templateINI, "---------- GENERAL ----------", $sectionGeneralINI);
        $templateINI = regexreplace($templateINI, "---------- APPSADDITIONAL ----------", $sectionAppsAdditionalINI);
        $templateINI = regexreplace($templateINI, "---------- APPSRENAME ----------", $sectionAppsRenameINI);
        $templateINI = regexreplace($templateINI, "---------- APPSICON ----------", $sectionAppsIconINI);
        $templateINI = regexreplace($templateINI, "---------- APPSHIDDEN ----------", $sectionAppsHiddenINI);

Finally, I overwrite the existing configuration file with the newly built template file.

Code: Select all

        $writeStatus = writefile($childLauncherINI, "$templateINI", "o", "utf8");

When viewing the newly written .ini in my editor, I can see an invisible EOL character where the <crlf> splitter broke each item. When I next run my script and try to use getsectionlist() to retrieve my saved settings, it returns no results. It won't return results until I remove each of the invisible characters from the file.

I've attached a .ini that showcases these invisible characters. It was generated using the commands I outlined above. If you use getsectionlist() on it you'll return with no results, even though that section clearly exists. However, getkey() still functions as normal.

I'm assuming this is either a bug with writefile() or getsectionlist().

If it's not, anyone know how I might use writefile() and not include these invisible line-ending characters? I've tried all the mode options, but I haven't been able to get it working.

*** Update ***
I just tried using setkey and got the same issue; An invisible EOL character is added and using getsectionlist() returns no results. After deleting the EOL character it works as expected.

[highend]

Without getting the template file I can't say much but apart of that: Your attached .ini file has only LF, not CRLF line endings. That's probably your problem...

[WirlyWirly]

Hmm, I'm not sure why it would, I believe I'm using <crlf> as intended. I also tried the chr(13) . chr(10) trick you showed me earlier and the result was the same.

I've attached the template.ini, it's pretty normal from what I can tell, no invisible EOL characters. When I call readfile() on it, the non-placeholder parts get written fine and don't have the invisible EOL characters.

template.7z

Also, like I said in the update to the OP post, the invisible EOL character also appears when using setkey, which has no mention of a <crlf>

[highend]

You have a wild mix regarding line endings in your ChildLauncher.ini file

1.png

If I use the corrected one (-saved.ini), it works fine

[WirlyWirly]

It must be my editor then, which is Vim through the GitBash terminal. I believe its using linux EOL, which is why there's a mix.

https://stackoverflow.com/questions/111 ... ine-in-vim

According to a reply on that StackOverflow, Vim is smart enough to adhere to existing EOL types, but a less intelligent editor wouldn't check and just use its default. Kind of like how some programs have a hard-coded call to explorer.exe, instead of asking the file system for its default manager.

I assume that's the case with these XY functions, they don't check to see what EOL characters are being used by the file, they just default to DOS EOL.

I would consider this a bug because the XY functions aren't checking for EOL, which will cause this issue when writing to any file with a Linux origin. However, being a windows program, I understand if it's outside the scope of XY.

I've run into similar problems before, the last time was when I first tried using your column edit script.

[Horst]

Time to change the Editor

[WirlyWirly]

Time to change the Editor

That's a bit drastic for an EOL issue

---

For anyone that may run into mixed EOL characters and experience similar issues, this is the Vim ex-command that will let you convert between unix/DOS linebreaks.

Code: Select all

:set fileformat=dos

and the reverse

Code: Select all

:set fileformat=unix

In my case, since I'm using a template.ini file, I converted all the linebreaks in that template file to DOS. Now that they're all DOS, reading/writing against the file using XY functions won't output mixed EOL's.

I still say this is a bug, though I'm sure it's very low-priority. Hopefully this will assist someone in the future who has similar issues

[admin]

I don't see a way to handle this special case without slowing down the normal cases which will be 99,9999... %. So I guess you will have to fix your input file.

Found a way, should work better in next beta.

[WirlyWirly]

Just tried out the fix, it works great. Files with mixed EOL are reading/writing proper.

Thanks a bunch! Like you said, it won't apply to 99.9999% of people, but for those .0001 % of us