SC: UTF-8 decoding the '§'?

[Filehero]

Code pages, locales and transformations, my beloved topic. So I add the right upfront.

When run from XY's script editor (Run script...) this

Code: Select all

msg("§§ $ &");

shows what it should.

When this string sequence is read from an UTF-8 encoded script or ini file it (the paragraph char '§') shows up as

Code: Select all

msg("§§ $ &");

Why is that?

[highend]

A bug, it's read correctly from an UTF-8 with BOM

[admin]

If you want BOM-less UTF-8 detection you have to tick this:
Configuration | Preview | Text preview | UTF-8 auto-detection

[highend]

It's not about previewing files

Create two .xys files:
<xyscripts>\1.xys
<xyscripts>\2.xys

Save the first with UTF-8 BOM
and the second with UTF-8 (no BOM!)

Content of both

Code: Select all

msg("§§ $ &");

Execute them with "Scripting - Load Selected Script File"

The first displays the § correctly, the second does not.

[admin]

Setting "UTF-8 auto-detection" does not only affect preview, but reading UTF8 files in general.

[highend]

That may be the case but it fails on this specific case.

UTF-8, no BOM .xys script file and
Configuration | Preview | Text preview | [x] UTF-8 auto-detection
-> Wrong display of §

[admin]

Okay, got it. It's a bug. Fix comes.

[Filehero]

Thanks highend for deeper analysis, thanks Don for fixing in advance.

[Filehero]

Code: Select all

v18.40.0001 - 2017-09-21 16:14
    ! Scripting | Load Selected Script File: Did not decode BOM-less 
      UTF8-encoded script files when "Configuration | Preview | Text preview | 
      UTF-8 auto-detection" was ON. Fixed.

Hmm, I still have this issue with 18.40.0002.

[highend]

Hmm, I still have this issue with 18.40.0002

Works fine here since 0001

Zip that script file and attach it?

[Filehero]

Key <key_var_group> / line 21

[highend]

Displaying the content after using readfile() works fine
so I guess you're using e.g. getkey()?

[Filehero]

so I guess you're using e.g. getkey()?

Yes. I should have mentioned it, sorry.

[admin]

BOM-less UTF-8 ini-reading is not supported. Simply add the BOM and all is good.

[Filehero]

Like this?

utf-8-bom.png (73.01 KiB) Viewed 1042 times

I know why I h*te this darn encoding hell, sorry for bothering..