Raw view could not display Unicode correctly

Things you’d like to miss in the future...
Forum rules
When reporting a bug, please include the following information: your XYplorer version (e.g., v27.90.0047), your Windows version (e.g., Win 11), and your screen scaling percentage (e.g., 125%). We recommend adding your Windows version and screen scaling percentage to your profile or signature. This will make debugging much easier for us.
nf_xp
Posts: 35
Joined: 10 Jul 2009 08:05

Raw view could not display Unicode correctly

Post by nf_xp »

OS Language: Chinese

Don, I guess you didn't notice this bug before because it would not repro on English or German system.

When display the first character which ASCII code >= 0x80 in the Hex view, the character could not be displayed and all following characters are missing. As well as the ASCII code view, but it's only missing from the very next line:
2009-8-10 14-39-30.jpg
2009-8-10 14-39-30.jpg (32.66 KiB) Viewed 2848 times
There is a screen-shot of another hex-view app which displays the same file correctly:
2009-8-10 14-57-35.jpg
2009-8-10 14-57-35.jpg (61.28 KiB) Viewed 2850 times
It treats all characters as Western European (Windows) encoding and display them using corresponding font, and that's pretty much acceptable. Because if you try to honor the Unicode or MBCS in some systems like Chinese, Japan, the two font-size width characters or one character break into two line issue will totally mess up the hex-view.

Thanks!

admin
Site Admin
Posts: 64882
Joined: 22 May 2004 16:48
Location: Win8.1, Win10, Win11, all @100%
Contact:

Re: Raw view could not display Unicode correctly

Post by admin »

Interesting, thanks! I think I could fix. Only you can test it...

nf_xp
Posts: 35
Joined: 10 Jul 2009 08:05

Re: Raw view could not display Unicode correctly

Post by nf_xp »

My pleasure :D

nf_xp
Posts: 35
Joined: 10 Jul 2009 08:05

Re: Raw view could not display Unicode correctly

Post by nf_xp »

Still repro on v8.10.0021, did not see any change...

admin
Site Admin
Posts: 64882
Joined: 22 May 2004 16:48
Location: Win8.1, Win10, Win11, all @100%
Contact:

Re: Raw view could not display Unicode correctly

Post by admin »

nf_xp wrote:Still repro on v8.10.0021, did not see any change...
Wow, shocking.

Could you please post a screenshot of the first lines of any PNG file as seen with your other hex-view app. It will tell me something about the character conversions happening in China...

noir
Posts: 113
Joined: 26 Dec 2008 00:38

Re: Raw view could not display Unicode correctly

Post by noir »

i couldn't reproduce this on a Japanese system, but I will admit there is a few problems with multibyte charsets.
Attachments
Clip.jpg
Clip.jpg (95.19 KiB) Viewed 2749 times

nf_xp
Posts: 35
Joined: 10 Jul 2009 08:05

Re: Raw view could not display Unicode correctly

Post by nf_xp »

There are screenshots of the same PNG file in two different other apps:
hex001.jpg
hex001.jpg (105.75 KiB) Viewed 2741 times
hex002.jpg
hex002.jpg (74.18 KiB) Viewed 2743 times
Hope this helps.

admin
Site Admin
Posts: 64882
Joined: 22 May 2004 16:48
Location: Win8.1, Win10, Win11, all @100%
Contact:

Re: Raw view could not display Unicode correctly

Post by admin »

nf_xp wrote:There are screenshots of the same PNG file in two different other apps:
hex001.jpg
hex002.jpg
Hope this helps.
Thanks! I assume both are taken on a Chinese system. Looks like both apps take different strategies for certain characters (e.g. the very first character). I would prefer doing it like the second app because I think that's the more common/expected way.

Anyway, it's really a buffling problem. I tried something else. Check out next beta version...

nf_xp
Posts: 35
Joined: 10 Jul 2009 08:05

Re: Raw view could not display Unicode correctly

Post by nf_xp »

It almost get fixed in the latest release, all characters are displayed in most cases :wink:

But there are still one or two bugs. To describe them, I prepared four test files and some screenshots.
1.gif
1.gif (27.7 KiB) Viewed 2677 times
Repro steps:

1. Minimize info panel (Yes, this bug is raw view visible area related, with different size, it's behavior varies.)

2. Select files like this:
2.gif
2.gif (10.73 KiB) Viewed 2678 times
Looks like deselect don't reset state
Select 1, then 4, display correctly

3. Select files like this:
3.gif
3.gif (23.19 KiB) Viewed 2677 times
Select 2, then 4, display correctly

4. All other selecting orders display correctly (at least the first screen)


Minor issues:
1. Display incorrectly when treat two chars as one multi-bytes char. For example, C4 E3 21 21 21 21 21 21 21 21 21 21 21 21 21 21, will display one MB char, plus 14 '!', and plus one unexpected char which comes from the last char of the previous line. The number of unexpected chars is equals to the number of MB chars.
I know you don't plan to display any MB char, so it wouldn't be a problem in the future.
2. 0xFF displays in two font-size width in Chinese system (An old issue, also found in previous versions say v8.00.0000)

nf_xp
Posts: 35
Joined: 10 Jul 2009 08:05

Re: Raw view could not display Unicode correctly

Post by nf_xp »

Attach the test files
Attachments
test_files.part3.rar
(256 KiB) Downloaded 131 times
test_files.part2.rar
(256 KiB) Downloaded 130 times
test_files.part1.rar
(256 KiB) Downloaded 135 times

nf_xp
Posts: 35
Joined: 10 Jul 2009 08:05

Re: Raw view could not display Unicode correctly

Post by nf_xp »

Thanks!
Attachments
test_files.part4.rar
(55.33 KiB) Downloaded 127 times

admin
Site Admin
Posts: 64882
Joined: 22 May 2004 16:48
Location: Win8.1, Win10, Win11, all @100%
Contact:

Re: Raw view could not display Unicode correctly

Post by admin »

nf_xp wrote:Thanks!
Welcome. Next version has fix #3. Let's see... :)

admin
Site Admin
Posts: 64882
Joined: 22 May 2004 16:48
Location: Win8.1, Win10, Win11, all @100%
Contact:

Re: Raw view could not display Unicode correctly

Post by admin »

nf_xp wrote:It almost get fixed in the latest release, all characters are displayed in most cases :wink:

But there are still one or two bugs. To describe them, I prepared four test files and some screenshots.
...
Just to be sure: the first 4 screenshots where you printed "1 correct", "2 correct" etc. in red color -- are these errors (they look like errors) or correct results??

nf_xp
Posts: 35
Joined: 10 Jul 2009 08:05

Re: Raw view could not display Unicode correctly

Post by nf_xp »

Sorry for the wrong color, they are correct results.

admin
Site Admin
Posts: 64882
Joined: 22 May 2004 16:48
Location: Win8.1, Win10, Win11, all @100%
Contact:

Re: Raw view could not display Unicode correctly

Post by admin »

nf_xp wrote:Sorry for the wrong color, they are correct results.
But they look different on my system: no Chinese chars here.

So, if I got you right, the files display correctly the first time but not after you select and unselect some lines in the raw view? Or you mean select and unselect the files themselves?

Post Reply