Results 1 to 12 of 12

Thread: unicode character displays ok on Windows, not on Linux

  1. #1
    Join Date
    Oct 2009
    Posts
    364
    Thanks
    10
    Thanked 37 Times in 36 Posts
    Qt products
    Qt4
    Platforms
    MacOS X Unix/X11 Windows

    Default unicode character displays ok on Windows, not on Linux

    my Qt application uses a text file that contains the degree character '°' (unicode U+00B0). It displays fine when my app runs on windows, but on linux I get: '°'
    The text file that contains this character displays fine outside of my app.

    I'm puzzled.

  2. #2
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,729
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: unicode character displays ok on Windows, not on Linux

    You have a mismatch between the encoding of the text file and what you are telling telling Qt it is. The UTF-8 encoding of U+00B0 is 0xC2 0xB0. Your output is what those two bytes produce when they are interpreted as ISO 8859-1 characters rather than UTF-8.

  3. #3
    Join Date
    Oct 2009
    Posts
    364
    Thanks
    10
    Thanked 37 Times in 36 Posts
    Qt products
    Qt4
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: unicode character displays ok on Windows, not on Linux

    Quote Originally Posted by ChrisW67 View Post
    You have a mismatch between the encoding of the text file and what you are telling telling Qt it is. The UTF-8 encoding of U+00B0 is 0xC2 0xB0. Your output is what those two bytes produce when they are interpreted as ISO 8859-1 characters rather than UTF-8.
    but the real kicker is that the same code runs fine when compiled in Qt Creator on windows. I only see the problem on linux - same code, just a rebuild

  4. #4
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,359
    Thanks
    3
    Thanked 5,015 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: unicode character displays ok on Windows, not on Linux

    That's because Windows and Linux use different "local8Bit" encodings. For Windows this is probably utf-8 while for your Linux it is presumably latin1. Use QString::fromUtf8() to explicitly convert a utf-8 string to Unicode.
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  5. The following user says thank you to wysota for this useful post:

    schnitzel (21st February 2011)

  6. #5
    Join Date
    Oct 2009
    Posts
    364
    Thanks
    10
    Thanked 37 Times in 36 Posts
    Qt products
    Qt4
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: unicode character displays ok on Windows, not on Linux

    Quote Originally Posted by wysota View Post
    That's because Windows and Linux use different "local8Bit" encodings. For Windows this is probably utf-8 while for your Linux it is presumably latin1. Use QString::fromUtf8() to explicitly convert a utf-8 string to Unicode.
    The string that contains this character is actually being provided by QSettings as follows:
    Qt Code:
    1. myset->value("units","").toString();
    To copy to clipboard, switch view to plain text mode 

    I tried the following but without success:
    Qt Code:
    1. QString::fromUtf8(myset->value("units","").toByteArray().constData());
    To copy to clipboard, switch view to plain text mode 

  7. #6
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,359
    Thanks
    3
    Thanked 5,015 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: unicode character displays ok on Windows, not on Linux

    How do you store the value in the settings?
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  8. #7
    Join Date
    Oct 2009
    Posts
    364
    Thanks
    10
    Thanked 37 Times in 36 Posts
    Qt products
    Qt4
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: unicode character displays ok on Windows, not on Linux

    from a text file which gets stored into QSettings like this:

    Qt Code:
    1. QSettings("myinifile", QSettings::IniFormat);
    To copy to clipboard, switch view to plain text mode 

    I just noticed QSettings::setIniCodec, looks like I need to play with that tomorrow.
    Last edited by schnitzel; 19th February 2011 at 09:46.

  9. #8
    Join Date
    Oct 2009
    Posts
    364
    Thanks
    10
    Thanked 37 Times in 36 Posts
    Qt products
    Qt4
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: unicode character displays ok on Windows, not on Linux

    Here is what I've tried, still no luck:

    Qt Code:
    1. QSettings *cfg = new QSettings(fn,QSettings::IniFormat);
    2. cfg->setIniCodec("UTF-8");
    3. ...
    4. QString tmp = QString::fromUtf8(myset->value("units","").toByteArray().constData());
    To copy to clipboard, switch view to plain text mode 

  10. #9
    Join Date
    Jan 2006
    Location
    Warsaw, Poland
    Posts
    33,359
    Thanks
    3
    Thanked 5,015 Times in 4,792 Posts
    Qt products
    Qt3 Qt4 Qt5 Qt/Embedded
    Platforms
    Unix/X11 Windows Android Maemo/MeeGo
    Wiki edits
    10

    Default Re: unicode character displays ok on Windows, not on Linux

    Shouldn't it be:
    Qt Code:
    1. QString tmp = myset->value("units", "").toString();
    To copy to clipboard, switch view to plain text mode 
    ?
    Your biological and technological distinctiveness will be added to our own. Resistance is futile.

    Please ask Qt related questions on the forum and not using private messages or visitor messages.


  11. #10
    Join Date
    Oct 2009
    Posts
    364
    Thanks
    10
    Thanked 37 Times in 36 Posts
    Qt products
    Qt4
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: unicode character displays ok on Windows, not on Linux

    Quote Originally Posted by wysota View Post
    Shouldn't it be:
    Qt Code:
    1. QString tmp = myset->value("units", "").toString();
    To copy to clipboard, switch view to plain text mode 
    ?
    that's what I had initially and I just tried it again, still no luck. It is as if the setIniCodec("UTF-8") does nothing.

  12. #11
    Join Date
    Mar 2009
    Location
    Brisbane, Australia
    Posts
    7,729
    Thanks
    13
    Thanked 1,610 Times in 1,537 Posts
    Qt products
    Qt4 Qt5
    Platforms
    Unix/X11 Windows
    Wiki edits
    17

    Default Re: unicode character displays ok on Windows, not on Linux

    To make sure I have the situation correct...
    You have a UTF-8 encoded INI file that is being read by QSettings. When you retrieve the value of a key and try to interpret it as a QString you get the unexpected result from your first post.

    This test program functions correctly if the codec is set before any access to the settings. If I access a setting, even a completely unrelated one, before setting the codec then the file is read and cached as Latin1 text.
    Qt Code:
    1. #include <QtGui>
    2. #include <QDebug>
    3.  
    4. int main(int argc, char *argv[])
    5. {
    6. QApplication app(argc, argv);
    7.  
    8. QSettings settings("test.ini", QSettings::IniFormat);
    9. // uncomment the following line to break it
    10. // qDebug() << settings.value("missing").toString();
    11. settings.setIniCodec("UTF-8");
    12. qDebug() << settings.value("test").toString();
    13.  
    14. return app.exec();
    15. }
    To copy to clipboard, switch view to plain text mode 
    with
    Qt Code:
    1. $ cat test.ini
    2. test=°
    3. $ od -tc -tx1 test.ini
    4. 0000000 t e s t = 302 260 \n
    5. 74 65 73 74 3d c2 b0 0a
    To copy to clipboard, switch view to plain text mode 
    outputs
    Qt Code:
    1. "°"
    To copy to clipboard, switch view to plain text mode 
    Last edited by ChrisW67; 21st February 2011 at 07:03.

  13. The following user says thank you to ChrisW67 for this useful post:

    schnitzel (21st February 2011)

  14. #12
    Join Date
    Oct 2009
    Posts
    364
    Thanks
    10
    Thanked 37 Times in 36 Posts
    Qt products
    Qt4
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: unicode character displays ok on Windows, not on Linux

    @ChrisW67: The test program worked fine

    And then it dawned on me. QSettings is only half of it. I am also reading the ini file contents into a QTextEdit and hence I needed to set the codec there as well even though I'm not displaying the editor right away.
    Everything is working now.

Similar Threads

  1. Replies: 10
    Last Post: 17th July 2014, 11:52
  2. QRegExp Unicode Character Classes
    By Delagen in forum Qt Programming
    Replies: 2
    Last Post: 4th March 2010, 13:17
  3. Character by Character (Unicode?) File Reading
    By mclark in forum Qt Programming
    Replies: 4
    Last Post: 22nd April 2009, 16:28
  4. How to read QStringList character by character
    By iamjayanth in forum Qt Programming
    Replies: 4
    Last Post: 3rd April 2009, 12:25
  5. Unicode Character Problem
    By prakash in forum Qt Programming
    Replies: 1
    Last Post: 11th May 2006, 08:25

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.