Results 1 to 9 of 9

Thread: QProcess "readAllStandardOutput" unicode issue

  1. #1
    Join Date
    May 2013
    Posts
    321
    Thanks
    9
    Thanked 8 Times in 8 Posts
    Qt products
    Qt5
    Platforms
    MacOS X Unix/X11 Windows

    Default QProcess "readAllStandardOutput" unicode issue

    Hi everybody!
    Everything fine with QProcess but the result text I have from Windows looks to not be UTF-8.
    Is there a way to have accurate UTF-8 from "readAllStandardOutput"?
    If possible I'm looking for a cross platform solution.
    Thanks a lot!

  2. #2
    Join Date
    Jan 2006
    Location
    Bremen, Germany
    Posts
    554
    Thanked 86 Times in 81 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: QProcess "readAllStandardOutput" unicode issue

    Since the console on windows does not output utf-8 encoded but locale aware strings, you have to use QString::fromLocal8Bit().

  3. #3
    Join Date
    May 2013
    Posts
    321
    Thanks
    9
    Thanked 8 Times in 8 Posts
    Qt products
    Qt5
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: QProcess "readAllStandardOutput" unicode issue

    I tried but I can't see correctly accent characters and instead there is a wrong character.
    I get the result from CMake to print the standard output and error output.
    Calling CMake from a terminal shows correctly the accent characters so it must be possible to get this full data correctly and have it in UTF-8...
    Last edited by Alundra; 7th February 2021 at 13:56.

  4. #4
    Join Date
    Jan 2006
    Location
    Bremen, Germany
    Posts
    554
    Thanked 86 Times in 81 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: QProcess "readAllStandardOutput" unicode issue

    How do you check? Use a QMessageBox instead stdout and show some code.

  5. #5
    Join Date
    May 2013
    Posts
    321
    Thanks
    9
    Thanked 8 Times in 8 Posts
    Qt products
    Qt5
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: QProcess "readAllStandardOutput" unicode issue

    Here the code used to call cmake:
    Qt Code:
    1. QProcess compileProcess;
    2. compileProcess.start("cmake", {"--build", ".", "--config", "Release"});
    3. compileProcess.waitForFinished();
    4. QByteArray allStandardOutput = compileProcess.readAllStandardOutput();
    To copy to clipboard, switch view to plain text mode 
    I added a breakpoint to see the value of of the allStandardOutput and I can see also in Visual Studio the data inside doesn't have the accent but is replaced by ','.
    Here a screenshot of the result from the breakpoint:
    UnicodeProblem.jpg
    You can see at the end of the image, the text should be "Tous droits réservés."
    It's like the data is in ASCII so any conversion code will never end to something different. Outside the ASCII range got replaced by a character.
    Last edited by Alundra; 7th February 2021 at 14:38.

  6. #6
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,536
    Thanked 284 Times in 279 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: QProcess "readAllStandardOutput" unicode issue

    Open standard CMD window and run CHCP. You will see the current codepage setting for the console.

  7. #7
    Join Date
    May 2013
    Posts
    321
    Thanks
    9
    Thanked 8 Times in 8 Posts
    Qt products
    Qt5
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: QProcess "readAllStandardOutput" unicode issue

    The result is: 850
    https://docs.microsoft.com/en-us/win...-commands/chcp
    Gives: 850 = Multilingual (Latin I)
    But I already tried to call QString::fromLatin1 and he bypass the accents.
    I also tried to put the text in a QMessageBox, same results:
    Qt Code:
    1. QMessageBox::critical(this, "Test", QString::fromLatin1(allStandardOutput));
    To copy to clipboard, switch view to plain text mode 

  8. #8
    Join Date
    Jan 2006
    Location
    Bremen, Germany
    Posts
    554
    Thanked 86 Times in 81 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: QProcess "readAllStandardOutput" unicode issue

    Did you actually read my first post? I told you that you have to use QString::fromLocal8Bit() to convert your input from QProcess to a QString. Now you're either using a plain QByteArray or QString:::fromLatin1() for no reason...

  9. #9
    Join Date
    May 2013
    Posts
    321
    Thanks
    9
    Thanked 8 Times in 8 Posts
    Qt products
    Qt5
    Platforms
    MacOS X Unix/X11 Windows

    Default Re: QProcess "readAllStandardOutput" unicode issue

    I tried without success yes:
    Qt Code:
    1. QMessageBox::critical(this, "Test", QString::fromLocal8Bit(allStandardOutput));
    To copy to clipboard, switch view to plain text mode 
    It gives the same result with the ',' instead of 'é'.
    The plain QByteArray is the result of the readAllStandardOutput call and the screenshot I posted previously shows his data from a breakpoint in Visual Studio.
    https://doc.qt.io/qt-6/qprocess.html...StandardOutput

    EDIT:
    I saved the text in a text file from the raw QByteArray data and I opened in Visual Studio Code and I could change the encoding and found that it has to be cp850.
    I wonder if there is a cross platform safe way to know the encoding to convert to UTF-8 in a safe way.


    Added after 16 minutes:


    I wonder if Qt6 has something safe converting cp850 to UTF8, looks like it's not a big deal looking this link:
    https://gist.github.com/pedrosancao/9810075
    But if Qt6 has something already in place to do it, that would be great.


    Added after 10 minutes:


    Looks like Qt5 has: QTextCodec *codec = QTextCodec::codecForName("IBM 850");
    I wonder why I don't see it listed in Qt6: https://doc-snapshots.qt.io/qt6-dev/...converter.html


    Added after 6 minutes:


    I will surely open a bug ticket about it.


    Added after 6 minutes:


    Bug ticket link: https://bugreports.qt.io/browse/QTBUG-90988
    Last edited by Alundra; 7th February 2021 at 15:49.

Similar Threads

  1. QWidgets replacing "1/2" characters with single unicode ½ symbol
    By jason.gibbs@x-navtech.com in forum Qt Programming
    Replies: 0
    Last Post: 4th April 2016, 19:57
  2. Replies: 1
    Last Post: 20th November 2015, 11:02
  3. Replies: 3
    Last Post: 16th March 2015, 08:31
  4. Replies: 10
    Last Post: 17th July 2014, 11:52
  5. Translation QFileDialog standart buttons ("Open"/"Save"/"Cancel")
    By victor.yacovlev in forum Qt Programming
    Replies: 4
    Last Post: 24th January 2008, 20:05

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.