Results 1 to 18 of 18

Thread: ReadAllStandardOutput() is returning empty characters if they are not recognised

  1. #1
    Join Date
    Jul 2019
    Posts
    32
    Thanks
    5
    Qt products
    Qt5
    Platforms
    Windows

    Default ReadAllStandardOutput() is returning empty characters if they are not recognised

    I have a QProcess that runs a youtube-dl command to get the title of videos. I read the output of the process when it's finished and if the title of the video has some japanese characters for example, readAllStandardOutput() is not returning them. For example, if I have title "Learn Hiragana *some japanese characters here* (Japanese alphabet)" the readAllStandardOutput() function is returning "Learn Hiragana (Japanese alphabet)".

    How can I solve this?

  2. #2
    Join Date
    Jan 2006
    Location
    Bremen, Germany
    Posts
    554
    Thanked 86 Times in 81 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    How do you display the text? Since readAllStandardOutput() returns a QByteArray you have to convert it properly to a QString with QString::fromLocal8Bit()

  3. #3
    Join Date
    Jul 2019
    Posts
    32
    Thanks
    5
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    Quote Originally Posted by ChristianEhrlicher View Post
    How do you display the text? Since readAllStandardOutput() returns a QByteArray you have to convert it properly to a QString with QString::fromLocal8Bit()
    I tried using QString::fromLocal8Bit() but it's the same because I'm printing the QByteArray as it is.
    My code:

    mainwindow.h

    Qt Code:
    1. #ifndef MAINWINDOW_H
    2. #define MAINWINDOW_H
    3.  
    4. #include <QWidget>
    5. class mainWindow : public QWidget
    6. {
    7. Q_OBJECT
    8. //here are defined some variables
    9. public:
    10. mainWindow(QWidget *parent = 0);
    11. ~mainWindow();
    12. void createUI();
    13. void process();
    14. QProcess *process = new QProcess(this);
    15.  
    16. private slots:
    17. void ReadOutput(int, QProcess::ExitStatus);
    18.  
    19. };
    20. #endif // MAINWINDOW_H
    To copy to clipboard, switch view to plain text mode 

    mainwindow.cpp
    Qt Code:
    1. #include <QWidget>
    2. #include <QPushButton>
    3. #include <QProcess>
    4. #include <QByteArray>
    5. #include <QTextCodec>
    6. #include <QString>
    7. #include <QDebug>
    8. #include "mainwindow.h"
    9.  
    10. mainWindow::mainWindow(QWidget *parent) : QWidget(parent)
    11. {
    12. createUI();
    13. connect(process, SIGNAL(finished(int, QProcess::ExitStatus)), this, SLOT(ReadOutput(int, QProcess::ExitStatus)));
    14. }
    15.  
    16.  
    17. mainWindow::~mainWindow()
    18. {
    19.  
    20. }
    21. void mainWindow::createUI(){
    22. //here I create the look of the window
    23. QPushButton buttonsearch = new QPushButton("Start process", this);
    24. buttonsearch->setToolTip("Start process");
    25. buttonsearch->setGeometry(200, 290, 100, 30);
    26. connect(buttonsearch, &QPushButton::clicked, [this]() {process(); });
    27. }
    28.  
    29. void mainWindow::process(){
    30. process->setProcessChannelMode(QProcess::MergedChannels);
    31. process->start("\"D:\\YTDownloader\\youtube-dl.exe\" -e --no-playlist https://www.youtube.com/watch?v=6V-wwfuxZxw");
    32.  
    33. void readOutput(int exitCode, QProcess::ExitStatus exitStatus){
    34. qDebug() << exitCode;
    35. qDebug() << exitStatus;
    36. QByteArray a = process->readAllStandardOutput();
    37. qDebug() << a; //here is not showing characters that are not recognised
    38. QTextCodec* utfCodec = QTextCodec::codecForName("UTF-8");
    39. processStdout = utfCodec->toUnicode(a);
    40. qDebug() << processStdout;
    41. }
    To copy to clipboard, switch view to plain text mode 

    main.cpp
    Qt Code:
    1. #include <QApplication>
    2. #include "mainwindow.h"
    3.  
    4. int main(int argl,char *argv[])
    5. {
    6. QApplication app(argl,argv);
    7.  
    8. mainWindow *window = new mainWindow();
    9. window->setWindowTitle("Test");
    10. window->setFixedSize(700, 335);
    11. window->show();
    12.  
    13. return app.exec();
    14. }
    To copy to clipboard, switch view to plain text mode 

  4. #4
    Join Date
    Jan 2006
    Location
    Bremen, Germany
    Posts
    554
    Thanked 86 Times in 81 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    Don't rely on windows' cmd.exe - it can't handle unicode properly. Use a QMessageBox or similar.

  5. #5
    Join Date
    Jul 2019
    Posts
    32
    Thanks
    5
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    Quote Originally Posted by ChristianEhrlicher View Post
    Don't rely on windows' cmd.exe - it can't handle unicode properly. Use a QMessageBox or similar.
    Sorry but I don't understand what you mean. How to use a QMessageBox to run a QProcess?

  6. #6
    Join Date
    Jan 2006
    Location
    Bremen, Germany
    Posts
    554
    Thanked 86 Times in 81 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    Quote Originally Posted by INeedADollar View Post
    Sorry but I don't understand what you mean. How to use a QMessageBox to run a QProcess?
    Where did I told you to do so? I said you should print the return value from readAllStandardOutput() in a QMessageBox (and don't forget to properly encode it into a QString with QString::fromLocal8Bit())

  7. #7
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,536
    Thanked 284 Times in 279 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    What You see in file result.txt when You try this from command line :
    Qt Code:
    1. D:\YTDownloader\youtube-dl.exe -e --no-playlist https://www.youtube.com/watch?v=6V-wwfuxZxw >>result.txt
    To copy to clipboard, switch view to plain text mode 

  8. #8
    Join Date
    Jul 2019
    Posts
    32
    Thanks
    5
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    Quote Originally Posted by ChristianEhrlicher View Post
    Where did I told you to do so? I said you should print the return value from readAllStandardOutput() in a QMessageBox (and don't forget to properly encode it into a QString with QString::fromLocal8Bit())
    Ok thank you, but I get the same result.

    Quote Originally Posted by Lesiok View Post
    What You see in file result.txt when You try this from command line :
    Qt Code:
    1. D:\YTDownloader\youtube-dl.exe -e --no-playlist https://www.youtube.com/watch?v=6V-wwfuxZxw >>result.txt
    To copy to clipboard, switch view to plain text mode 
    I get the same result. For example, if I have title "Petric? Moise - ??tia-s b?n??enii mei" in the txt file I get "Petric Moise - tia-s bnenii mei".

  9. #9
    Join Date
    Jan 2008
    Location
    Alameda, CA, USA
    Posts
    5,248
    Thanks
    304
    Thanked 868 Times in 855 Posts
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    I get the same result.
    Then maybe your "youtube-dl" program itself does not support Unicode.
    <=== The Great Pumpkin says ===>
    Please use CODE tags when posting source code so it is more readable. Click "Go Advanced" and then the "#" icon to insert the tags. Paste your code between them.

  10. #10
    Join Date
    Jul 2019
    Posts
    32
    Thanks
    5
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    Quote Originally Posted by d_stranz View Post
    Then maybe your "youtube-dl" program itself does not support Unicode.
    Hmmm, curious because when I run the command in cmd I get the correct title with all characters.

  11. #11
    Join Date
    Jan 2006
    Location
    Bremen, Germany
    Posts
    554
    Thanked 86 Times in 81 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    Can you please show us your code?

  12. #12
    Join Date
    Jul 2019
    Posts
    32
    Thanks
    5
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    Quote Originally Posted by ChristianEhrlicher View Post
    Can you please show us your code?
    Yes, of course.

    mainwindow.h
    Qt Code:
    1. #ifndef MAINWINDOW_H
    2. #define MAINWINDOW_H
    3.  
    4. #include <QWidget>
    5. #include <QProcess>
    6. #include <QString>
    7. #include <QLabel>
    8.  
    9. class mainWindow : public QWidget
    10. {
    11. Q_OBJECT
    12.  
    13. public:
    14. mainWindow(QWidget *parent = nullptr);
    15. ~mainWindow();
    16. void createUI();
    17. void process();
    18. QProcess *process1 = new QProcess(this);
    19. QString processStdout;
    20. QLabel *label;
    21.  
    22. private slots:
    23. void ReadOutput(int, QProcess::ExitStatus);
    24.  
    25. };
    26. #endif // MAINWINDOW_H
    To copy to clipboard, switch view to plain text mode 

    mainwindow.cpp
    Qt Code:
    1. #include "mainwindow.h"
    2. #include <QPushButton>
    3. #include <QtDebug>
    4. #include <QTextCodec>
    5.  
    6. mainWindow::mainWindow(QWidget *parent) : QWidget(parent)
    7. {
    8. createUI();
    9. connect(process1, SIGNAL(finished(int, QProcess::ExitStatus)), this, SLOT(ReadOutput(int, QProcess::ExitStatus)));
    10. }
    11.  
    12. mainWindow::~mainWindow()
    13. {
    14.  
    15. }
    16. void mainWindow::createUI(){
    17. label = new QLabel(this);
    18. label->setGeometry(100, 200, 100, 30);
    19.  
    20. QPushButton *buttonsearch = new QPushButton("Start process", this);
    21. buttonsearch->setToolTip("Start process");
    22. buttonsearch->setGeometry(200, 290, 100, 30);
    23. connect(buttonsearch, &QPushButton::clicked, [this]() {process(); });
    24. }
    25.  
    26. void mainWindow::process(){
    27. process1->setProcessChannelMode(QProcess::MergedChannels);
    28. process1->start("\"D:\\YTDownloader\\youtube-dl.exe\" -e --no-playlist https://www.youtube.com/watch?v=6V-wwfuxZxw");
    29. }
    30.  
    31. void mainWindow::ReadOutput(int exitCode, QProcess::ExitStatus exitStatus){
    32. qDebug() << exitCode;
    33. qDebug() << exitStatus;
    34. QByteArray a = process1->readAllStandardOutput();
    35. qDebug() << a;
    36. QTextCodec* utfCodec = QTextCodec::codecForName("UTF-8");
    37. processStdout = utfCodec->toUnicode(a);
    38. qDebug() << processStdout;
    39. label->setText(processStdout);
    40. }
    To copy to clipboard, switch view to plain text mode 

    main.cpp
    Qt Code:
    1. #include <QApplication>
    2. #include "mainwindow.h"
    3.  
    4. int main(int argl,char *argv[])
    5. {
    6. QApplication app(argl,argv);
    7.  
    8. mainWindow *window = new mainWindow();
    9. window->setWindowTitle("Test");
    10. window->setFixedSize(700, 335);
    11. window->show();
    12.  
    13. return app.exec();
    14. }
    To copy to clipboard, switch view to plain text mode 

  13. #13
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,536
    Thanked 284 Times in 279 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    Perhaps the problem is the Windows code page. I did the test with "Polish letters" and everything is OK but the result from youtube-dl is encoded in 1250 (8-bit code page) and not UTF-8. This is the link used for the test: https://www.youtube.com/watch?v=ndG8bM-9CMA
    Can you provide the Japanese alphabet link to test?
    P.S.
    Test with arabic alphabet : https://www.youtube.com/watch?v=7Tc7LWpe7mg. Output generated on Windows is 29 bytes long, on Linux 85 bytes long. In my opinion it's an internal problem of youtube-dl.
    Last edited by Lesiok; 19th August 2019 at 13:08.

  14. #14
    Join Date
    Jul 2019
    Posts
    32
    Thanks
    5
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    Quote Originally Posted by Lesiok View Post
    Perhaps the problem is the Windows code page. I did the test with "Polish letters" and everything is OK but the result from youtube-dl is encoded in 1250 (8-bit code page) and not UTF-8. This is the link used for the test: https://www.youtube.com/watch?v=ndG8bM-9CMA
    Can you provide the Japanese alphabet link to test?
    P.S.
    Test with arabic alphabet : https://www.youtube.com/watch?v=7Tc7LWpe7mg. Output generated on Windows is 29 bytes long, on Linux 85 bytes long. In my opinion it's an internal problem of youtube-dl.
    Link with Japanese Alphabet: https://www.youtube.com/watch?v=Bsfi4XbPE8M. I tried your link both with cmd and then with QProcess. When I tried the command in cmd with arabic alphabet link I got something like this ????? ??????? ?????? _35_ ????? ?????? The Ultimate Ninj, but if I copy paste the output(the title) in a browser I get the correct title.

    When I tried with QProcess I got something like this: " _35_ The Ultimate Ninj". Maybe it's a problem with youtube-dl, I don't know.

    I attached a photo to see how title looks after I copy-paste it in a browser from cmd.
    Attached Images Attached Images
    Last edited by INeedADollar; 19th August 2019 at 13:40.

  15. #15
    Join Date
    Mar 2008
    Location
    Kraków, Poland
    Posts
    1,536
    Thanked 284 Times in 279 Posts
    Qt products
    Qt4
    Platforms
    Unix/X11 Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    definitely an environmental problem (operating system). On linux in a graphical environment (fonts installed for all alphabets) this result:
    gc.PNG
    On linux in a character environment like this:
    tc.jpg

  16. #16
    Join Date
    Jul 2019
    Posts
    32
    Thanks
    5
    Qt products
    Qt5
    Platforms
    Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    Quote Originally Posted by Lesiok View Post
    definitely an environmental problem (operating system). On linux in a graphical environment (fonts installed for all alphabets) this result:
    gc.PNG
    On linux in a character environment like this:
    tc.jpg
    Ok thank you very much! Thank you very much all that tried to help me!

  17. #17
    Join Date
    Jan 2006
    Location
    Bremen, Germany
    Posts
    554
    Thanked 86 Times in 81 Posts
    Qt products
    Qt3 Qt4
    Platforms
    Unix/X11 Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    You're still outputting the data to the command line instead using a QMessageBox as I said in my second post so it can't work... https://www.qtcentre.org/threads/704...088#post306088

  18. #18
    Join Date
    Jan 2006
    Location
    Graz, Austria
    Posts
    8,416
    Thanks
    37
    Thanked 1,544 Times in 1,494 Posts
    Qt products
    Qt3 Qt4 Qt5
    Platforms
    Unix/X11 Windows

    Default Re: ReadAllStandardOutput() is returning empty characters if they are not recognised

    If it is a problem with the normal output of the Windows version of youtube-dl you could try its JSON output options.

    Cheers,
    _

Similar Threads

  1. Replies: 10
    Last Post: 6th February 2015, 13:41
  2. Replies: 13
    Last Post: 27th August 2014, 12:06
  3. QProcess::readAllStandardOutput and QTextEdit
    By naptizaN in forum Qt Programming
    Replies: 1
    Last Post: 13th December 2012, 08:50
  4. Returning const &
    By frenk_castle in forum Newbie
    Replies: 2
    Last Post: 24th November 2009, 08:01
  5. remove directory empty or not empty
    By raphaelf in forum Newbie
    Replies: 12
    Last Post: 27th October 2006, 08:30

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
Digia, Qt and their respective logos are trademarks of Digia Plc in Finland and/or other countries worldwide.