ReadAllStandardOutput() is returning empty characters if they are not recognised

I have a QProcess that runs a youtube-dl command to get the title of videos. I read the output of the process when it's finished and if the title of the video has some japanese characters for example, readAllStandardOutput() is not returning them. For example, if I have title "Learn Hiragana *some japanese characters here* (Japanese alphabet)" the readAllStandardOutput() function is returning "Learn Hiragana (Japanese alphabet)".

How can I solve this?

How do you display the text? Since readAllStandardOutput() returns a QByteArray you have to convert it properly to a QString with QString::fromLocal8Bit()

Quote:

---

Originally Posted by ChristianEhrlicher

How do you display the text? Since readAllStandardOutput() returns a QByteArray you have to convert it properly to a QString with QString::fromLocal8Bit()

---

I tried using QString::fromLocal8Bit() but it's the same because I'm printing the QByteArray as it is.
My code:

mainwindow.h

Code:

---

#ifndef MAINWINDOW_H
#define MAINWINDOW_H
 
#include <QWidget>
class mainWindow : public QWidget
{
    Q_OBJECT
    //here are defined some variables
 public:
    mainWindow(QWidget *parent = 0);
    ~mainWindow();
    void createUI();
    void process();
    QProcess *process = new QProcess(this);
 
private slots:
    void ReadOutput(int, QProcess::ExitStatus);
 
};
#endif // MAINWINDOW_H

---

mainwindow.cpp

Code:

---

#include <QWidget>
#include <QPushButton>
#include <QProcess>
#include <QByteArray>
#include <QTextCodec>
#include <QString>
#include <QDebug>
#include "mainwindow.h"
 
mainWindow::mainWindow(QWidget *parent) : QWidget(parent)
{
    createUI();
    connect(process, SIGNAL(finished(int, QProcess::ExitStatus)), this, SLOT(ReadOutput(int, QProcess::ExitStatus)));
}
 
 
mainWindow::~mainWindow()
{
 
}
void mainWindow::createUI(){
    //here I create the look of the window
     QPushButton buttonsearch = new QPushButton("Start process", this);
     buttonsearch->setToolTip("Start process");
     buttonsearch->setGeometry(200, 290, 100, 30);
     connect(buttonsearch, &QPushButton::clicked, [this]() {process(); });
}
 
void mainWindow::process(){
    process->setProcessChannelMode(QProcess::MergedChannels);
    process->start("\"D:\\YTDownloader\\youtube-dl.exe\" -e --no-playlist https://www.youtube.com/watch?v=6V-wwfuxZxw");
 
void readOutput(int exitCode, QProcess::ExitStatus exitStatus){
    qDebug() << exitCode;
    qDebug() << exitStatus;
    QByteArray a = process->readAllStandardOutput();
    qDebug() << a; //here is not showing characters that are not recognised
    QTextCodec* utfCodec = QTextCodec::codecForName("UTF-8");
    processStdout = utfCodec->toUnicode(a);
    qDebug() << processStdout;
}

---

main.cpp

Code:

---

#include <QApplication>
#include "mainwindow.h"
 
int main(int argl,char *argv[])
{
    QApplication app(argl,argv);
 
    mainWindow *window = new mainWindow();
    window->setWindowTitle("Test");
    window->setFixedSize(700, 335);
    window->show();
 
    return app.exec();
}

---

Don't rely on windows' cmd.exe - it can't handle unicode properly. Use a QMessageBox or similar.

Quote:

---

Originally Posted by ChristianEhrlicher

Don't rely on windows' cmd.exe - it can't handle unicode properly. Use a QMessageBox or similar.

---

Sorry but I don't understand what you mean. How to use a QMessageBox to run a QProcess?

Quote:

---

Originally Posted by INeedADollar

Sorry but I don't understand what you mean. How to use a QMessageBox to run a QProcess?

---

Where did I told you to do so? I said you should print the return value from readAllStandardOutput() in a QMessageBox (and don't forget to properly encode it into a QString with QString::fromLocal8Bit())

What You see in file result.txt when You try this from command line :

Code:

---

D:\YTDownloader\youtube-dl.exe -e --no-playlist https://www.youtube.com/watch?v=6V-wwfuxZxw >>result.txt

---

Quote:

---

Originally Posted by ChristianEhrlicher

Where did I told you to do so? I said you should print the return value from readAllStandardOutput() in a QMessageBox (and don't forget to properly encode it into a QString with QString::fromLocal8Bit())

---

Ok thank you, but I get the same result.

Quote:

---

Originally Posted by Lesiok

What You see in file result.txt when You try this from command line :

Code:

---

D:\YTDownloader\youtube-dl.exe -e --no-playlist https://www.youtube.com/watch?v=6V-wwfuxZxw >>result.txt

---

---

I get the same result. For example, if I have title "Petric? Moise - ??tia-s b?n??enii mei" in the txt file I get "Petric Moise - tia-s bnenii mei".

Quote:

---

I get the same result.

---

Then maybe your "youtube-dl" program itself does not support Unicode.

Quote:

---

Originally Posted by d_stranz

Then maybe your "youtube-dl" program itself does not support Unicode.

---

Hmmm, curious because when I run the command in cmd I get the correct title with all characters.

Can you please show us your code?

Quote:

---

Originally Posted by ChristianEhrlicher

Can you please show us your code?

---

Yes, of course.

mainwindow.h

Code:

---

#ifndef MAINWINDOW_H
#define MAINWINDOW_H
 
#include <QWidget>
#include <QProcess>
#include <QString>
#include <QLabel>
 
class mainWindow : public QWidget
{
    Q_OBJECT
 
 public:
    mainWindow(QWidget *parent = nullptr);
    ~mainWindow();
    void createUI();
    void process();
    QProcess *process1 = new QProcess(this);
    QString processStdout;
    QLabel *label;
 
private slots:
    void ReadOutput(int, QProcess::ExitStatus);
 
};
#endif // MAINWINDOW_H

---

mainwindow.cpp

Code:

---

#include "mainwindow.h"
#include <QPushButton>
#include <QtDebug>
#include <QTextCodec>
 
mainWindow::mainWindow(QWidget *parent) : QWidget(parent)
{
    createUI();
    connect(process1, SIGNAL(finished(int, QProcess::ExitStatus)), this, SLOT(ReadOutput(int, QProcess::ExitStatus)));
}
 
mainWindow::~mainWindow()
{
 
}
void mainWindow::createUI(){
     label = new QLabel(this);
     label->setGeometry(100, 200, 100, 30);
 
     QPushButton *buttonsearch = new QPushButton("Start process", this);
     buttonsearch->setToolTip("Start process");
     buttonsearch->setGeometry(200, 290, 100, 30);
     connect(buttonsearch, &QPushButton::clicked, [this]() {process(); });
}
 
void mainWindow::process(){
    process1->setProcessChannelMode(QProcess::MergedChannels);
    process1->start("\"D:\\YTDownloader\\youtube-dl.exe\" -e --no-playlist https://www.youtube.com/watch?v=6V-wwfuxZxw");
}
 
void mainWindow::ReadOutput(int exitCode, QProcess::ExitStatus exitStatus){
    qDebug() << exitCode;
    qDebug() << exitStatus;
    QByteArray a = process1->readAllStandardOutput();
    qDebug() << a; 
    QTextCodec* utfCodec = QTextCodec::codecForName("UTF-8");
    processStdout = utfCodec->toUnicode(a);
    qDebug() << processStdout;
    label->setText(processStdout);
}

---

main.cpp

Code:

---

#include <QApplication>
#include "mainwindow.h"
 
int main(int argl,char *argv[])
{
    QApplication app(argl,argv);
 
    mainWindow *window = new mainWindow();
    window->setWindowTitle("Test");
    window->setFixedSize(700, 335);
    window->show();
 
    return app.exec();
}

---

Perhaps the problem is the Windows code page. I did the test with "Polish letters" and everything is OK but the result from youtube-dl is encoded in 1250 (8-bit code page) and not UTF-8. This is the link used for the test: https://www.youtube.com/watch?v=ndG8bM-9CMA
Can you provide the Japanese alphabet link to test?
P.S.
Test with arabic alphabet : https://www.youtube.com/watch?v=7Tc7LWpe7mg. Output generated on Windows is 29 bytes long, on Linux 85 bytes long. In my opinion it's an internal problem of youtube-dl.

Quote:

---

Originally Posted by Lesiok

Perhaps the problem is the Windows code page. I did the test with "Polish letters" and everything is OK but the result from youtube-dl is encoded in 1250 (8-bit code page) and not UTF-8. This is the link used for the test: https://www.youtube.com/watch?v=ndG8bM-9CMA
Can you provide the Japanese alphabet link to test?
P.S.
Test with arabic alphabet : https://www.youtube.com/watch?v=7Tc7LWpe7mg. Output generated on Windows is 29 bytes long, on Linux 85 bytes long. In my opinion it's an internal problem of youtube-dl.

---

Link with Japanese Alphabet: https://www.youtube.com/watch?v=Bsfi4XbPE8M. I tried your link both with cmd and then with QProcess. When I tried the command in cmd with arabic alphabet link I got something like this ????? ??????? ?????? _35_ ????? ?????? The Ultimate Ninj, but if I copy paste the output(the title) in a browser I get the correct title.

When I tried with QProcess I got something like this: " _35_ The Ultimate Ninj". Maybe it's a problem with youtube-dl, I don't know.

I attached a photo to see how title looks after I copy-paste it in a browser from cmd.

definitely an environmental problem (operating system). On linux in a graphical environment (fonts installed for all alphabets) this result:
Attachment 13227
On linux in a character environment like this:
Attachment 13228

Quote:

---

Originally Posted by Lesiok

definitely an environmental problem (operating system). On linux in a graphical environment (fonts installed for all alphabets) this result:
Attachment 13227
On linux in a character environment like this:
Attachment 13228

---

Ok thank you very much! Thank you very much all that tried to help me!

You're still outputting the data to the command line instead using a QMessageBox as I said in my second post so it can't work... https://www.qtcentre.org/threads/704...088#post306088

If it is a problem with the normal output of the Windows version of youtube-dl you could try its JSON output options.

Cheers,
_