efficient way to display opencv image into Qt

[gerardpuducul]

Hi all,

I had develop an application to record and play rtp stream or local file using opencv.

My opencv process run into seperate thread and I use opencv function to display the video :

void opencv_thread::play_video(VideoCapture capture)
{
    emit write_LOG(0,QTime::currentTime(),"playing " );
    stop_video_flag = false;
 
    frame_index = 0;
#ifdef DISPAY_by_OPENCV
    namedWindow("Player",CV_WINDOW_AUTOSIZE);
#endif
 
    /* Loop until frame ended or ESC is pressed */
    while(1)
    {
        /* grab frame image, and retrieve */
        Mat frame;
        capture >> frame; // get a new frame from camera
 
        if (frame.empty())
            break;
 
        Mat frame_flipped;
 
        if (image_flip_is_Activate == true)
        {
            cv::flip(frame,frame_flipped,1);
            frame= frame_flipped;
        }
#ifdef DISPAY_by_OPENCV
        imshow("Player", frame);
#endif
        emit Qimage_available(MatToQImage(frame));
#endif
 
        Mat src_gray;
        cvtColor( frame, src_gray, CV_BGR2GRAY );
 
        if (flag_gray == true)
        {
            display_gray(src_gray);
        }
        if (flag_histogram == true)
        {
            histogram(frame);
        }
        if (flag_edge == true)
        {
            edge(frame,src_gray);
        }
        if (flag_face_detect == true)
        {
 
            face_detect(frame,src_gray);
        }
 
        /* if ESC is pressed then exit loop */
        char c = cvWaitKey(33);
        if(c==27)
            break;
        if ((stop_video_flag == true) || (flag_pause == true))
            break;
 
        capture_current_frame = capture.get(CV_CAP_PROP_POS_FRAMES);
 
        int capture_number_of_frame = capture.get(CV_CAP_PROP_FRAME_COUNT);
        emit update_current_frame(capture_current_frame,capture_number_of_frame);
 
        frame_index ++ ;
    }
 
    /* destroy pointer to video */
    capture.release();
 
}

void opencv_thread::play_video(VideoCapture capture)
{
    emit write_LOG(0,QTime::currentTime(),"playing " );
    stop_video_flag = false;
    
    frame_index = 0;
#ifdef DISPAY_by_OPENCV
    namedWindow("Player",CV_WINDOW_AUTOSIZE);
#endif

    /* Loop until frame ended or ESC is pressed */
    while(1)
    {
        /* grab frame image, and retrieve */
        Mat frame;
        capture >> frame; // get a new frame from camera

        if (frame.empty())
            break;

        Mat frame_flipped;

        if (image_flip_is_Activate == true)
        {
            cv::flip(frame,frame_flipped,1);
            frame= frame_flipped;
        }
#ifdef DISPAY_by_OPENCV
        imshow("Player", frame);
#endif
        emit Qimage_available(MatToQImage(frame));
#endif

        Mat src_gray;
        cvtColor( frame, src_gray, CV_BGR2GRAY );

        if (flag_gray == true)
        {
            display_gray(src_gray);
        }
        if (flag_histogram == true)
        {
            histogram(frame);
        }
        if (flag_edge == true)
        {
            edge(frame,src_gray);
        }
        if (flag_face_detect == true)
        {

            face_detect(frame,src_gray);
        }

        /* if ESC is pressed then exit loop */
        char c = cvWaitKey(33);
        if(c==27)
            break;
        if ((stop_video_flag == true) || (flag_pause == true))
            break;

        capture_current_frame = capture.get(CV_CAP_PROP_POS_FRAMES);

        int capture_number_of_frame = capture.get(CV_CAP_PROP_FRAME_COUNT);
        emit update_current_frame(capture_current_frame,capture_number_of_frame);

        frame_index ++ ;
    }

    /* destroy pointer to video */
    capture.release();

}

To copy to clipboard, switch view to plain text mode 

and everything work fine (not to much CPU load)............................................. except that I display video into a opencv and I would like to display it into Qt widget.

So I try to convert Mat image from opencv to Qimage and emit a signal every time an image is available:

QImage opencv_thread::MatToQImage(const Mat& mat)
{
    // 8-bits unsigned, NO. OF CHANNELS=1
    if(mat.type()==CV_8UC1)
    {
        // Set the color table (used to translate colour indexes to qRgb values)
        QVector<QRgb> colorTable;
        for (int i=0; i<256; i++)
            colorTable.push_back(qRgb(i,i,i));
        // Copy input Mat
        const uchar *qImageBuffer = (const uchar*)mat.data;
        // Create QImage with same dimensions as input Mat
        QImage img(qImageBuffer, mat.cols, mat.rows, mat.step, QImage::Format_Indexed8);
        img.setColorTable(colorTable);
        return img;
    }
    // 8-bits unsigned, NO. OF CHANNELS=3
    else if(mat.type()==CV_8UC3)
    {
        // Copy input Mat
        const uchar *qImageBuffer = (const uchar*)mat.data;
        // Create QImage with same dimensions as input Mat
        QImage img(qImageBuffer, mat.cols, mat.rows, mat.step, QImage::Format_RGB888);
        return img.rgbSwapped();
    }
    else
    {
//        qDebug() << "ERROR: Mat could not be converted to QImage.";
        return QImage();
    }
}

QImage opencv_thread::MatToQImage(const Mat& mat)
{
    // 8-bits unsigned, NO. OF CHANNELS=1
    if(mat.type()==CV_8UC1)
    {
        // Set the color table (used to translate colour indexes to qRgb values)
        QVector<QRgb> colorTable;
        for (int i=0; i<256; i++)
            colorTable.push_back(qRgb(i,i,i));
        // Copy input Mat
        const uchar *qImageBuffer = (const uchar*)mat.data;
        // Create QImage with same dimensions as input Mat
        QImage img(qImageBuffer, mat.cols, mat.rows, mat.step, QImage::Format_Indexed8);
        img.setColorTable(colorTable);
        return img;
    }
    // 8-bits unsigned, NO. OF CHANNELS=3
    else if(mat.type()==CV_8UC3)
    {
        // Copy input Mat
        const uchar *qImageBuffer = (const uchar*)mat.data;
        // Create QImage with same dimensions as input Mat
        QImage img(qImageBuffer, mat.cols, mat.rows, mat.step, QImage::Format_RGB888);
        return img.rgbSwapped();
    }
    else
    {
//        qDebug() << "ERROR: Mat could not be converted to QImage.";
        return QImage();
    }
}

To copy to clipboard, switch view to plain text mode 

This work pretty well for low resolution and frame rate. But when resolution/frame rate increase , CPU load increase too. ( For example with the video example form windows 7 Wildlife.wmv (1280x720@30fps) : With open_cv display CPU load = 10% ,With Qt display CPU load = 48% )

Is there a efficiently way to display an opencv image into QT?

Thanks

[gerardpuducul]

No one?

I am very surprise that no one has done this kind of experience!

No idea?

thanks

[stampede]

Use OpenGl display window. Then you can load your image with glTexImage2D(...) without any conversion to QImage.
Google for some code examples.

[kachhadiyatushar]

Hello Guys,

I have a Qt application in which I have used opengl to display opencv video on qt central widget in windows machine but now I have ported this application into raspberry pi and while compiling this app I am getting error like "glRaster2i, glOrtho, glDrawpixels not declared in this scope" if anybody have any idea regarding this error then please let me know. I will appreciate your help.

Thanks

Tushar Kachhadiya

[d_stranz]

"glRaster2i, glOrtho, glDrawpixels not declared in this scope"

Usually that means you haven't included one or more header files. Figure out the correct header file where these symbols are defined and #include it. Then you can post your next question, which will be how to solve the link error about these unresolved symbols.

[kachhadiyatushar]

Hi,

I have included all required header files. If I compiled my application without these function it was compiling successful and runs in raspberry pi but when I opened camera on Qt QLabel on click of push button the Mainwindow was hanged. Then I must need to restart os to bring back to home screen.

Thanks,

Tushar Kachhadiya