Rendering a QQuickItem into a QImage (Qt6)

[Bilbon]

Hi Qt Specialist,

I'm really desperate to make it work, I try different approaches but nothing works like expected. And most of the examples found by Googling concern Qt5.

First the Use Case: I have to do a color picker into a complex 2D scene, this scene is constructed with QSGNodes tree inside a QQuickItem. Of course, to make it more difficult, I have to pick the color without the drawing overlays and I should be able to pick outside of the item bounds.

I try this:
1. I create a simplified QQuickItem
2. try to bind it to an offscreen window (setParentItem & QQuickRenderControl)
3. add a render target (fromPaintDevice, I have no clue how to transform any 'kind' of textures into a QImage)
3. resize
4. render with polishItems, beginFrame, ....
5. check the image

I get this error:
QQuickWindow: No render target (neither swapchain nor custom target was provided)

Here is the code using Vulkan:

QVulkanInstance* inst = new QVulkanInstance;
    inst->create();
 
    _render_ctrl = new QQuickRenderControl;
 
    // this window will never be shown on-screen
    _off_screen_window = new QQuickWindow(_render_ctrl);
    _off_screen_window->setVulkanInstance(inst);
 
    if (!_render_ctrl->initialize())
        qWarning() << "failed to init";
 
    _image = new QImage(300, 200, QImage::Format_RGBA8888_Premultiplied);
    _image->fill(qRgb(255,255,0));
 
    _off_screen_window->setRenderTarget(QQuickRenderTarget::fromPaintDevice(_image));
 
    _off_screen_item->setParentItem(_off_screen_window->contentItem());
    _off_screen_item->setSize(QSize(300,200));
    _off_screen_item->setVisible(true);
 
    _off_screen_window->resize(300, 200);
    //_off_screen_window->show();
 
    _render_ctrl->polishItems();       
    _render_ctrl->beginFrame();
    _render_ctrl->sync();
    _render_ctrl->render();
    _render_ctrl->endFrame();
 
    _image->save("xxx.png"); // nothing draw just the original image

QVulkanInstance* inst = new QVulkanInstance;
    inst->create();

    _render_ctrl = new QQuickRenderControl;

    // this window will never be shown on-screen
    _off_screen_window = new QQuickWindow(_render_ctrl);
    _off_screen_window->setVulkanInstance(inst);

    if (!_render_ctrl->initialize())
        qWarning() << "failed to init";

    _image = new QImage(300, 200, QImage::Format_RGBA8888_Premultiplied);
    _image->fill(qRgb(255,255,0));

    _off_screen_window->setRenderTarget(QQuickRenderTarget::fromPaintDevice(_image));
    
    _off_screen_item->setParentItem(_off_screen_window->contentItem());
    _off_screen_item->setSize(QSize(300,200));
    _off_screen_item->setVisible(true);

    _off_screen_window->resize(300, 200);
    //_off_screen_window->show();

    _render_ctrl->polishItems();       
    _render_ctrl->beginFrame();
    _render_ctrl->sync();
    _render_ctrl->render();
    _render_ctrl->endFrame();

    _image->save("xxx.png"); // nothing draw just the original image

To copy to clipboard, switch view to plain text mode 

Could someone explain to me how in Qt6 I could render offscreen a QQuickItem in a QImage? Or at least into a texture and how to obtain an average color from the extracted texture?

Is the call QQuickWindow::setGraphicsApi(QSGRendererInterface: :Vulkan); compatible with the offscreen QQuickWindow? The Qt examples and documentation do not really cover this case.

The only solution, for now, I'm using (slow - no shaders to help - and need to duplicate the rendering) is the QPainter over a QImage.

Br,
Bilbon

[wysota]

The docs claim you could redirect rendering in Vulkan to VkImage:

QQuickRenderTarget rt = QQuickRenderTarget::fromVulkanImage(vulkanImage, VK_IMAGE_LAYOUT_PREINITIALIZED, pixelSize);
quickWindow->setRenderTarget(rt);

QQuickRenderTarget rt = QQuickRenderTarget::fromVulkanImage(vulkanImage, VK_IMAGE_LAYOUT_PREINITIALIZED, pixelSize);
quickWindow->setRenderTarget(rt);

To copy to clipboard, switch view to plain text mode 

Then you can either sample that image or try to convert it to some other image API (e.g. QImage)

[Steboirm]

Rendering a QQuickItem into a QImage in Qt6 can be achieved by redirecting rendering in Vulkan to a VkImage using QQuickRenderTarget. Have you tried using the suggested approach with QQuickRenderTarget::fromVulkanImage, and if so, what issues are you encountering in your implementation?

Additionally, have you considered other methods or libraries to achieve your color picking in a complex 2D scene, or are you focused on solving this within the Qt framework?