Render qwidgets on 3d scene

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
ickby
Posts: 2927
Joined: Wed Oct 05, 2011 7:36 am

Render qwidgets on 3d scene

Postby ickby » Thu May 22, 2014 6:14 am

Hello Werner,

over the last week I experimented with drawing QWidgets ontop of the 3d scene with transparent background. I noticed from some unused code that you tried this before. The problem, as you know, is that you can not simply add child widgets to the 3d widget as the 3d content is unknown to the qt render system and therefore all child widgets get some weird background. If you bring in transperency you can't see "through" the child to the 3d scene but only see gibberish.

I found 2 ways to make this working, but before going any further with a real implementation I like to hear your opinion about the topic itself and what would be a sensible approach. Let me explain both methods:

1. Draw the widgets inside the 3d scene
---------------------------------------------------

it is possible to Initialise a QPainter on the openGL context and use it to render all qt stuff with opengl commands. This makes it possible to render arbitrary QWidgets directly into the 3d scene from within the current render code View3dInventorViewer::renderScene(). (pseudocode as I don't have acces to my code right now):

Code: Select all

glPushAttrip(Atr_all)
QPainter painter(getGlWidget());
my3dwidget->render(painter);
painter.end();
glPopAttrip()
Advantages:
- Transperency and all works very nicely.
- You can draw anything qt can draw directly into the scene, basicly no adjustment of freecad render code needed

Disadvantage:
- As the widget is hidden it won't get any events from qt. So to make it behave as normal you need to forward all events. I made this work for mouse events so that you can use the widgets, but it is impossible to get updates from within the widget. It all feels a bit flaky
- every widget change need a full scene redraw inclusing a redraw of every other widget added to the scene, which makes the widgets slow for big scenes
- every scene update need a full widget redraw, which makes the scene update a bit slower

2. Make the 3d content known to qt drawing system
-------------------------------------------------------------------
If we want standart qt parent-child relationship to get all evetn handling and stuff one need to make the 3d content known to qt. I made this work by rendering the 3d scene to a framebuffer (already supported by freecad) and then draw this framebuffer into a qwidget. Than you can add childs to this widget and transperency and stuff works nicely.

Advantage:
- Transperency and all works very nicely.
- Full qt event handling
- QT update optimisations: Only the updated widget gets redrawn, not the whole scene or any other widget
- Widgets stay responsive even for very big scenes
- rednering the scene does not need to render widgets

Disadvantage:
- Needs quite intrusive changes to freecad render scene (as the normal render code would be obsolede, only the framebuffer code would be needed)
- rendering to framebuffer may be slower or have some disadvantages not know yet as it is not widly used by freecad



In my opinion method two is more promising, but maybe too intrusive? Or is this whole conept of overlay widgets something you would rather not add to freecad? I would realy love to have the possibility to use qidgets ontop of the 3d scene some day, so I look forward to hear your opinion.

Stefan
wmayer
Site Admin
Posts: 15000
Joined: Thu Feb 19, 2009 10:32 am

Re: Render qwidgets on 3d scene

Postby wmayer » Thu May 22, 2014 10:22 am

Hello Stefan,
over the last week I experimented with drawing QWidgets ontop of the 3d scene with transparent background. I noticed from some unused code that you tried this before. The problem, as you know, is that you can not simply add child widgets to the 3d widget as the 3d content is unknown to the qt render system and therefore all child widgets get some weird background. If you bring in transperency you can't see "through" the child to the 3d scene but only see gibberish.
IMO, the main problem is that SoQt decouples the rendering of a 3d scene (i.e. the method actualRedraw()) from Qt's rendering mechanism (i.e. paintEvent). Somewhere in an SoQt widget the paintEvent() method is implemented to just start a timer which makes the OpenGL rendering asynchronous to Qt.
1. Draw the widgets inside the 3d scene
In the past I tried many things but was never happy with any of the "solutions". Also it's quite unclear if this is really platform-independent.
2. Make the 3d content known to qt drawing system
Disadvantage:
- Needs quite intrusive changes to freecad render scene (as the normal render code would be obsolede, only the framebuffer code would be needed)
- rendering to framebuffer may be slower or have some disadvantages not know yet as it is not widly used by freecad
I wonder whether selection in the 3d scene is possible with framebuffer. The idea of using framebuffer is that you can achieve some kind of overlay painting where you render the static scene and then paint some additional stuff on top of it. This is e.g. already used for our rubberband. But the selection in the 3d scene is a different thing because here you need the 3d effect of a highlighted or selected face.

Some time ago I also followed this approach were I used the Qt classes QGraphicsView and QGraphicsScene because they already support to have Qt widgets inside a graphic scene.
In my opinion method two is more promising, but maybe too intrusive? Or is this whole conept of overlay widgets something you would rather not add to freecad? I would realy love to have the possibility to use qidgets ontop of the 3d scene some day, so I look forward to hear your opinion.
It would definitely be a nice thing to support overlay widgets. If you e.g. look at the API documentation of TGS Inventor they have a lot of special widget nodes where they can render primitive widgets. But I do not know if they have to handle the event mechanism and the rendering themselves or if they can use Qt's mechanisms.
ickby
Posts: 2927
Joined: Wed Oct 05, 2011 7:36 am

Re: Render qwidgets on 3d scene

Postby ickby » Fri May 23, 2014 8:41 am

Hello,

thanks for the feedback.
IMO, the main problem is that SoQt decouples the rendering of a 3d scene (i.e. the method actualRedraw()) from Qt's rendering mechanism (i.e. paintEvent). Somewhere in an SoQt widget the paintEvent() method is implemented to just start a timer which makes the OpenGL rendering asynchronous to Qt.
I think this is a generla problem, when you search for this then you will notice that this also happens if one draws with opengl calls inside of the glwidgets paintEvent, e.g. http://stackoverflow.com/questions/1160 ... ge-results
In the past I tried many things but was never happy with any of the "solutions". Also it's quite unclear if this is really platform-independent.
Platform independencie is a good point. In this regard a less intrusive method is definitly prefered to enable some testing without the possibility of a completly failing 3d view
I wonder whether selection in the 3d scene is possible with framebuffer. The idea of using framebuffer is that you can achieve some kind of overlay painting where you render the static scene and then paint some additional stuff on top of it. This is e.g. already used for our rubberband. But the selection in the 3d scene is a different thing because here you need the 3d effect of a highlighted or selected face.
Yes it is when you render every scene redraw into the framebuffer. So don't use it as static scene as it is used now in freecad but as dynamic render target. You can pass all events to coin as it is now and on redraw the framebuffer gets rerendered with the selection highlighted.
Some time ago I also followed this approach were I used the Qt classes QGraphicsView and QGraphicsScene because they already support to have Qt widgets inside a graphic scene.
I tried that too, but it was a mess, I never got the 3d content displayed in a acceptable manner. Therefore I did not put much effort in this approach
It would definitely be a nice thing to support overlay widgets. If you e.g. look at the API documentation of TGS Inventor they have a lot of special widget nodes where they can render primitive widgets. But I do not know if they have to handle the event mechanism and the rendering themselves or if they can use Qt's mechanisms.
Ah interesting. But then they are very simple. My goal with QWidgets is for example to have the tree view inside the scene. It would be awesome to use existing widgets withour recoding anything.


I will put up some test branches over the next time, maybe something of interst will come up!
wmayer
Site Admin
Posts: 15000
Joined: Thu Feb 19, 2009 10:32 am

Re: Render qwidgets on 3d scene

Postby wmayer » Fri May 23, 2014 9:31 am

I think this is a generla problem, when you search for this then you will notice that this also happens if one draws with opengl calls inside of the glwidgets paintEvent, e.g. http://stackoverflow.com/questions/1160 ... ge-results
In this context you should have a look at an old Qt Quarterly article: http://doc.qt.digia.com/qq/qq26-openglcanvas.html
ickby
Posts: 2927
Joined: Wed Oct 05, 2011 7:36 am

Re: Render qwidgets on 3d scene

Postby ickby » Tue May 27, 2014 6:28 am

I tried the approach mentioned in this article before. In theory it works very nice, you can call View3DInventorViewer::renderScene() from within "drawBackground". However, SoQT does some work with setting up the opengl context for coin rendering. Without this special context in "drawBackground" the scene renders flawed. One would need to setup the QGraphicsScene viewport in an identical way somehow... I will experiment a more bit with this :)
mrlukeparry
Posts: 655
Joined: Fri Jul 22, 2011 8:37 pm
Contact:

Re: Render qwidgets on 3d scene

Postby mrlukeparry » Sat May 31, 2014 10:56 am

This would be a great feature in terms of usability, but I would be cautious.

I know it may be a while before we migrate to Qt5, but there are various options presented and if for example we wanted to go with QML, GL context sharing may become difficult unless a framebuffer is used which I don't think is a good idea.
However, I'd suggest you to take a look at this as it may give you a few ideas:

http://devdays.kdab.com/wp-content/uplo ... rapper.pdf

If I remember right they re-write their own SoQt class to handle event handling. I think to discover a more elegant solution we will have to do something similar.

Also this is how it's used within Qt 5.1 for small applications...
https://devdays.kdab.com/?page_id=338
ickby
Posts: 2927
Joined: Wed Oct 05, 2011 7:36 am

Re: Render qwidgets on 3d scene

Postby ickby » Sat May 31, 2014 2:20 pm

Hello Luke,

I came to the same conclusion, if one wants more than simple rendering into the current context.one needs.to change soqt, every thing else will be very hacky. I actually already started to adjust soqt to my needs. If this approach is feasible for freecad will be to decide later. But it is a quite interesting work.

Thanks for the.links, I will have a look!
ickby
Posts: 2927
Joined: Wed Oct 05, 2011 7:36 am

Re: Render qwidgets on 3d scene

Postby ickby » Thu Jun 05, 2014 1:02 pm

Hello,

sooo I've tested many things, digged through lot's of code and came to two possible ways of doing this. Actually it is the same way of working but two different implementations. A detailed explanation (also for documentation purposes) will be below, but the main Infos first. Remember that the implemenations should show how it is supposed to work, they are by no way complete or useful yet.

Way 1: render the qt widgets inside the coin3d scene update and handle the eventy manually
test code: https://github.com/ickby/FreeCAD_sf_mas ... /qtoverlay


Way 2: Change SoQt to use a QGraphicsView
SoQt source(binary compatible): https://github.com/ickby/SoQtGraphicsView
FreeCad that utilyses changed SoQT: https://github.com/ickby/FreeCAD_sf_mas ... iewWidgets
(you need to change cmake variable to use new soqt)

As written earlyer, I would work further on this if there is any chance this gets included into freecad. Maybe the principles are just not clean enough and it is to hackish to include it in the source, don't know. So I'm very itnerested to hear opinions!

Detailed analysis of the problem
*****************************************
QT renders everything with it's QPainter. QPainter uses a platform-specific QPaintEngine, which is different for mac, x11, windows etc. When freecad renders it's 3d view with opengl, this happens outside of this framework. The content is not known to qt and can not be handled by the standart qt drawing system, it is just drawn "above" it. The result: transparent widgets are not transperant but the background is rubbish if they are just added as childs. This was discussed earlyer in this thread.

Werner posted this link http://doc.qt.digia.com/qq/qq26-openglcanvas.html which shows how it can still be done with a QGraphicsView. So how does this work? Pretty simple: The QGraphicsView can have a QGLWidget as viewport. If this is the case, the QGraphicsView::paintEvent initializes a QPainter with an openGL QRenderengine. This (simplified) call stack is executed:

Code: Select all

void paintEvent() {
    QPainter painter(viewport)  // painter uses openGL as initialized with a QGLWidget
    drawBackground(painter)     // here our opengl background scene is rendered
    drawWidgets(painter)          // all QT widgets get rendered with openGL calls, as painter uses the openGL engine
    drawForeground(painter)
} 
So instead of using the standart renderengine everything is rendered with openGL ontop of each other. This can easily be copied and used directly in our scenegraph rendering. This is what I've done in "Way 1". After rendering the 3d scene I just initialize a QPainter on the our QGLWidget (and make it therefore use openglcalls) and render all QWidgets with it ontop of the scene. There is only one thing to be aware of: when QPainter gets initialized it changes the glcontext to make it appropriate for its drawing. when the painter finishes (QPainter::end()) the context is not restored! this must be done by "manual" with saving and restoring the context attributes before and after the QPainter use.

Now this works nice, but has the drawback that one needs to do all event handling manual. Therefore using QGraphicsView seems to have a huge advantage as it does take care of this. But there other problems arise. First, one can not just draw the 3d scene in "drawBackground" as this is called after the QPainter initialization and therefore the glcontext is already changed in a way that leads to a wrong rendering. So one has to override the QGraphicsView::paintEvent to fist draw the scene and then the standart graphicsview stuff. But here another problem appears: To render the scene you need to call SoQtRenderArea::redraw() to get some important initialisations, but this also swaps opengl buffers. So everything drawn by QGraphicsView is rendered to the wrong buffer and not visible. This and a few other Coin annoyances (they try real hard to make their own render system decoupled from QT) lead to the fact that one needs to addjust all this in SoQt itself. And that is "Way 2".
User avatar
yorik
Site Admin
Posts: 11585
Joined: Tue Feb 17, 2009 9:16 pm
Location: São Paulo, Brazil
Contact:

Re: Render qwidgets on 3d scene

Postby yorik » Thu Jun 05, 2014 2:02 pm

Impressive work ickby... I also hope it ends up to an implemented solution. Having qt widgets on the 3d window can open a lot of interesting possibilities.
wmayer
Site Admin
Posts: 15000
Joined: Thu Feb 19, 2009 10:32 am

Re: Render qwidgets on 3d scene

Postby wmayer » Thu Jun 05, 2014 4:15 pm

Way 2: Change SoQt to use a QGraphicsView
SoQt source(binary compatible): https://github.com/ickby/SoQtGraphicsView
FreeCad that utilyses changed SoQT: https://github.com/ickby/FreeCAD_sf_mas ... iewWidgets
(you need to change cmake variable to use new soqt)
I doubt that you will succeed to push your modified SoQt version to any official repositories.

Btw, a 3rd way could be to pick some useful code from Quarter (the successor of SoQt) and write our own little library.