Now I have new questions. Why are two documents needed? App::Document and Gui::Document?
The Gui::Document doesn't provide that much functionality. It's more like a container to manage the ViewProviders. All the important logic like adding and deleting objects, undo/redo, recompute of the objects is done in the App::Document. And again, Gui::Document is needed to split the visual representation from application logic.
The advantage of using boost::signal is that you don't have to inherit from Observer/Observable, right?
Yes. With boost::signal it's sufficient to just add the (signal) methods you need and you are able to temporarily block a signal if needed. From functionality point of view boost::signal is similar to Qt's signal/slot system (but technically they are totally different).
A much further question is if it is possible to replace GUI side with another GUI toolkit quickly? say wxWidgets?
Quickly? For sure not. And we do not only use Qt in the GUI but also its non-GUI libraries (QtCore, QtXml, ...) in some of our App modules.
If it is possible, what is necessary to do during the transmission.
It is possible but not realistic. Trying to replace Qt with wxWidgets more or less means to rewrite FreeCAD from ground up.
Since the GUI part in our old code is a nightmare.
In this case it might be better to port your application to the FreeCAD platform.
However I found FreeCAD architecture is quite complicated.
Complicated? IMO, the architecture of FreeCAD is quite simple. At the first view it seems complicated for somebody who was never involved in the project but when looking deeper into the source code it should become obvious that's kept rather simple. And again, the iron rule in our architecture is to separate App from Gui logic.
Btw, if you ever try to understand the OCAF framework of OpenCascade (that's the application logic of the CAD kernel we use) then you know what "complicated" means.
Anyway, if you try to port your old application to FreeCAD then have a look at the developer section of our Wiki how to start a new application module. There you should find information about writing your own sub-classes of DocumentObject and their ViewProviders, possibly your own Property sub-classes and a custom workbench with all the toolbars and menus. For convenience you can run the Python script fcbt.py to create the skeleton of a new module.
A minor question, why using boost::signal not observable/observer pattern directly?