IPython notebook integration

Need help, or want to share a macro? Post here!
Maninesan
Posts: 11
Joined: Wed Mar 09, 2016 4:07 am

IPython notebook integration

Postby Maninesan » Thu Mar 17, 2016 6:14 pm

This thread is all about Ipython/jupyter notebook integration with FreeCAD, suggested as an idea for gsoc [1]. After a few days of exploration I came up with some suggestions and still there are some questions I wanna to get answers!

1. I found this repo [2] in "Qt console python" thread, which has a great work on it! Through this, we can add FreeCAD as a kernel into the notebook and visualize the generated models on default FreeCAD viewer. But according to the gsoc page, we need to visualize those models inside the ipython notebook. For that we want to generate a webgl scene and that webgl canvas should be attached inside the ipython/jupyter notebook.Then what about ipython console/qtconsole? Will they generate the webgl/html5 canvas file in the default browser of any system?

2. We have to choose a suitable javascript library which converts a FreeCAD scene into a 3D webgl form, Scenejs is mentioned in the idea page. I found an ipython widget called Pythreejs [4] which also can be used to generate webgl scenes in notebook,correct me if I'm wrong. Also I found another impressive repo [3] called "nwgl" in github, which exports a FreeCAD shape into a webgl form and save it as a html file. It uses some basic webgl javascript libraries to do the work but it exports only one scene. Is this the way to achieve Ipython integration, with more proper scene graphs conversions for all nodes? If we consider this work as the start point, What are the functionalities we need to add more?

Please guide me to further resources to proceed with. Especially the guidelines for analyze the code structure! Thoughts and Suggestions are warmly welcome. Thanks :)

[1] http://www.freecadweb.org/wiki/index.ph ... ntegration
[2] https://github.com/looooo/jupyter_freecad_kernel
[3] https://github.com/kwahoo2/nwgl
[4] https://github.com/jovyan/pythreejs
ickby
Posts: 2922
Joined: Wed Oct 05, 2011 7:36 am

Re: IPython notebook integration

Postby ickby » Thu Mar 17, 2016 9:24 pm

Hello,

good to see you are still interested in this topic! I will try to provide some answers and backgrounds.
. I found this repo [2] in "Qt console python" thread, which has a great work on it! Through this, we can add FreeCAD as a kernel into the notebook and visualize the generated models on default FreeCAD viewer.
This works already very well and is helpful for quite a few applications. However, there are also a lot of applications where using the freecad gui is not desired. In scentific context often multiple python libraries are used simultaniously and what one wants is a unified environment to handle those. This is I pyhton with its console and the jupyter notebook. FreeCAD is in this context used as one part of a problem solving process. For example one starts creating some geometrie in freecad an visualizes it in a ipython cell, in the next one uses a matplotlib plot to evaluate something. There are numerous examples where this tight integration into one scientific environment is useful. Especially as one can save and share the notebook with all plots (and also the interactive 3d) as html which allows sharing without executing code. This allows freecad visualisations even if it is not installed.
For that we want to generate a webgl scene and that webgl canvas should be attached inside the ipython/jupyter notebook.Then what about ipython console/qtconsole? Will they generate the webgl/html5 canvas file in the default browser of any system?
Please have a look at the ipython display modes. Most modes are supported by both, qtconsole and jupyter notebook. For example the JavaScript display mode works for both. If interactive WebGL will work for both needs to be investigated further. But the goal is to support qtconsole and jupyter. Normal console is out of scope as it has no display capabilities at all.
2. We have to choose a suitable javascript library which converts a FreeCAD scene into a 3D webgl form, Scenejs is mentioned in the idea page. I found an ipython widget called Pythreejs [4] which also can be used to generate webgl scenes in notebook,correct me if I'm wrong. Also I found another impressive repo [3] called "nwgl" in github, which exports a FreeCAD shape into a webgl form and save it as a html file. It uses some basic webgl javascript libraries to do the work but it exports only one scene. Is this the way to achieve Ipython integration, with more proper scene graphs conversions for all nodes? If we consider this work as the start point, What are the functionalities we need to add more?
I did not know those projects, but kawahoo is long lasting community member, he did some great stuff and lots of interesting experiments. The Projects can definitely be used as starting point, if we can reuse work it is of course always a good thing. But to decide if we start over from scratch or use any of this existing projects we need to do a analysis of their capabilities and all the restrictions coming from using them. I think creating a specification / list of what the requirements are for a webgl viewer is a first good step. Than one can systematically analyse if any of this projects is a good starting point. I won't be able to have a close look at the links you provided till the weekend, but than I will look into it.
Please guide me to further resources to proceed with.
There are no further resources we could provide. Here comes your creativity in, and also your skill of analyzing a problem and finding solutions. I think the best way to proceed is to learn about the freecad scene graph structure, for this I can recommend the book "The inventor mentor". It explains everything you need to know about open inventor scene graphs. It can be found by google, a pdf version for example is here: https://webdocs.cs.ualberta.ca/~graphic ... mentor.pdf
Than it may be a good idea to learn about how the ipython qtconsole/jupter display system works.
Especially the guidelines for analyze the code structure! Thoughts and Suggestions are warmly welcome. Thanks
Here you need to be a bit more specific. The code base is large and explaining everything will take too long. If you provide a more specialized question I (and for sure many others) will try our best to support you. What often hepls is to study the Python Power User Hub of the wiki first to grasp the general freecad conceps. The basics are all the same for c++ and python.
http://www.freecadweb.org/wiki/?title=Power_users_hub

Regards,
Stefan
Maninesan
Posts: 11
Joined: Wed Mar 09, 2016 4:07 am

Re: IPython notebook integration

Postby Maninesan » Sat Mar 19, 2016 12:27 am

Thanks for your elaborated answers! After a deep walkthrough over the code, I'm get confused about python bindings! Multiple wrapper/extension techniques are found there, like SWIG, Boost.python and Cxx! What's purpose of that? Also If we want to create those open inventer actions in C++ to manipulate the javascript,we need to bind some javascript engine with this project! Is there any javascript binding available now? I saw one webgl import action in "src/Mod/Arch/" directory,but it is written in python? So can we write those inventor actions in python?
ickby
Posts: 2922
Joined: Wed Oct 05, 2011 7:36 am

Re: IPython notebook integration

Postby ickby » Sat Mar 19, 2016 9:15 am

Maninesan wrote:After a deep walkthrough over the code, I'm get confused about python bindings! Multiple wrapper/extension techniques are found there, like SWIG, Boost.python and Cxx! What's purpose of that?
FreeCAD itself uses Cxx for wrapping its objects and functions to python. AFAIK Swig and BoostPython are used for projects which provide python wrappers created with them, for example Pivy. but I need to investigate this a bit further myself :)

Also If we want to create those open inventer actions in C++ to manipulate the javascript,we need to bind some javascript engine with this project! Is there any javascript binding available now?
No, the idea is not to have a JavaScript wrapper. There should be no live connection between the c++ and JavaScript side, only a "static export". You already found the current webgl/html exporter. What it does is to create html + javascript code as string whiich is then saved as html file. The same idea should be used for IPython. Create a Javascript code as a string and pass it to ipythons display functions which accept javascript code. This does not need any JavaScript backend in FreeCAD, and also no wrapping. It is more like a static export in the sense that we export a javascript code which can be executed standalone in any browser and by Ipython. (Note: once created this should also replace the current webgl export, as it is very limited)

Have a look at the ipython display system, and how it can work with javascript and html: https://ipython.org/ipython-doc/3/api/g ... splay.html

Note that a "live connection", a wrapper do access the scene graph, render and manipulate it from javascript, would also be nice, but definitely is out of scope of this gsoc project. This may be something interesting to continue after GSoC or as a next year project. Maybe even add it as optional item in your proposal, just in case some time is left at the end.
I saw one webgl import action in "src/Mod/Arch/" directory,but it is written in python? So can we write those inventor actions in python?
The code you found is a good resource to understand the general concept. But if you have a close look at the code you see that the current exporter does not export the scene graph, but only a very limited set of document objects. In "getObjectData" you find that only Part::Feature and Mesh::Feature objects are exported. This is a nice start, but not sufficient. The problem with the idea of exporting document objects separately is that for every new object the exporter needs to be adjusted, which is much work and a maintenance hell. Furthermore there is no way to manipulate materials, colors etc. in the output. This is why the GSoC project aims at exporting the scene graph instead of the document objects, it is way more complete and general.

The current exporter is written in python. You can also access the open inventor scene graph from python, search the wki for "pivy", which is the inventor python wrapper we use. So it is possible to write the action, which creates the javascript, in python. But note that the scene graph can get big, and hence it can become a performance issue to use python. The choice is in the end up to you, both c++ and python are possible.

I had a brief look at the projects you found. Let me give you some first impressions:

https://github.com/kwahoo2/nwgl: This resembles the idea we have very well, it creates a standalone webgl bsed html file. It does however have the same issue as our current html export: it works on a document objects. We want something more general, hence we want to mimic the whole scene graph.

Pythreejs: This seems to be a interesting tool for the "live connection" kind of javascript/python interaction. It is not what we want for the GSoC project. One of the nice things of the "static export" as opposed to a pythreejs version is that it can be saved together with the notebook and be displayed even if freecad is not installed on the machine. For the pythreejs version everyone wanting to open the notebook and having a look at the 3d scene needs pythreejs and freecad installed and needs to execute the code.

Annother note: most of the code we already reviewed uses threejs for webgl. This is of course possible also for this gsoc project, it is your choice which library you use. But there are others out there, for example scenejs. I would choose a library where the openinventor nodes are easily ported to. For example exporting a openinventor scenegraph to threejs seems cumbersome, as threejs does not use a scenegraph structure. It therefore needs much logic handling. Maybe for scenejs, which also is based on scenegraphs, it is a simple one to one mapping, eg. a opennvetor node becomes a scenejs node. This would be much simper. It is your task to analyse the available projects and find a good fit.
wmayer
Site Admin
Posts: 14886
Joined: Thu Feb 19, 2009 10:32 am

Re: IPython notebook integration

Postby wmayer » Sat Mar 19, 2016 11:06 am

Maninesan wrote: After a deep walkthrough over the code, I'm get confused about python bindings! Multiple wrapper/extension techniques are found there, like SWIG, Boost.python and Cxx! What's purpose of that?
Some words about the Python bindings.

SWIG: FreeCAD itself doesn't use SWIG to wrap its classes. However, SWIG is needed to unwrap a Python object and get back a pointer to the C++ object. As ickby said it's needed to e.g. convert objects between pivy and OpenInventor. So, the usage of SWIG inside FreeCAD is very limited and it won't be extended in the future.

Boost.python: Until now this wrapper is only used in the "area" module of the Path workbench and also only because "area" was taken as is from HeeksCAD. Similarly to SWIG the usage of Boost.python won't be extended as it has some serious disadvantages. It heavily uses C++ templates that slows down compilation and mainly on Linux systems creates bloated libraries because by default all symbols get exported.

PyCXX: Is a nice and lightweight library to create Python bindings but also to limit the work with the C API of Python.

shiboken: shiboken is the Python wrapper used for PySide -- the Python binding for Qt. Like SWIG we only use it to convert between PySide and Qt objects.
looo
Posts: 2905
Joined: Mon Nov 11, 2013 5:29 pm

Re: IPython notebook integration

Postby looo » Fri Feb 10, 2017 12:08 pm

this one could be of interest: http://osgjs.org/
please help with my conda-packaging efforts: https://liberapay.com/looooo/