[DONE] conda: update to gcc7 and py3.7

Having trouble installing or compiling FreeCAD? Get help here.
looo
Posts: 2818
Joined: Mon Nov 11, 2013 5:29 pm

[DONE] conda: update to gcc7 and py3.7

Postby looo » Fri Nov 02, 2018 1:25 pm

I use this topic to track the update of conda-packages to gcc7 and py3.7.

[x] boost-cpp
[x] boost (is updated to 1.68)
[x] qt 5.6 (5.9 should be also available, 5.12 needs osx10.12) https://github.com/conda-forge/qt-feedstock/pull/79
[x] occt https://github.com/conda-forge/occt-feedstock/pull/26
[x] coin3d https://github.com/conda-forge/coin3d-feedstock/pull/10
[x] soqt (depends on qt and coin) https://github.com/conda-forge/soqt-feedstock/pull/1
[x] simage https://github.com/conda-forge/simage-feedstock/pull/4
[x] calculix https://github.com/conda-forge/calculix ... ock/pull/1
[x] pyside2 (depends on qt) https://github.com/conda-forge/pyside2- ... ck/pull/23
[x] pivy (depends on soqt, coin and simage)
[x] netgen (depends on occt and ?) https://github.com/conda-forge/netgen-feedstock/pulls
[x] vtk (heavy challenge :x :https://github.com/conda-forge/vtk-feedstock/pull/72)
[x] smesh (depends on occt, netgen and vtk)
[x] freecad 0.18

status of all packages can be seen here: https://conda-forge.org/status/
Last edited by looo on Sat Mar 16, 2019 1:04 pm, edited 8 times in total.
looo
Posts: 2818
Joined: Mon Nov 11, 2013 5:29 pm

Re: [wip] conda: update to gcc7 and py3.7

Postby looo » Thu Nov 29, 2018 10:55 am

I encounter a problem with netgen on osx and it seems like I can't solve it by my own. Maybe some c/c++ experts can help.
By disabling the gui components of netgen, I was able to compile on linux with gcc7. This was also suggestd by a dev of netgen. So I guess the gui of netgen is quite outdated and we should not build it anymore.
But building without gui seems to be not well tested on osx. At least I get 3 failing builds (py3.6 / py3.7, old and new toolchain) The test at the end of the build returns this exception:

Code: Select all


import: 'netgen'
Traceback (most recent call last):
  File "/Users/distiller/miniconda3/conda-bld/netgen_1543334948821/test_tmp/run_test.py", line 2, in <module>
    import netgen
  File "/Users/distiller/miniconda3/conda-bld/netgen_1543334948821/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/python3.7/site-packages/netgen/__init__.py", line 17, in <module>
    from . import libngpy
ImportError: dlopen(/Users/distiller/miniconda3/conda-bld/netgen_1543334948821/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeh/lib/python3.7/site-packages/netgen/libngpy.so, 2): Symbol not found: _Ng_Redraw
So I guess the function Ng_Redraw is not compiled. But I can't see why it should work for linux and windows and not for osx.


Btw. vtk seems to be the next big beast. But I guess nobody wants to fix 18 failing builds... https://github.com/conda-forge/vtk-feedstock/pull/72 Is there an option to build freecad with fem but without vtk? I guess we should really try to make vtk an optional dependency.
wmayer
Site Admin
Posts: 14776
Joined: Thu Feb 19, 2009 10:32 am

Re: [wip] conda: update to gcc7 and py3.7

Postby wmayer » Thu Nov 29, 2018 2:22 pm

So I guess the function Ng_Redraw is not compiled. But I can't see why it should work for linux and windows and not for osx.
It's always a PITA to get built netgen. Don't know why it apparently works on Linux and Win32 but not on OSX. What you can try at least is offering a dummy implementation of Ng_Redraw where you simply add the function body "{}" with no code inside.
Btw. vtk seems to be the next big beast. But I guess nobody wants to fix 18 failing builds... https://github.com/conda-forge/vtk-feedstock/pull/72 Is there an option to build freecad with fem but without vtk? I guess we should really try to make vtk an optional dependency.
This was possible in the past with the old smesh sources. But now (since 2016) vtk is a mandatory part of smesh I am not sure if FEM can still be built without vtk (and thus without smesh).
User avatar
Kunda1
Posts: 5481
Joined: Thu Jan 05, 2017 9:03 pm

Re: [wip] conda: update to gcc7 and py3.7

Postby Kunda1 » Thu Nov 29, 2018 2:48 pm

mhochsteger wrote:ping
Perhaps the netgen devs would be interested in this thread.
Want to contribute back to FC? Checkout:
#lowhangingfruit | Use the Source, Luke. | How to Help FreeCAD | How to report FC bugs and features
looo
Posts: 2818
Joined: Mon Nov 11, 2013 5:29 pm

Re: [wip] conda: update to gcc7 and py3.7

Postby looo » Thu Nov 29, 2018 3:16 pm

wmayer wrote:
Thu Nov 29, 2018 2:22 pm
It's always a PITA to get built netgen. Don't know why it apparently works on Linux and Win32 but not on OSX. What you can try at least is offering a dummy implementation of Ng_Redraw where you simply add the function body "{}" with no code inside.
I guess I have found the problem with this hint. There is a visual_dummy.cpp which libsrc is not linked to if USE_GUI is disabled. I hope this will fix the osx builds.
wmayer wrote:
Thu Nov 29, 2018 2:22 pm
This was possible in the past with the old smesh sources. But now (since 2016) vtk is a mandatory part of smesh I am not sure if FEM can still be built without vtk (and thus without smesh).
You are right. Smesh is directly dependent on vtk. So there is no way around....
Kunda1 wrote:
Thu Nov 29, 2018 2:48 pm
Perhaps the netgen devs would be interested in this thread.
It doesn't feel like there is much interest in build-feedback...
https://github.com/NGSolve/netgen/pull/21
https://github.com/NGSolve/netgen/issues/20
looo
Posts: 2818
Joined: Mon Nov 11, 2013 5:29 pm

Re: [wip] conda: update to gcc7 and py3.7

Postby looo » Thu Dec 06, 2018 10:17 am

wmayer wrote:
Thu Nov 29, 2018 2:22 pm
It's always a PITA to get built netgen.
Somehow I cannot solve the problems for osx. Maybe you can have a short look and give some feedback on the errors:

@mingwandroid from anaconda made this comment: https://github.com/conda-forge/netgen-f ... -443011992

The osx-builds are failing with:

Code: Select all

# (py36 new toolchain)
import: 'netgen'
Fatal Python error: PyThreadState_Get: no current thread

Code: Select all

# (py37 old and new toolchain)
import: 'netgen'
Segmentation fault: 11
only one osx-build succeed (py36 old toolchain).

recently this hack was removed: https://github.com/NGSolve/netgen/commi ... b82155a307
which should be solved by pybind11: https://github.com/pybind/pybind11/pull/1211
But we are not yet using a pybind11-version including this PR.

So, do you think the error is correlated to the "evil-hack" or is there a linking problem like @mingwandroid stated? Maybe you have an idea what "Fatal Python error: PyThreadState_Get: no current thread" means and how to solve it.

build-logs:
py36-old-toolchain: https://circleci.com/gh/conda-forge/net ... build-link
py36-new-toolchain: https://circleci.com/gh/conda-forge/net ... build-link
py37-old-toolchain: https://circleci.com/gh/conda-forge/net ... build-link
py37-new-toolchain: https://circleci.com/gh/conda-forge/net ... build-link
wmayer
Site Admin
Posts: 14776
Joined: Thu Feb 19, 2009 10:32 am

Re: [wip] conda: update to gcc7 and py3.7

Postby wmayer » Thu Dec 06, 2018 2:17 pm

Fatal Python error: PyThreadState_Get: no current thread
This error message comes from the Python interpreter and is raised by its C API. There are several functions in Python that want to access the current thread state (see PyThreadState in pystate.h) and therefore call the function PyThreadState_Get. If the current thread state is null the interpreter aborts with a fatal error.
@mingwandroid from anaconda made this comment: https://github.com/conda-forge/netgen-f ... -443011992
It's clearly a runtime error and not a linking error. This means somewhere in the build process the Python interpreter is invoked.
recently this hack was removed: https://github.com/NGSolve/netgen/commi ... b82155a307
This looks very suspicious to me as it is exactly doing something with the thread state.
which should be solved by pybind11: https://github.com/pybind/pybind11/pull/1211
I might be wrong but to me this PR is something different. According to the OP it says that it fixes a deadlock when trying to access the GIL when it's already hold.
A deadlock makes an application to lock because two threads are waiting for each other, so the process never ends but cannot continue either. In your case the Python process exits due to an error.
According to the Python C code there can be several cases how to nullify the thread state but the most obvious is that something like this is called:

Code: Select all

PyThreadState* ts = PyThreadState_Swap(NULL);
but then it was forgotten to restore it with

Code: Select all

PyThreadState_Swap(ts);
So, do you think the error is correlated to the "evil-hack" or is there a linking problem like @mingwandroid stated? Maybe you have an idea what "Fatal Python error: PyThreadState_Get: no current thread" means and how to solve it.
It must be the evil hack stuff. Can't you simply revert the change to restore the evil hack?
looo
Posts: 2818
Joined: Mon Nov 11, 2013 5:29 pm

Re: [wip] conda: update to gcc7 and py3.7

Postby looo » Sat Dec 08, 2018 10:05 am

wmayer wrote:
Thu Dec 06, 2018 2:17 pm
It must be the evil hack stuff. Can't you simply revert the change to restore the evil hack?
The problem occurs in both cases. With and without the "evil hack". I am now trying to clean the cmake a bit as there is much apple-stuff not needed for conda...
looo
Posts: 2818
Joined: Mon Nov 11, 2013 5:29 pm

Re: [wip] conda: update to gcc7 and py3.7

Postby looo » Sat Dec 08, 2018 1:02 pm

I guess this is what @mingwandroid is saying:

https://stackoverflow.com/questions/156 ... 6_15678153

So "import netgen" from the conda-python will import the netgen-bindings which are linked to system python...
wmayer
Site Admin
Posts: 14776
Joined: Thu Feb 19, 2009 10:32 am

Re: [wip] conda: update to gcc7 and py3.7

Postby wmayer » Sat Dec 08, 2018 2:59 pm

Ok, this means that there are two different Python runtime libraries loaded while the latter is not initialized and thus causes the application to crash. So, yes it must be avoided to link against different Python versions.