Subfolder for Python modules

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
Post Reply
simonvanderveldt
Posts: 62
Joined: Tue Mar 14, 2017 2:11 pm

Re: Subfolder for Python modules

Post by simonvanderveldt »

looo wrote:
That's what I meant as well. The current FreeCAD module (.so file) would need to be moved into a package to make it work.
I don't think we have to do this. For the modules we have to prefix the shared objects anyway. So these can stay where they are. Maybe we will do this for FreeCAD - shared library aswell, but for the moment I would just stay with the current structure. My suggestion would look something like this:

Code: Select all

└── lib
	└── freecad (in sys.path)
		└── bin
			└── FreeCAD
			└── FreeCADCmd
		└── lib (in sys.path, we still allow direct import of FreeCAD)
			└── FreeCAD.so
			└── FreeCADGui.so
			└── _PartDesign.so.so
		└── Mod
			└── OldModuleStillSupported
			└── ...
		└── freecad (this is the entry for namespace packages)
		    └── __init__.py  (import FreeCAD as App, import FreeCADGui as Gui)
		    └── Arch
		    	└── Init.py	
		    	└── InitGui.py		        
		    	└── __init__.py
		    	└── ...
		    └── PartDesign
		    	└── __init__.py (from _PartDesign import *)
		    	└── ...
		    └── ...
Ah, I understand. You'd want to import FreeCAD from the global namespace in freecad/__init__.py. That would indeed work. Even though it's a bit unorthodox it would be a good migration strategy because then it's not necessary to move everything into the freecad namespace immediately but you can already start to make use of it.
looo wrote:Do we really need to import from freecad.modules?
As I had not too much time to experiment with namespace-packages: would it be possible to use them with this structure?
Yeah, you'd need a modules sub-package, so you know where to find your modules. Otherwise you'd have to do something ugly like keep a list of sub-packages that aren't modules and exclude them when initializing the modules :cry:
Last edited by simonvanderveldt on Tue Mar 21, 2017 5:28 pm, edited 1 time in total.
simonvanderveldt
Posts: 62
Joined: Tue Mar 14, 2017 2:11 pm

Re: Subfolder for Python modules

Post by simonvanderveldt »

simonvanderveldt wrote:After looking through the current modules I decided to start with the Start module to try to switch it over to a normal Python module because it's easy to see if it works since it's the first screen that pops up :) and it only has one dependency, the Web workbench.

It's working and the modules as installed are pretty much normal Python modules now :)
Only work left to do is make sure it can access it's resources, in this case the HTML page and the accompanying assets.
Code lives here https://github.com/simonvanderveldt/Fre ... ce-modules

[edit] Unfortunately it doesn't really work, because for some reason FreeCAD still expects Start.so to live in ./lib/Start.so. Does anyone have an idea why this is?

[edit] Fixed, it was caused by an explicit target_link_libraries for Start in Start/Gui/CMakeLists.txt.
Well, the Start workbench/modules is now working for me as a separate module :)
Can someone else give it a try/have a look at it?

You'd have to use

Code: Select all

cmake -DCMAKE_INSTALL_PREFIX=<wherever you want to temporarily install this build) ../<path to freecad sources>
make install
because I've only fixed the make install targets, not really understanding all the custom CMake macro's that also copy stuff when just doing a make
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Post by looo »

to test this it would be nice to have the old imports also working. Would it be possible to re-add the deleted stuff: here and here.
simonvanderveldt
Posts: 62
Joined: Tue Mar 14, 2017 2:11 pm

Re: Subfolder for Python modules

Post by simonvanderveldt »

looo wrote:to test this it would be nice to have the old imports also working. Would it be possible to re-add the deleted stuff: here and here.
Yeah, I was already considering doing so. Will do so asap, though you should be able to start FreeCAD and the Start module/workbench with the current code.
Can you try it with the current code in the meantime?
Last edited by simonvanderveldt on Tue Mar 21, 2017 6:47 pm, edited 2 times in total.
simonvanderveldt
Posts: 62
Joined: Tue Mar 14, 2017 2:11 pm

Re: Subfolder for Python modules

Post by simonvanderveldt »

ian.rees wrote:
simonvanderveldt wrote:Or are there also non-Python consumers of the .so files?
This is what I was referring to - there may be more, but the only example I'm aware of is https://github.com/FreeCAD/FreeCAD/pull/624 (and I'm happy to change that method if someone knows of a better way to do it!). That's within FreeCAD anyways and would be trivial to update if the .so name changes. -Ian-
Well, I guess there are a couple of alternative options:
- Since FreeCAD is currently already modifying sys.path, that code could maybe be fixed in such a way that the location of FreeCAD.so is the first item in that list? (not saying modifying sys.path is a good idea, but it's the smallest change)
- Alternatively if the FreeCAD.so module is moved into a package it would be enough to simply get the __path__ of the package, which seems the most Pythonic way to fix this issue :)
simonvanderveldt
Posts: 62
Joined: Tue Mar 14, 2017 2:11 pm

Re: Subfolder for Python modules

Post by simonvanderveldt »

looo wrote:to test this it would be nice to have the old imports also working. Would it be possible to re-add the deleted stuff: here and here.
Alright, git was being a bit annoying so it took a bit longer than expected, but I've updated it now so that the new normal module imports are added next to the existing imports :)
This means it's actually pretty easy to gradually move from the old modules to the new proper ones :)

One thing I did run into was this error when starting the Arch workbench

Code: Select all

global name 'Log' is not defined
Which seems to be caused by the fact that before the init code was exec'd in place and now it's properly being imported, which means the imported Python module doesn't have access to these variables. Not a real problem though, this is pretty easily fixed by adding something like this to the module relevant

Code: Select all

import FreeCAD

Log = FreeCAD.Console.PrintLog
Msg = FreeCAD.Console.PrintMessage
Err = FreeCAD.Console.PrintError
Wrn = FreeCAD.Console.PrintWarning
This could become annoying though if there a modules that use this extensively. Ideally the logging shouldn't work with aliased/global values like this, but just use Python's native logging API, but that's something for another time ;)

P.S. I did force-push the changes, so keep that in mind when pulling the code from git :)
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Post by looo »

thanks, currently building.
simonvanderveldt wrote:One thing I did run into was this error when starting the Arch workbench
This isn't a big problem. Removing the magic parts from FreeCAD will really help people understand the code. And function which work without import are quite magic ;)
This could become annoying though if there a modules that use this extensively. Ideally the logging shouldn't work with aliased/global values like this, but just use Python's native logging API, but that's something for another time ;)
I think such things can only occur in Init.py and InitGui.py.
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Post by looo »

One thing that came into my mind:

Code: Select all

import Part
will become

Code: Select all

from freecad.modules import Part
and if someone uses the console quite often this is maybe a bit annoying...
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Post by looo »

first try fails (python3):

Code: Select all

Error in FreeCADGuiInit.py: Parent module 'freecad.modules.Arch' not loaded, cannot perform relative import
Traceback (most recent call last):
  File "<string>", line 158, in <module>
  File "<string>", line 142, in InitApplications
  File "/home/lo/anaconda/envs/freecad/lib/python3.5/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 981, in _gcd_import
  File "<frozen importlib._bootstrap>", line 931, in _sanity_check
<class 'SystemError'>: Parent module 'freecad.modules.Arch' not loaded, cannot perform relative import
ian.rees
Posts: 696
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Re: Subfolder for Python modules

Post by ian.rees »

simonvanderveldt wrote:
ian.rees wrote:
simonvanderveldt wrote:Or are there also non-Python consumers of the .so files?
This is what I was referring to - there may be more, but the only example I'm aware of is https://github.com/FreeCAD/FreeCAD/pull/624 (and I'm happy to change that method if someone knows of a better way to do it!). That's within FreeCAD anyways and would be trivial to update if the .so name changes. -Ian-
Well, I guess there are a couple of alternative options:
- Since FreeCAD is currently already modifying sys.path, that code could maybe be fixed in such a way that the location of FreeCAD.so is the first item in that list? (not saying modifying sys.path is a good idea, but it's the smallest change)
- Alternatively if the FreeCAD.so module is moved into a package it would be enough to simply get the __path__ of the package, which seems the most Pythonic way to fix this issue :)
Not sure I'm following the first of those options. The issue that patch fixes is; one way or another, an external Python has found the FreeCAD.so and the FreeCAD.so needs to figure out where it actually is within the filesystem. So, the code in the patch is run before sys.path is modified by FreeCAD. It doesn't really matter where the path to the .so/.dll is within the sys.path list, so long as one of the paths points to that .so/.dll .

The second option depends on how __path__ is created for a package - is that something that the interpreter does automatically?

At any rate, I think the structural stuff you guys are looking at should come first. My impression is that few people are using FreeCAD as a module for external Python, and it's more important that the common use case works well. -Ian-
Post Reply