Subfolder for Python modules

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
simonvanderveldt
Posts: 25
Joined: Tue Mar 14, 2017 2:11 pm

Re: Subfolder for Python modules

Postby simonvanderveldt » Mon Mar 20, 2017 3:01 pm

looo wrote:@simonvanderveldt

It would be nice to have your suggested way to handle imports in the master. But I think there are two things that shouldn't be changed:

- import FreeCAD -> should import the shared library (we can work around by adding a dir called FreeCAD and there place a "from _FreeCAD import *")

That's the way FreeCAD currently already works, right? Nothing needs to be changed for this?

- use FreeCAD instead of freecad for namespace packages. I think this has historic reasons. Also there is no suggestion from python to use lower case. PySide and PyQt also use upper case...

This has a couple of issues. First of all, it actually is normal python convention to use lowercase for package and module names, see https://www.python.org/dev/peps/pep-000 ... dule-names.
Now whilst PEP-8 also states
PEP-8 wrote:New modules and packages (including third party frameworks) should be written to these standards, but where an existing library has a different style, internal consistency is preferred.
note that it can be pretty annoying when you're proficient in a language and it's idioms and want to help/pickup a project and everything is non-standard.

But, more importantly, since FreeCAD is already a Python module this isn't possible. There can't be both a Python module and a Python package with the same name. So either the current FreeCAD Python module needs to be moved to a package like freecad.FreeCAD (or preferably freecad.freecad) or the FreeCAD module can stay as it is for now and a new package freecad.modules can be created to contain the new modules.
Moving the modules to a new package will also give a nice and clean separation between old modules and new ones.

I'll work on extending the PoC I had into something that actually works. I've started with the Arch workbench, had to pick one :)
Is that OK?
looo
Posts: 699
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Postby looo » Mon Mar 20, 2017 4:31 pm

simonvanderveldt wrote:That's the way FreeCAD currently already works, right? Nothing needs to be changed for this?

No, "import FreeCAD" directly loads the shared library.

This has a couple of issues. First of all, it actually is normal python convention to use lowercase for package and module names, see https://www.python.org/dev/peps/pep-000 ... dule-names.

This pep is only a suggestion. As of historic reasons, and also to be consistent with c++ I would vote for staying with UpperCase. But I am not the one to be convinced ;)

But, more importantly, since FreeCAD is already a Python module this isn't possible. There can't be both a Python module and a Python package with the same name.


Making a package out of the FreeCAD module wouldn't change the imports. and then we can use this package as entry point for your suggested namespace packages. I don't see the reason why this wouldn't work.

So either the current FreeCAD Python module needs to be moved to a package like freecad.FreeCAD (or preferably freecad.freecad) or the FreeCAD module can stay as it is for now and a new package freecad.modules can be created to contain the new modules.


I think having a module (FreeCAD) plus a package (freecad) will confuse more then having all available via FreeCAD.
And "from freecad import FreeCAD" is also quite strange. and also annoying.
looo
Posts: 699
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Postby looo » Mon Mar 20, 2017 7:16 pm

After thinking about this again, "freecad" lower case would really be a better solution.

Code: Select all

#old                     -> new
import FreeCAD as App    -> from freecad import App
import FreeCADGui as Gui -> from freecad import Gui
import FemCommands       -> from freecad.Fem import FemCommands


This looks quite nice. Plus we would have the option to make modules extendable. And 3rd party-modules could be installed with pip.

We will get problems if we exdent the __path__ of the FreeCAD module like was decided previously. For example we cannot add a package"Vector" because this will lead to name clashes.

What does the community think about this?

@Bernd Maybe you wait a bit with updating the imports ;)
User avatar
kkremitzki
Posts: 321
Joined: Thu Mar 03, 2016 9:52 pm
Location: Texas

Re: Subfolder for Python modules

Postby kkremitzki » Mon Mar 20, 2017 10:35 pm

I think it's a good idea as it makes the FreeCAD Python interface more Pythonic.
simonvanderveldt
Posts: 25
Joined: Tue Mar 14, 2017 2:11 pm

Re: Subfolder for Python modules

Postby simonvanderveldt » Mon Mar 20, 2017 10:42 pm

After looking through the current modules I decided to start with the Start module to try to switch it over to a normal Python module because it's easy to see if it works since it's the first screen that pops up :) and it only has one dependency, the Web workbench.

It's working and the modules as installed are pretty much normal Python modules now :)
Only work left to do is make sure it can access it's resources, in this case the HTML page and the accompanying assets.
Code lives here https://github.com/simonvanderveldt/Fre ... ce-modules

[edit] Unfortunately it doesn't really work, because for some reason FreeCAD still expects Start.so to live in ./lib/Start.so. Does anyone have an idea why this is?

[edit] Fixed, it was caused by an explicit target_link_libraries for Start in Start/Gui/CMakeLists.txt.
Last edited by simonvanderveldt on Tue Mar 21, 2017 3:00 pm, edited 2 times in total.
ian.rees
Posts: 526
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Re: Subfolder for Python modules

Postby ian.rees » Tue Mar 21, 2017 8:15 am

kkremitzki wrote:I think it's a good idea as it makes the FreeCAD Python interface more Pythonic.

+1

One relatively minor question/consideration is that the FreeCAD module name (which as I understand things is only based on the .so/.dll for it?) might have dependencies outside of the FreeCAD Python interface - packaging scripts or whatever. None come to mind, and it likely wouldn't be much of a maintenance problem. But, that might be another nudge toward using "FreeCAD" for the module and "freecad" for the package. -Ian-
simonvanderveldt
Posts: 25
Joined: Tue Mar 14, 2017 2:11 pm

Re: Subfolder for Python modules

Postby simonvanderveldt » Tue Mar 21, 2017 1:11 pm

looo wrote:Making a package out of the FreeCAD module wouldn't change the imports. and then we can use this package as entry point for your suggested namespace packages. I don't see the reason why this wouldn't work.

You mean like your example here?
looo wrote:

Code: Select all

import FreeCAD as App    -> from freecad import App


That's what I meant as well. The current FreeCAD module (.so file) would need to be moved into a package to make it work. Which would mean imports would need to updated to something along the lines of your example.

This would need some work on the C++ side though. For some reason it keeps throwing cannot import name FreeCAD at me when I use the following structure

Code: Select all

~/s/t/importtest ❯ tree                   
.
└── freecad
    ├── FreeCAD.so
    └── __init__.py


I guess this is because FreeCADInit.py is run as a script instead of as a module
simonvanderveldt
Posts: 25
Joined: Tue Mar 14, 2017 2:11 pm

Re: Subfolder for Python modules

Postby simonvanderveldt » Tue Mar 21, 2017 1:12 pm

ian.rees wrote:
kkremitzki wrote:I think it's a good idea as it makes the FreeCAD Python interface more Pythonic.

+1

One relatively minor question/consideration is that the FreeCAD module name (which as I understand things is only based on the .so/.dll for it?) might have dependencies outside of the FreeCAD Python interface - packaging scripts or whatever. None come to mind, and it likely wouldn't be much of a maintenance problem. But, that might be another nudge toward using "FreeCAD" for the module and "freecad" for the package. -Ian-

Do you mean purely from an install script/moving files perspective?
Or are there also non-Python consumers of the .so files?
looo
Posts: 699
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Postby looo » Tue Mar 21, 2017 4:22 pm

simonvanderveldt wrote:
looo wrote:Making a package out of the FreeCAD module wouldn't change the imports. and then we can use this package as entry point for your suggested namespace packages. I don't see the reason why this wouldn't work.

You mean like your example here?


No this was my opinion at the time when I wasn't aware of the nameclashes between FreeCAD (shared library) and the packages available via FreeCAD.

That's what I meant as well. The current FreeCAD module (.so file) would need to be moved into a package to make it work.


I don't think we have to do this. For the modules we have to prefix the shared objects anyway. So these can stay where they are. Maybe we will do this for FreeCAD - shared library aswell, but for the moment I would just stay with the current structure. My suggestion would look something like this:

Code: Select all

└── lib
   └── freecad (in sys.path)
      └── bin
         └── FreeCAD
         └── FreeCADCmd
      └── lib (in sys.path, we still allow direct import of FreeCAD)
         └── FreeCAD.so
         └── FreeCADGui.so
         └── _PartDesign.so.so
      └── Mod
         └── OldModuleStillSupported
         └── ...
      └── freecad (this is the entry for namespace packages)
          └── __init__.py  (import FreeCAD as App, import FreeCADGui as Gui)
          └── Arch
             └── Init.py   
             └── InitGui.py             
             └── __init__.py
             └── ...
          └── PartDesign
             └── __init__.py (from _PartDesign import *)
             └── ...
          └── ...


Do we really need to import from freecad.modules?
As I had not too much time to experiment with namespace-packages: would it be possible to use them with this structure?
ian.rees
Posts: 526
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Re: Subfolder for Python modules

Postby ian.rees » Tue Mar 21, 2017 4:35 pm

simonvanderveldt wrote:Or are there also non-Python consumers of the .so files?

This is what I was referring to - there may be more, but the only example I'm aware of is https://github.com/FreeCAD/FreeCAD/pull/624 (and I'm happy to change that method if someone knows of a better way to do it!). That's within FreeCAD anyways and would be trivial to update if the .so name changes. -Ian-