Subfolder for Python modules

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
User avatar
bernd
Posts: 3924
Joined: Sun Sep 08, 2013 8:07 pm
Location: Zürich, Switzerland

Subfolder for Python modules

Postby bernd » Mon Mar 06, 2017 6:28 pm

On Fem we introduced two new directories to get the lots of python modules sorted some how. See https://github.com/FreeCAD/FreeCAD/tree ... rc/Mod/Fem The names are PyObjects and PyGui . There is no Fem inside the directory name. There was some concern about name clashing if other workbenches would start to use these directory names too. Attaches some post about this:

qingfeng.xia wrote:Regarding on the recent submodule directory names "PyGui" and "PyObjects", I am a bit worry of The naming clash/shadow.

It may happen if other big workbench like Arch, also has a "PyGui" folder, and then "PyGui" in Arch will shadow the Fem's. It seems FC will append each Mod folder into search path, that is why each file need a Mod Suffix like "Fem*.py". Correct me if I am not correct.

In that case, "PyFem"/"PyFemGui" or any kind with Fem inside would be safer, C++ Fem/Gui -> "FemGui.so python module" I believe Fem may be reused by other modules, not only Cfd.


I would like to here some opinions from others in this regard? Would it be better to name these directories PyGuiFem and PyObjectsFem
Last edited by bernd on Mon Mar 06, 2017 9:13 pm, edited 1 time in total.
looo
Posts: 768
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Postby looo » Mon Mar 06, 2017 7:20 pm

As you may know, this issue is caused by freecad not properly dealing with python modules. At startup every Module-directory (eg. Fem) is added to the sys.path. This way we don't need a __init__.py in the Mod-dirs but have this annoying naming issues.
I think this should be fixed the proper way. But this will need some time to be done, as every module has to be changed for proper importing.

As a workaround we can try this:

1. prefix the shared object file Fem.so with _ (underscore) -> _Fem.so
2. add a __init__.py in Mod/Fem and import everything from _Fem (from _Fem import *)
3. remove the FEM-dir from the path-variable in the Init.py:

Code: Select all

import os, sys

dirs = sys.path
for dir in dirs:
   if "Mod" + os.path.sep + "Fem" in dir:
      sys.path.remove(dir)
      break


4. change all imports effected by this
import FemGmshTools -> from Fem import FemGmshTools

5. optionally remove the Fem prefix as there is no naming-issue anymore:
from Fem import GmshTools

Once every module support proper importing we could remove the code relevant for appending the directories to the path at startup and also the added workaround (3.).
DeepSOIC
Posts: 4673
Joined: Fri Aug 29, 2014 12:45 am
Location: Saint-Petersburg, Russia

Re: Subfolder for Python modules

Postby DeepSOIC » Mon Mar 06, 2017 7:36 pm

What I did in Part-o-magic: put everything into a directory called PartOMagic (so it is Mod/Part-o-magic/PartOMagic/stuff.py). This effectively converts the workbench into a python package, and hopefully eliminates any name collisions.
triplus
Posts: 4988
Joined: Mon Dec 12, 2011 4:45 pm

Re: Subfolder for Python modules

Postby triplus » Mon Mar 06, 2017 8:35 pm

What i realized in the past is developers share FreeCAD and therefore clashes can occur. In a lot of places. For example if you don't name your command uniquely clash will occur. If you don't name your Python module uniquely clash will occur...

Simple system currently in use solves this issues. That is add prefix to anything you are doing and could result in clash. You can use suffix but in my opinion prefix is preferable. For example PartDesign Container, Part Container or Fem Container sound better to me compared to Container PartDesign, Container Part or Container Fem.
User avatar
yorik
Site Admin
Posts: 8653
Joined: Tue Feb 17, 2009 9:16 pm
Location: São Paulo, Brazil
Contact:

Re: Subfolder for Python modules

Postby yorik » Mon Mar 06, 2017 9:12 pm

I would also vote for prefixing everything with Fem (FemObjects, FemScripts, whatever), makes things clear for anyone looking at modules... Plus, in alphabetic lists of modules (for ex in doxygen), all fem related things will appear together
User avatar
bernd
Posts: 3924
Joined: Sun Sep 08, 2013 8:07 pm
Location: Zürich, Switzerland

Re: Subfolder for Python modules

Postby bernd » Mon Mar 06, 2017 9:15 pm

Just would like to mentions it is not about the modules! The modules all have prefixes. It is about the directories the modules are in! In Fem we have subdirectory PyObject (all object class modules) and subdirectory PyGui (all object gui modules, command classes, task panel classes and uis). Again all files have prefix but how about the directories they are in, should they have some Fem inside the name too? https://github.com/FreeCAD/FreeCAD/tree ... rc/Mod/Fem

AFAIK we need the __init__.py inside the module directories PyGui and PyObjects (but not in main Fem directory)! See git commit 43a19ac
DeepSOIC
Posts: 4673
Joined: Fri Aug 29, 2014 12:45 am
Location: Saint-Petersburg, Russia

Re: Subfolder for Python modules

Postby DeepSOIC » Mon Mar 06, 2017 9:22 pm

OK, experimenting...
Can I import _FemMaterial directly?

Code: Select all

>>> import _FemMaterial
Traceback (most recent call last):
  File "<input>", line 1, in <module>
ImportError: No module named _FemMaterial

Can I import _FemMaterial as submodule?

Code: Select all

>>> import PyObjects._FemMaterial
>>> #yes, imported without errors


Now, I created:
* directory %AppData%/FreeCAD/Mod/importtest
* directory %AppData%/FreeCAD/Mod/importtest/PyObjects
* file %AppData%/FreeCAD/Mod/importtest/PyObjects/__init__.py
* file %AppData%/FreeCAD/Mod/importtest/PyObjects/importtest.py
( %AppData%/FreeCAD/Mod/ is the location where add-on workbenches are installed by the official macro)

Let's repeat...

Code: Select all

>>> import PyObjects._FemMaterial
Traceback (most recent call last):
  File "<input>", line 1, in <module>
ImportError: No module named _FemMaterial
>>> import PyObjects.importtest
>>>

So, I screwed up FEM by creating a package named "PyObjects"
I guess you can draw conclusions yourself...

OS: Windows 10
Word size of OS: 64-bit
Word size of FreeCAD: 64-bit
Version: 0.17.10423 (Git)
Build type: Release
Branch: master
Hash: 45bf8ed91ed1bd65d7c5750685f8c435c9b092cf
Python version: 2.7.8
Qt version: 4.8.7
Coin version: 4.0.0a
OCC version: 7.0.0
triplus
Posts: 4988
Joined: Mon Dec 12, 2011 4:45 pm

Re: Subfolder for Python modules

Postby triplus » Mon Mar 06, 2017 9:44 pm

Thanks for testing.

P.S. Therefore FemPyObjects it is.
looo
Posts: 768
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Postby looo » Mon Mar 06, 2017 11:37 pm

I wrote:As you may know, this issue is caused by freecad not properly dealing with python modules.

bernd wrote:Just would like to mentions it is not about the modules!


replace module with package in my post. For me a module was the same as a package, but regarding this I definitely meant a package. In FreeCAD we call it also Mod(ule).

DeepSoic wrote:What I did in Part-o-magic: put everything into a directory called PartOMagic (so it is Mod/Part-o-magic/PartOMagic/stuff.py). This effectively converts the workbench into a python package, and hopefully eliminates any name collisions.


This is another workaround. But once we remove the Mod-dirs from the sys.path, this won't work anymore.

bernd wrote:AFAIK we need the __init__.py inside the module directories PyGui and PyObjects (but not in main Fem directory)!

If python would have such naming issues, nobody would use it ;) So why not take there approach and place a __init__.py in the main dir of a freecad-package?

here is a branch where I have added my suggestion:
https://github.com/looooo/FreeCAD/tree/naming_issue_fem

The importing works like this:

Code: Select all

>>> import Fem
>>> Fem.__file__
'/home/lo/anaconda/envs/freecad/Mod/Fem/__init__.py'
>>> from Fem import PyGui
>>> PyGui.__file__
'/home/lo/anaconda/envs/freecad/Mod/Fem/PyGui/__init__.py'
>>> import PyGui
Traceback (most recent call last):
  File "<input>", line 1, in <module>
ImportError: No module named 'PyGui'
>>> from Fem.PyGui import FemCommands
>>> FemCommands.__file__
'/home/lo/anaconda/envs/freecad/Mod/Fem/PyGui/FemCommands.py'
DeepSOIC
Posts: 4673
Joined: Fri Aug 29, 2014 12:45 am
Location: Saint-Petersburg, Russia

Re: Subfolder for Python modules

Postby DeepSOIC » Mon Mar 06, 2017 11:50 pm

looo wrote:But once we remove the Mod-dirs from the sys.path, this won't work anymore.
I would rather expect that everything won't work anymore, including this. So no big deal. :mrgreen: