Subfolder for Python modules

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
looo
Posts: 788
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Postby looo » Wed Mar 08, 2017 4:26 pm

@bernd: exactly. If we properly handle the structure of the modules, we do not have to care about the names of the python-files / directories

@wmayer: My suggestion was only referring to warnings for 3rd party-modules which haven't been ported to the new style. I don't think we can print a warning when a module is imported the old way. This would be very nice to have, but I don't think there is an easy way to do it.

I think we should do the non braking things as soon as possible. This is:
- change the FreeCAD.__path__ variable
- prefix the .so/.dll with _
- place an __init__.py in the module base dir and import the shared library there

With this changes done we can then slowly move towards the new importing system...
looo
Posts: 788
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Postby looo » Thu Mar 09, 2017 7:40 am

@Bernd: The problem with clashing files will also exist in the transition phase. But anyone who experience this can simple remove his workbench from the sys.path and go for proper importing. So if FirstWorkBench and SecondWorkBench both want to have a directory called PyGui, one of these has to remove the workbench path from sys.path. I already have posted how to achieve this, but we could also make a function for this. Something like:

Code: Select all

FreeCAD._useNewStyleModule("MyModule")


Here is a first PR for step1: https://github.com/FreeCAD/FreeCAD/pull/602
feel free to discuss.
triplus
Posts: 5016
Joined: Mon Dec 12, 2011 4:45 pm

Re: Subfolder for Python modules

Postby triplus » Thu Mar 09, 2017 2:23 pm

Prefixing therefore won't go away for now. Due to the current and new way of importing being supported. As for the useNewStyleModule nice work. But i do imagine only experience users will grasp what to do with it and why? For the rest likely prefixing is easier solution to understand until prefixing will be supported.

As for the new style. If it gets merged i could start exploring it by using it in the Tux module. Modules not part of FreeCAD will need to continue use prefixing as likely they want to support FreeCAD 0.16 for now. Therefore i am guessing something like:

Code: Select all

from FreeCAD.Tux import PieMenu


Should work? I will eventually write some documentation and therefore no point in using "old style" for documentation purposes anymore if this gets merged.

In short (not thinking about every possible angle ATM):

  • Prefixing is still obligatory to avoid clashes. Or useNewStyleModule function must be used instead (FreeCAD 0.17+).
  • New style of importing can start to be used in code and in documentation (FreeCAD 0.17+).

In the future it could happen current style of importing could be removed. If and when that happens prefixing or using useNewStyleModule function to avoid potential clashes should not be needed anymore.
looo
Posts: 788
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Postby looo » Thu Mar 09, 2017 6:24 pm

triplus wrote:But i do imagine only experience users will grasp what to do with it and why?

I think users with python background won't miss the old style module... Therefor, if this is properly documented the chance is there somebody will use it.

triplus wrote:As for the new style. If it gets merged i could start exploring it by using it in the Tux module. Modules not part of FreeCAD will need to continue use prefixing as likely they want to support FreeCAD 0.16 for now.


I have done this with my gear workbench: https://github.com/looooo/FCGear/commit ... 63ed951bae
in this way it's also FreeCAD 0.16 compatible.

Btw.
The pullrequest will allow both:

Code: Select all

from FreeCAD.Tux import PieMenu
from Tux import PieMenu


I don't know if we should really force to always import from FreeCAD.

triplus wrote:I will eventually write some documentation and therefore no point in using "old style" for documentation purposes anymore if this gets merged.


That would be very nice. I think there isn't any documentation about the structure of a FreeCAD module. And thinking back wehen I tried my first module I used to look at how it is done in draft.

I think a directory structure and explaining why there are 3 typed of init-files will be a huge improvement.

Code: Select all

MyModule 
├── Init.py 
├── InitGui.py 
├── __init__.py 
├── subPackage1 
    ├── __init__.py 
    ├── subModule11.py
    └── subModule12.py 
├── subPackage2 
    ├── __init__.py 
    ├── subModule21.py 
    └── subModule22.py
triplus
Posts: 5016
Joined: Mon Dec 12, 2011 4:45 pm

Re: Subfolder for Python modules

Postby triplus » Thu Mar 09, 2017 6:39 pm

Yes developer usually has to look for reference in code and i don't remember i have ever read anywhere you need to prefix just about anything to avoid clashes. Note that i don't really have issues with the fact you need to prefix everything. As the system is simple and it works. And such system will likely still be needed elsewhere. For example command and icon names and ... But if proposed new system of importing will be made available i will try it out and use it in the Tux module in the future.
triplus
Posts: 5016
Joined: Mon Dec 12, 2011 4:45 pm

Re: Subfolder for Python modules

Postby triplus » Thu Mar 09, 2017 6:49 pm

Just one more question:

Code: Select all

FreeCAD._useNewStyleModule("MyModule")


Possible race condition won't happen here?
looo
Posts: 788
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Postby looo » Thu Mar 09, 2017 7:57 pm

Can you explain your concerns?

As far as I know the modules get initialized one by one. So not in parallel. I don't know if this is planed for the future... ? If it is I would vote to do it after modules using the new style import ;)
triplus
Posts: 5016
Joined: Mon Dec 12, 2011 4:45 pm

Re: Subfolder for Python modules

Postby triplus » Thu Mar 09, 2017 8:35 pm

Yes you are correct. As long as you do that before anything else (inside the individual module by using the mentioned function) it should work OK regarding the race condition. You used in Init.py:

Code: Select all

import FreeCAD

# delete from sys.path for new style module
FreeCAD._newStyleModule("FCGear")


And that should do it for any of the default FreeCAD modules if set. As for downstream modules you used:

Code: Select all

import FreeCAD, sys

# delete from sys.path for new style module
try:
    FreeCAD._newStyleModule("FCGear")
except AttributeError:
    sys.path.append("/home/lo/.FreeCAD/Mod")


First check if _newStyleModule() function is available and if not take care of the path. That likely comes down to having some additional path related module FCGearPath.py to return:

Code: Select all

import os

def path():
    """Return module path location."""

return os.path.dirname(__file__)


As "/home/lo/.FreeCAD/Mod" won't do. Now if all downstream modules will mess with that. That is until they will absolutely need to do that. I am not exactly sure about that. ;)

P.S. And likely the terminology new style module isn't the best. As it signals this are some special new and different FreeCAD modules. In reality they are exactly the same they are just imported differently. Therefore the term Import could be added in the function name and description?
wmayer
Site Admin
Posts: 11357
Joined: Thu Feb 19, 2009 10:32 am

Re: Subfolder for Python modules

Postby wmayer » Thu Mar 09, 2017 9:49 pm

I don't know if we should really force to always import from FreeCAD.

I prefer it because then it's clear that this is a FreeCAD module. The second point is that a 3rd party package may use the same name as we use for our modules and then we again have a naming clash.
looo
Posts: 788
Joined: Mon Nov 11, 2013 5:29 pm

Re: Subfolder for Python modules

Postby looo » Thu Mar 09, 2017 10:28 pm

triplus wrote:As "/home/lo/.FreeCAD/Mod" won't do. Now if all downstream modules will mess with that. That is until they will absolutely need to do that. I am not exactly sure about that.

it needed some time to see the problem :D

it should be:

Code: Select all

os.path.realpath(FreeCAD.ConfigGet("UserAppData")+"Mod")


Yes, it will get a bit messy if a module want to use this new kind of import with older freecad version..., but it is possible. We would need something like

Code: Select all

from future_FreeCAD import fix_my_bug
;)

triplus wrote:P.S. And likely the terminology new style module isn't the best. As it signals this are some special new and different FreeCAD modules. In reality they are exactly the same they are just imported differently. Therefore the term Import could be added in the function name and description?


FreeCAD._useNewStyleImport() ?

wmayer wrote:I prefer it because then it's clear that this is a FreeCAD module. The second point is that a 3rd party package may use the same name as we use for our modules and then we again have a naming clash.


that's true. But the imports will get very long.