pip-integration in addon-manager

Need help, or want to share a macro? Post here!
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
User avatar
DeepSOIC
Veteran
Posts: 7896
Joined: Fri Aug 29, 2014 12:45 am
Location: used to be Saint-Petersburg, Russia

Re: pip-integration in addon-manager

Post by DeepSOIC »

sgrogan wrote: Wed Aug 29, 2018 11:01 am Packaging of pip is still messed in your version. It should be fixed in the newest x64 builds.
OK, will try.


I tried in my conda install, it worked flawlessly from anaconda prompt. But it only installed the workbench for the conda freecad. Other freecads don't see the new workbench. I guess, it is supposed to be like that...
User avatar
sgrogan
Veteran
Posts: 6499
Joined: Wed Oct 22, 2014 5:02 pm

Re: pip-integration in addon-manager

Post by sgrogan »

DeepSOIC wrote: Wed Aug 29, 2018 11:05 am I tried in my conda install, it worked flawlessly from anaconda prompt. But it only installed the workbench for the conda freecad. Other freecads don't see the new workbench. I guess, it is supposed to be like that...
Yes, this is what looo and I am discussing in the other thread. We need a way to get pip to install into %appdata%/freecad/mod or the like. For packages with compiled extensions we somehow have to resolve potential dependency problems.
For pure python I think this should work https://stackoverflow.com/questions/416 ... tion-path#
"fight the good fight"
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

Re: pip-integration in addon-manager

Post by looo »

2 things:
1. pip.main won't work for newer versions of pip. So better use supprocess.Popen(["pip", "install", ..])

2. to install to any non-std-directory:
pip install --install-option="--install-purelib=/python/packages" package_name

or setting the environment variable "PYTHONUSERBASE", which might be the better option.
regarding: https://stackoverflow.com/questions/291 ... -using-pip
User avatar
DeepSOIC
Veteran
Posts: 7896
Joined: Fri Aug 29, 2014 12:45 am
Location: used to be Saint-Petersburg, Russia

Re: pip-integration in addon-manager

Post by DeepSOIC »

looo wrote: Wed Aug 29, 2018 11:33 am 1. pip.main won't work for newer versions of pip. So better use supprocess.Popen(["pip", "install", ..])

Code: Select all

>>> import subprocess
>>> subprocess.Popen(["pip"])
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "C:\FreeCAD\builds-snapshots\Default\bin\lib\subprocess.py", line 390, in __init__
    errread, errwrite)
  File "C:\FreeCAD\builds-snapshots\Default\bin\lib\subprocess.py", line 640, in _execute_child
    startupinfo)
WindowsError: [Error 2] 
>>> 
That's from same portable FC. From conda freecad, it works.
User avatar
DeepSOIC
Veteran
Posts: 7896
Joined: Fri Aug 29, 2014 12:45 am
Location: used to be Saint-Petersburg, Russia

Re: pip-integration in addon-manager

Post by DeepSOIC »

sgrogan wrote: Wed Aug 29, 2018 11:19 am We need a way to get pip to install into %appdata%/freecad/mod or the like.
Actually, per-instance installs are also useful. I'd like there be an option. For example, FCGear seems to somehow cause a crash onto my debug builds, so I have to uninstall it temporarily to do C++ work.
User avatar
sgrogan
Veteran
Posts: 6499
Joined: Wed Oct 22, 2014 5:02 pm

Re: pip-integration in addon-manager

Post by sgrogan »

Code: Select all

>>> subprocess.Popen(["C:/Users/chris/Downloads/FreeCAD_0.18.14487_x64_dev_win/bin/Scripts/pip"])
<subprocess.Popen object at 0x0000000009CB8358>
>>> 
I tried adding the path to sys.path but it didn't work.
"fight the good fight"
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

Re: pip-integration in addon-manager

Post by looo »

sgrogan wrote: Wed Aug 29, 2018 12:14 pm

Code: Select all

>>> subprocess.Popen(["C:/Users/chris/Downloads/FreeCAD_0.18.14487_x64_dev_win/bin/Scripts/pip"])
<subprocess.Popen object at 0x0000000009CB8358>
>>> 
I tried adding the path to sys.path but it didn't work.
sys.path is only for python. As pip is a system command you have to add it to $PATH. But I don't know what the windows equivalent is.
User avatar
sgrogan
Veteran
Posts: 6499
Joined: Wed Oct 22, 2014 5:02 pm

Re: pip-integration in addon-manager

Post by sgrogan »

Code: Select all

>>> import sys
>>> import subprocess
>>> subprocess.Popen([sys.exec_prefix + "\\Scripts\\pip"])
<subprocess.Popen object at 0x000000000A5FBD30>
>>>
Should make sure we use FreeCAD's Python.
"fight the good fight"
User avatar
DeepSOIC
Veteran
Posts: 7896
Joined: Fri Aug 29, 2014 12:45 am
Location: used to be Saint-Petersburg, Russia

Re: pip-integration in addon-manager

Post by DeepSOIC »

looo wrote: Wed Aug 29, 2018 12:23 pm As pip is a system command you have to add it to $PATH. But I don't know what the windows equivalent is.
This seems to work:

Code: Select all

>>> os.environ['PATH'] = os.environ['PATH'] + r';S:\_vt\dev\PC\Qt\FreeCAD\builds-snapshots\Default\bin\Scripts' #terribly non-crossplatform, just for test
>>> subprocess.Popen(["pip"])
<subprocess.Popen object at 0x00000257A3533A58>
>>> 
Just in case, here's my content of PATH:

Code: Select all

>>> for val in os.environ['PATH'].split(';'): print(val)
... 
C:\FreeCAD\builds-snapshots\Default\bin
C:\FreeCAD\builds-snapshots\Default\Mod\Sketcher
C:\FreeCAD\builds-snapshots\Default\Mod\ReverseEngineering
C:\FreeCAD\builds-snapshots\Default\Mod\Show
C:\FreeCAD\builds-snapshots\Default\Mod\Image
C:\FreeCAD\builds-snapshots\Default\Mod\Surface
C:\FreeCAD\builds-snapshots\Default\Mod\PartDesign
C:\Users\EMB\AppData\Roaming\FreeCAD\Mod\TabBar
C:\FreeCAD\builds-snapshots\Default\Mod\Measure
C:\Users\EMB\AppData\Roaming\FreeCAD\Mod\Lattice2
C:\FreeCAD\builds-snapshots\Default\Mod\Ship
C:\FreeCAD\builds-snapshots\Default\Mod\Plot
C:\FreeCAD\builds-snapshots\Default\Mod\MeshPart
C:\FreeCAD\builds-snapshots\Default\Mod\OpenSCAD
C:\FreeCAD\builds-snapshots\Default\Mod\Fem
C:\FreeCAD\builds-snapshots\Default\Mod\Spreadsheet
C:\FreeCAD\builds-snapshots\Default\Mod\Start
C:\FreeCAD\builds-snapshots\Default\Mod\Points
C:\FreeCAD\builds-snapshots\Default\Mod\Draft
C:\Users\EMB\AppData\Roaming\FreeCAD\Mod\_disabled
C:\FreeCAD\builds-snapshots\Default\Mod\Test
C:\FreeCAD\builds-snapshots\Default\Mod\Import
C:\FreeCAD\builds-snapshots\Default\Mod\TechDraw
C:\FreeCAD\builds-snapshots\Default\Mod\Tux
C:\FreeCAD\builds-snapshots\Default\Mod\Complete
C:\FreeCAD\builds-snapshots\Default\Mod\Web
C:\FreeCAD\builds-snapshots\Default\Mod\Material
C:\FreeCAD\builds-snapshots\Default\Mod\Raytracing
C:\FreeCAD\builds-snapshots\Default\Mod\Mesh
C:\Users\EMB\AppData\Roaming\FreeCAD\Mod\Curves
C:\FreeCAD\builds-snapshots\Default\Mod\Path
C:\FreeCAD\builds-snapshots\Default\Mod\Part
C:\FreeCAD\builds-snapshots\Default\Mod\Arch
C:\FreeCAD\builds-snapshots\Default\Mod\Robot
C:\FreeCAD\builds-snapshots\Default\Mod\AddonManager
C:\Users\EMB\AppData\Roaming\FreeCAD\Mod\Part-o-Magic
C:\FreeCAD\builds-snapshots\Default\Mod\Idf
C:\FreeCAD\builds-snapshots\Default\Mod\Inspection
C:\FreeCAD\builds-snapshots\Default\Mod\Drawing
C:\Windows\system32
C:\Windows
C:\Windows\System32\Wbem
C:\Windows\System32\WindowsPowerShell\v1.0\
C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common
C:\Program Files (x86)\Skype\Phone\
C:\Qt\Git\cmd
C:\WINDOWS\system32
C:\WINDOWS
C:\WINDOWS\System32\Wbem
C:\WINDOWS\System32\WindowsPowerShell\v1.0\
C:\WINDOWS\System32\OpenSSH\
C:\Users\EMB\AppData\Local\Microsoft\WindowsApps
C:\Qt\git\bin
C:\Users\EMB\AppData\Local\Microsoft\WindowsApps
S:\_vt\dev\PC\Qt\FreeCAD\libpack\active\bin
C:\Users\EMB\AppData\Local\GitHubDesktop\bin
S:\_vt\dev\PC\Qt\FreeCAD\builds-snapshots\Default\bin\Scripts
>>> 
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

Re: pip-integration in addon-manager

Post by looo »

One weak point about using pip/pypi are compiled dependencies. If we are allowing users to install packages to directories shared by multiple versions of FreeCAD we have to restrict the packages which can be installed with pip to noarch-python packages (This is how pure python packages are called in the conda-world) At least we have to make sure that things like numpy / scipy / ... are updated with pip. We can achieve this by adding a constraint file which constraints packages which are already included in the freecad-bundle to there versions. This way pip will not try to update them (hopefully).

regarding https://github.com/pypa/pip/issues/3090 this can be achieved by eg.:

Code: Select all

pip install future -c constraint.txt
But somehow this doesn't work with adding an additional --target (which is the obvious way to install to a non-std directory). So maybe we should try to set the env-variable "PYTHONUSERBASE" and see if pip installs to the this directory without any additional flag. If this works, I guess adding the constraint file should also work. Then we only have to create a constraint.txt from a clean bundle (appimage / windows portable) and somehow add this constraint.txt to the bundle. Maybe adding it to Ext/freecad/pip-integration or something like this as this file should be part of the bundle and also it should be easily accessible.
Post Reply