Add ons manager - development and bugs topic

[vocx]

1. Is there any current implementation of the mentioned ideas?

Basically, the idea is to avoid using the Addon Manager, and instead use something that already exists, like pip or conda as a package manager.

The Addon Manager was created to help install the workbenches, which is essentially just a copy and paste operation. But the manager is unnecessary if the workbenches are packaged like proper Python packages. This is the idea that looo developed with his "Workbench Starterkit". See freecad.workbench_starterkit. This is what we informally call a "new-style" workbench, or namespace workbench.

Essentially, instead of placing the contents of your workbench at the top level, you place them inside a freecad directory; then your workbench becomes part of the freecad namespace. Moreover, if you add a setup.py file, this is adding the required metadata, and makes it suitable for installation by a tool such as pip. You can install it in the Mod/ directory like usual, or in the system's site-packages/ directory, like other Python packages. This has the advantage that it is already in the Python path, and you can do fancy things like create virtual environments with a particular version of Python and your package. I suggest taking a look at looo's workbenches to give yourself a good idea of how this is supposed to work.

So, you can of course improve the Addon Manager, but at the same time, what would make more sense is for you to re-structure your own workbench (https://github.com/mnesarco/MarzWorkbench). Essentially, if all external workbenches are structured in this way, then we don't need to even use the Addon Manager, all workbenches could be hosted in PyPi, and could be installed from there.

----

But about your specific idea of a metadata file. I like your proposal of the json file. I think there are old workbenches that will never be migrated to the new style, so they will have to rely on the existing infrastructure for many years. So, you could do both, restructure your workbench, and add the json code. There is no decision on that, as you can see Kunda's link hasn't been discussed in years, but if you introduce it, I think it should work well.

[mnesarco]

Basically, the idea is to avoid using the Addon Manager, and instead use something that already exists, like pip or conda as a package manager.

Ok I got the idea. But IMHO having an AddonManager is much more user friendly than pip or conda for non-developer users. I will take a look at looo's work.

Maybe a modern AddonManager that uses pip or conda under the hood is a better approach.

Thanks.

[vocx]

...
Maybe a modern AddonManager that uses pip or conda under the hood is a better approach.

Yes, that's essentially the idea.

The current Addon Manager is basically an interface on top of Git, so we can install things from an online Git repository. But what if we have the workbench in a local directory? Or if we want to install it in a different directory, or for different Python versions? Making the Addon Manager an interface on top of pip would be better. Otherwise we are just duplicating work by implementing our own complex manager to install purely Python packages.

[mnesarco]

Following the discussions in this thread: https://forum.freecadweb.org/viewtopic. ... 40#p305015

Looo finally concluded that pip is problematic (it can install incompatible packages, does not work on all platforms, etc...) and the topic was abandoned from may-2019.

I have read the full thread, the problem is not easy to solve and any solution seems to have to coexists with the current approach. The thing is that current git based approach have its problems but there is no workable pip solution yet.

Things involved:

1. Dependency management
2. Python versions 2/3
3. Platforms: Linux/Windows/Mac
4. Portable distributions (ie AppImage)
5. Shared extensions dir (.FreeCAD) used by multiple installed FreeCAD versions
6. User friendly interface (avoid command line if possible)
7. Keep legacy approach working

Given that the AddonManager will coexists with any new approach anyway, I think that improving it will not hurt. So I will keep the impulse and will be back if something useful raises.

[Kunda1]

Please see https://github.com/FreeCAD/FreeCAD-addons/issues/29

Ok I have read the thread, same idea but:

1. Is there any current implementation of the mentioned ideas?
2. Which format was finally selected? INI?
3. Is someone actually working on that?

Great! I mentioned that just to get you caught up.

1. There is some implementation but still trivial. I can't find where in src/Mod/AddonManager, maybe someone can help with that?
2. Yes, but this is still flexible due to step 1.
3. Not currently.

Honestly I don't think FreeCAD is 'married' to this model. It was chosen to emulate QGIS.

[mnesarco]

Great! I mentioned that just to get you caught up.

1. There is some implementation but still trivial. I can't find where in src/Mod/AddonManager, maybe someone can help with that?
2. Yes, but this is still flexible due to step 1.
3. Not currently.

Honestly I don't think FreeCAD is 'married' to this model. It was chosen to emulate QGIS.

Good. There is a need for a manifest file for sure, I think it can be ini or json, both formats can be parsed with builtin functions.

1. What is better?: manifest.json vs manifest.ini

Personally i think ini is beautiful for human reading but json is more portable and flexible for possible future scenarios and integration with external tools.

2. What should be put in the manifest?

- Metadata like title, icon, authors, license, etc
- Package source address and method (Git/Http/pip/downloaded zip/...)
- Dependencies on system packages
- Dependencies on other FreeCAD extensions
- Anything else?

[vocx]

...
Looo finally concluded that pip is problematic (it can install incompatible packages, does not work on all platforms, etc...) and the topic was abandoned from may-2019.

No package manager is going to be perfect. However, using something like pip or conda is going to be better because the work of handling dependencies is already done, particularly if they are Python only dependencies.

1. Dependency management
2. Python versions 2/3
3. Platforms: Linux/Windows/Mac
4. Portable distributions (ie AppImage)
5. Shared extensions dir (.FreeCAD) used by multiple installed FreeCAD versions
...

The dependency part, I have always stressed that workbench developers should explicitly mention in their README.md the dependencies they require, and let the user figure it out how to install them. In most cases, these are simple Python packages, and don't require much. In the case of big dependencies, like Numpy, the operating system normally takes care of these. For example, the FreeCAD program will have it as one of its dependencies and will install it.

There's no need to worry about Python 2. It has reached end of life, so we should not consider that any longer.

About the shared directory for extensions, this is precisely an issue that I think looo wanted to solve with virtual environments and things like that.

[mnesarco]

No package manager is going to be perfect. However, using something like pip or conda is going to be better because the work of handling dependencies is already done, particularly if they are Python only dependencies.

Yes you are right. But because this is not solved yet, I will continue with the improvement of the AddonManager, I will try to keep in mind the Looo's work so it will be easily integrated at some point.

This is my proposed Addon Layout by the moment:

https://github.com/mnesarco/FreeCAD_ExampleWorkbench

The idea is to have something that works with current AddonManager but with small adjustments that allow some improvements in the AddonManager.

[vocx]

This is my proposed Addon Layout by the moment:
...

Mmm. You are adding duplicated Init.py and InitGui.py. Why? You basically just need to use looo's structure. Just add the manifest.ini file, that's all you need to do.

Resolve the import issues (For example __file__ problems)
Mitigate the name clash issues (Addons can introduce name clashes very easy right now)

What are you talking exactly about?

The addon code are in a python package. This is following Looo's idea of making it a real python package, but I avoid the freecad namespace package to make it work with current FreeCAD loading process.

The freecad namespace already works with the current FreeCAD loading process, you can see this in the workbenches that looo himself has written. That's the whole reason looo implemented that. So, you don't need to invent a third loading process. The old system uses Init.py and InitGui.py at the root of the directory; the new method works with __init__.py and init_gui.py inside a freecad.workbench directory, this is good, and we should keep this structure in order to prevent polluting the global namespace.

I thought you just wanted to implement a better method for getting the icon for the Addon Manager. But what you are proposing now is more complex. I don't think you need to do this. How is it better than what looo already proposed, and which already works?

[mnesarco]

The freecad namespace already works with the current FreeCAD loading process, you can see this in the workbenches that looo himself has written. That's the whole reason looo implemented that. So, you don't need to invent a third loading process. The old system uses Init.py and InitGui.py at the root of the directory; the new method works with __init__.py and init_gui.py inside a freecad.workbench directory, this is good, and we should keep this structure in order to prevent polluting the global namespace.

I thought you just wanted to implement a better method for getting the icon for the Addon Manager. But what you are proposing now is more complex. I don't think you need to do this. How is it better than what looo already proposed, and which already works?

I misunderstood something, i thought that looo's approach as a WIP project not integrated into FreeCAD. If looo's layout works out of the box with current FreeCAD and AddonManager of course i don't want to reinvent the wheel.