Alpha tester wanted: Collaborative FC add-on

Have some feature requests, feedback, cool stuff to share, or want to know where FreeCAD is going? This is the place.
Forum rules
Be nice to others! Read the FreeCAD code of conduct!
ickby
Veteran
Posts: 3116
Joined: Wed Oct 05, 2011 7:36 am

Alpha tester wanted: Collaborative FC add-on

Post by ickby »

Hello,

over the last years I had a little side project ongoing. The idea was to have an add-on which allows users to concurrently edit FreeCAD files. It is not yet in a workable state to do that reliable, but I think I'm already quite far when it comes to the back-end. Hence I like some alpha testers to iron out the current state of things before going on. I'm becoming a bit blind of bugs already, hence a fresh set of eyes would be helpful.

The general idea can be seen here:
phpBB [video]


Questions for brave souls who want to give me some testing help:
1. Can you install it (see instructions below)
2. Does it run on your machine as shown in the video?
3. Can you find FreeCAD workflows, actions, etc. that lead to unsynchronized documents in the multiple open FreeCAD instances? Do the following:
3.1. Setup two FreeCad instances and and make sure both are connected to the network
3.2 Open a document on the first instance and share it.
3.3 In the second instance it will show up as "node" type document, open it there
4.4 In any instance, try a FreeCAD workflow, e.g. PartDesign, Draft, Arch, FEM, Path... whatever you can think of
4.5 On each step check, if the second instance does replicate your action correctly and the documents are the same
4.6 If the documents are not the same, or an error occurred in the report view, try to make a minimal example out of it that fails reproducible
4.7 Get creative in breaking it (except the known issues :) )

Installation:
1. Add this repository as custom repository to the addon manager (configure menu): https://github.com/OpenCollaborationPla ... borativeFC
2. Restart add-on manager, Install CollaborativeFC
3. Restart FreeCAD

Things known not to work:
1. Links into other documents
2. On Linux with python 3.9 your CPU will go to 100%. There is a problem with an python module, hopefully it gets fixed soon
3. Does not work reliable with different FreeCAD versions. All 0.18 or all 0.19 should work, but not a mix
4. Concurrent editing will fail at the moment. You can work in both FreeCAD instances and it should be mirrored, but if you do it at the same time synchronization is not guaranteed yet. E.g. if one instance has a longrunning update on a model, and the other instance edits stuff during the update, it currently breaks.
5. If you are brave enough to test it over different PCs most likely it will fail quite fast

How it works from the technical site:
1. Everything happens peer to peer, so your data is not uploaded or stored on any server
2. It starts a p2p node in the background, with which the FreeCAD instance communicates. There is one node per PC, hence in the video both FreeCAD instances connect to the same node
3. It stores the FC document data locally and provides synchronization to other nodes which joined the same document (but this is still somewhat yankie)

Site note:
The p2p node does crate a folder called .ocp in your home folder ($HOME in linux) and a config file in your OS default config folder, which must be cleaned manual when you want to fully remove everything afterwards.

Future plans:
1. Create regression tests for all problems found by alpha testers
2. Add the transaction stuff, so that real concurrent editing works
3. Iron out the multi-PC workflow to be stable

Happy testing and thanks for any help and comment!
User avatar
thomas-neemann
Veteran
Posts: 11921
Joined: Wed Jan 22, 2020 6:03 pm
Location: Osnabrück DE 🇩🇪
Contact:

Re: Alpha tester wanted: Collaborative FC add-on

Post by thomas-neemann »

ickby wrote: Wed Mar 24, 2021 5:50 pm
sensational. does it also support the arch wb?
Gruß Dipl.-Ing. (FH) Thomas Neemann

https://www.youtube.com/@thomasneemann5 ... ry=freecad
User avatar
Kunda1
Veteran
Posts: 13434
Joined: Thu Jan 05, 2017 9:03 pm

Re: Alpha tester wanted: Collaborative FC add-on

Post by Kunda1 »

This is fantastic!
Alone you go faster. Together we go farther
Please mark thread [Solved]
Want to contribute back to FC? Checkout:
'good first issues' | Open TODOs and FIXMEs | How to Help FreeCAD | How to report Bugs
ickby
Veteran
Posts: 3116
Joined: Wed Oct 05, 2011 7:36 am

Re: Alpha tester wanted: Collaborative FC add-on

Post by ickby »

thomas-neemann wrote: Wed Mar 24, 2021 7:15 pm does it also support the arch wb?
It should, yes. It's a fully generic implementation, and everything in FreeCAD should be supportet. But as I only use very little of FC in my own workflows I did not test much outside Part/PartDesign. Hence it would be nice if some other perople could test their standard workflows.
wmayer
Founder
Posts: 20309
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: Alpha tester wanted: Collaborative FC add-on

Post by wmayer »

ickby wrote: Wed Mar 24, 2021 5:50 pm Installation:
1. Add this repository as custom repository to the addon manager: https://github.com/OpenCollaborationPla ... borativeFC
2. Install CollaborativeFC in the add-on manager
I somehow couldn't find an appropriate entry in the Addon manager. So, I downloaded your repository as ZIP file and extracted it in my FreeCAD's local Mod folder.
ickby wrote: Wed Mar 24, 2021 5:50 pm 3. Restart FreeCAD
This raises an error: During initialization the error "cannot import name 'resources'" occurred in /home/user/.FreeCAD/Mod/CollaborativeFC/InitGui.py. When using the -l option I get the reason of this failure:
Log: ----------------------------------------------------------------------------------------------------
Log: Traceback (most recent call last):
File "<string>", line 131, in InitApplications
File "<string>", line 54, in <module>
File "/home/user/.FreeCAD/Mod/CollaborativeFC/Interface/__init__.py", line 2, in <module>
from Resources import resources
ImportError: cannot import name 'resources'
Log: ----------------------------------------------------------------------------------------------------
After adding an empty __init__.py file to the Resources directory allowed the registration of the wb.
wmayer
Founder
Posts: 20309
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: Alpha tester wanted: Collaborative FC add-on

Post by wmayer »

When pressing the button to start a p2p session a dialog comes up that the following packages cannot be found: ocp, msgpack, autobahn, aiofiles

This is wrong because python3-autobahn was already installed and I installed the apt packages python3-msgpack and python3-aiofiles. The dialog should only list the packages that are really missing.

In the dialog you offer an Install option but inside the code you use this line

Code: Select all

cmd = f"python -m pip install -r {self.__requirements}"
which may install the package for Py2 if python is a symlink to python2. So, I decided to get it directly from there:
https://files.pythonhosted.org/packages ... ne-any.whl

After re-running FreeCAD it now raises an error that qasync cannot be imported. I found it here https://pypi.org/project/qasync/#files
wmayer
Founder
Posts: 20309
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: Alpha tester wanted: Collaborative FC add-on

Post by wmayer »

After I have installed all missing packages and click the icon I can start an ocp node but trying to connect the API to the node gives me an error:
AttributeError: module 'asyncio' has no attribute 'get_default_loop'
ickby
Veteran
Posts: 3116
Joined: Wed Oct 05, 2011 7:36 am

Re: Alpha tester wanted: Collaborative FC add-on

Post by ickby »

Thanks for testing werner!
I somehow couldn't find an appropriate entry in the Addon manager
The option for this comes up in the configure menu within the addon manager. I updated the description
After adding an empty __init__.py file to the Resources directory allowed the registration of the wb.
I add it in the next update
This is wrong because python3-autobahn was already installed and I installed the apt packages python3-msgpack and python3-aiofiles. The dialog should only list the packages that are really missing.
Yes, it currently shows all dependencies if a single one cannot be found. I need to think how to change that best
In the dialog you offer an Install option but inside the code you use this line
cmd = f"python -m pip install -r {self.__requirements}"
which may install the package for Py2 if python is a symli8nk to python2. So, I decided to get it directly from there.
Hm, right, I did not consider this scenario. Do you have an idea howto detect such cases? I think this can happen in linux only, as in the other os' the packaged python is used.
After re-running FreeCAD it now raises an error that qasync cannot be imported. I found it here https://pypi.org/project/qasync/#files
The requirement for this was an oversigth from me, I removed all occurances in the code now.
AttributeError: module 'ocp' has no attribute 'node_path'
Hm, interesting. Could you provide me with the output of this python code on your PC?

Code: Select all

import platform
platform.machine()
platform.system()
and here I gave up.
This already helps a lot, thanks!
wmayer
Founder
Posts: 20309
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: Alpha tester wanted: Collaborative FC add-on

Post by wmayer »

ickby wrote: Thu Mar 25, 2021 2:08 pm Yes, it currently shows all dependencies if a single one cannot be found. I need to think how to change that best
The easiest is to add each package to its own try/except block and in case of a failure add it to a list. If the list is empty all went fine.

Code: Select all

import platform
platform.machine() # this gives: 'x86_64'
platform.system()# this gives: 'Linux'
BTW, when restarting FreeCAD after having installed all needed packages I get this:

Code: Select all

[   ERROR]        Qasync._QEventLoop:   Task exception was never retrieved
future: <Task finished coro=<Node.__asyncInit() done, defined at /home/user/.FreeCAD/Mod/CollaborativeFC/OCP/Node.py:214> exception=Exception('No node running: cannot read online config',)>
Traceback (most recent call last):
  File "/home/user/.FreeCAD/Mod/CollaborativeFC/OCP/Node.py", line 215, in __asyncInit
    await self.__update()
  File "/home/user/.FreeCAD/Mod/CollaborativeFC/OCP/Node.py", line 364, in __update
    p2pPort = await self.__fetchConfig("p2p.port", running)
  File "/home/user/.FreeCAD/Mod/CollaborativeFC/OCP/Node.py", line 297, in __fetchConfig
    raise Exception("No node running: cannot read online config")
Exception: No node running: cannot read online config
I think it shouldn't print anything at this moment.

When restarting FreeCAD after having created the ocp node I get this output:

Code: Select all

[   ERROR]        Qasync._QEventLoop:   Task exception was never retrieved
future: <Task finished coro=<Node.__startLogging() done, defined at /home/user/.FreeCAD/Mod/CollaborativeFC/OCP/Node.py:217> exception=OSError(22, 'Das Argument ist ungültig')>
Traceback (most recent call last):
  File "/home/user/.FreeCAD/Mod/CollaborativeFC/OCP/Node.py", line 224, in __startLogging
    await self.__logReader.blockRun()
  File "/home/user/.FreeCAD/Mod/CollaborativeFC/OCP/Node.py", line 49, in blockRun
    await self.follow()
  File "/home/user/.FreeCAD/Mod/CollaborativeFC/OCP/Node.py", line 65, in follow
    await self.__file.seek(int(-10e3), 2) # only use last 10kB
  File "/usr/lib/python3/dist-packages/aiofiles/threadpool/utils.py", line 36, in method
    return (yield from self._loop.run_in_executor(self._executor, cb))
  File "/home/user/.FreeCAD/Mod/CollaborativeFC/Qasync/__init__.py", line 140, in run
    r = callback(*args, **kwargs)
OSError: [Errno 22] Das Argument ist ungültig
ickby
Veteran
Posts: 3116
Joined: Wed Oct 05, 2011 7:36 am

Re: Alpha tester wanted: Collaborative FC add-on

Post by ickby »

wmayer wrote: Thu Mar 25, 2021 2:07 pm After I have installed all missing packages and click the icon I can start an ocp node but trying to connect the API to the node gives me an error:
AttributeError: module 'asyncio' has no attribute 'get_default_loop'
get_default_loop is not used in my code. Was there any more information, a stack-trace maybe? I wonder where this is coming from

await self.__file.seek(int(-10e3), 2) # only use last 10kB
Seems backward seeking gives some trouble. I will double check
Post Reply