Interfacing FreeCAD with Object Storage solutions

[vejmarie]

Hi,

I was wondering if any work attempt has been made to interface FreeCAD to some object storage solutions (other than github module made by Yorik) ? The FreeCAD format is a ZIP archive of BRep files and "metadata" associated to them to build the model. I looked at ways to implement a connection to solutions like OpenStack Swift, CEPH or even Haystack but the only way I found to do it is by "hacking" the Writer class, which are deeply embedded into the code. There is a ZipWriter (which is the standard stuff used) and FileWriter class written by wmayer but I didn't found (yet) where it is used (for the second one).

I am looking for such solutions, as I do work with FreeCAD with people around the globe on models which are getting bigger and bigger and this is a pain to share them currently. Part of my goal will be to offer the capacity to store files into a "cloud" storage solution by updating only objects which needs to be updated (to save bandwidth and storage capacity). That will be a step to push for collaboration in FreeCAD.

I spent a lot of time thinking about the need or not to have a web based FreeCAD, and after a couple of months I am not convinced about the use case and the need to do it (feel free to challenge me on this). What might be good is to have a "youtube" like repo where people could see models, offers some improvements and feedback the developpers through FreeCAD application (I think we could do that "easily" with nodeocc and parsing of FreeCAD native files). Roughly a design workflow which is closed to github where everything happens on the client (we have a good client running on many O/S), with the capacity to ease sharing and model browsing could be a goot step forward.

One of the issue I face with Object Storage is the complexity of the implementation, there are many different API currently, and that s... . We could work on an abstraction layer, and write connection modules, but this is adding some work. I will focus first on running implementation on top of open source storage solution which could ease adoption in SMB environments and allow end user to decide where the data are stored.

Any thoughts ? If we got a positive feedback I loved to drive an initiative to make it happen in 0.19. I think it is doable even if I am still dangling between doing it as a FreeCAD Module or into the core of the app.

vejmarie

[wmayer]

If you want to have your custom writer class then simply inherit from Base::Writer or any of its subclasses and re-implement the method writeFiles(). It's up to you to define in the constructor to what kind of storage you want to save.

Then you need a new command class where you can invoke the writer class to save the document into.

Code: Select all

App::Dcoument* doc = ...
MyWriter writer(stream);
writer.putNextEntry("Document.xml");
doc->Save(writer);

// Special handling for Gui document.
doc->signalSaveDocument(writer);

writer.writeFiles();

FYI:
For FreeCAD's autosave/recovery function we use the class RecoveryWriter that doesn't write into a zip file but into a directory that creates an extra file for each touched property.
https://github.com/FreeCAD/FreeCAD/blob ... r.cpp#L161

[vejmarie]

Thanks, this does correspond to what I understood while reading the code . I will start working on the code.

[vejmarie]

I made some good progress regarding that feature. I have to add 2 dependencies which are to libopenssl and libcurl. Is that an issue to any of us ? I am trying to make it as small as I could, but I need them to communicate with the server and most of the API are using SHA1 encrypted header, and a web API.

I will start pushing some PR next week hopefully, if these dependencies are not an issue. I believe on MacOS and Linux that shall be ok, but I don't know if they are part of the Windows build or not, as I don't know anything regarding that O/S.

[bernd]

ping

[DeepSOIC]

I'm not sure it's relevant, but I have made some python hacks to build FCStd files on the fly (in memory) from selected objects, write them to disk or potentially wherever python allows, read out specific objects from existing FCStd files into current project or update them or even write them back into the file. It's a bit unfinished... it's in Part-o-magic, called FilePlant.
Currently, there are two features that use the functionality:
* FCStd file export format (currently registers itself as FCStd1, due to FC not letting its FCStd extension through into the dialog)
* object duplication utility
Having something like that supported directly by FreeCAD would be nice.

FilePlant is actually a more-or-less self-contained python module, not really using much FreeCAD at all.

Also, Realthunder's branch has something like that already, I just don't quite know what it's capable of.

[DeepSOIC]

Here, I wrote a quick feature list and how-to:
https://github.com/DeepSOIC/Part-o-magi ... /FilePlant

[sgrogan]

I will start pushing some PR next week hopefully, if these dependencies are not an issue. I believe on MacOS and Linux that shall be ok, but I don't know if they are part of the Windows build or not, as I don't know anything regarding that O/S.

If you can post I link to your branch, I can try. The necessary stuff isn't in the Win Libpack now, but a quick DuckDuckGo suggests it should be possible, in release mode anyway.

[vejmarie]

Thanks for your answer. I am expecting to publish a branch by the end of the week (I hope), or within the next couple of days for the writer. It is currently working with min.io on MacOS and linux.

Stay tune !

[vejmarie]

Here, I wrote a quick feature list and how-to:
https://github.com/DeepSOIC/Part-o-magi ... /FilePlant

Thanks that will be super useful for the server part of the work I did initiated . I have a basic stupid dream to build a github for open hardware, and with some stuff I worked on with node-occ we could have "quickly" the capacity to publish a public database of model directly coming from published file, with preview, versioning and feedbacks, which will be my first implementation step.