Starting parts

[jcc242]

I have a friend who is working on a clone of thingiverse. He mentioned open sourcing the code when he was done. A website like that would allow people to easily upload their own parts and profiles, and I think he was working on a webGL (or something?) viewer for .stl parts. I haven't talked to him in a while, but I could touch base with him and see how it is coming along and if we could make this a long-term goal.

You know what would be really cool? A website frontend that stores all the parts in a git repo so you could easily download and keep updated with all the parts.

Can you selectively download files from a git repo? If not and the repo grows humungous, then everyone's harddrive space will be filled with parts they may not want or need.

[shoogen]

Can you selectively download files from a git repo? If not and the repo grows humungous, then everyone's harddrive space will be filled with parts they may not want or need.

You can do "sparse checkout"s which still means that you have the whole repository, but things you don't need are not in your working copy and therefore might benefit from compression.
You can do a "shallow clone" which means that you get only the last revisions of each file.
And you can split up the project into "submodules". But i think none of the above will perfectly fit the problem.
IMHO the FreeCAD Resource Project should manage a local cache. And download missing Parts from a server.
Unless the part library stays smaller than freecad by itsself, i don't see a problem.
And on the day it grows to big, will get the FreeCAD Resource Project ready.

[NormandC]

I have a friend who is working on a clone of thingiverse.

LOL, another one. Problem is, there are too many - and none is getting it right. Or having success...

I think he was working on a webGL (or something?) viewer for .stl parts.

There are already a couple open source STL viewers, I hope he's not reinventing the wheel.

You know what would be really cool? A website frontend that stores all the parts in a git repo so you could easily download and keep updated with all the parts.

Already done.

https://cubehero.com/
http://www.fabfabbers.com/

Can you selectively download files from a git repo?

It's possible from Github, but not really user friendly.

[yorik]

Okay I created a repo here (on my account first, when it grows we fork it on the official freecad account): https://github.com/yorikvanhavre/FreeCAD-library I propose to store stuff there to begin with, and it will always be time to move to some better system when we have one. I also gave write access to rockn, normandc and shoogen. Anyone else interested? Just ask me.

Basically, for who is new to git and wants to contribute (after gaining write access), you must install a git client on your machine, then you sync to the github repo (this will create a local copy of the files on your machine), then you make changes, add files, etc, then commit ("register") the changes, then sync again, which will upload your changes to the repo. But just ask here and we'll help you.

Ah github has an embedded stl viewer now I believe...

[shoogen]

Great Yorik, thank you.
Does it make any sense to save all the FCStd files uncompressed (level 0), to allow git use do it's delta compression? This might help to keep the repository smaller, while the working copy gets bigger.
The background is that ZIP-compression only works a single item (like a shape in the FCStd file) while the delta compression of git should benefit from multiple shapes in a FCStd file and from multiple revisions.

[rockn]

Can you selectively download files from a git repo?

It's possible from Github, but not really user friendly.

Yes you have to right-click and "download the link target" otherwise github displays the file as text.
Thanks yorik, I would try to push something later.

[jcc242]

I have a friend who is working on a clone of thingiverse.

LOL, another one. Problem is, there are too many - and none is getting it right. Or having success...

I think he was working on a webGL (or something?) viewer for .stl parts.

There are already a couple open source STL viewers, I hope he's not reinventing the wheel.

You know what would be really cool? A website frontend that stores all the parts in a git repo so you could easily download and keep updated with all the parts.

Already done.

https://cubehero.com/
http://www.fabfabbers.com/

Can you selectively download files from a git repo?

It's possible from Github, but not really user friendly.

Ah, I obviously hadn't done any research. I didn't know other clones existed or stl viewers and stuff. Well I look forward to the library though, thank you for working on it everyone

[yorik]

Does it make any sense to save all the FCStd files uncompressed (level 0), to allow git use do it's delta compression? This might help to keep the repository smaller, while the working copy gets bigger.

That's interesting... Would it work? I'm not sure that even with 0 compression it won't still be a binary blob with no possibility of diff... I already did some experience with uncompressing fcstd files (inside, it's almost only text, since brep files are actually text files!) but it is a bit complicated, and might be much hassle for such simple files... I believe those Parts won't get many versions...

[shoogen]

That's interesting... Would it work? I'm not sure that even with 0 compression it won't still be a binary blob with no possibility of diff...

Yes, it's still one continuous blob. But I'd like to distinguish three aspects. diffs, patches, and (delta) compression. There basicly two types of diffs those, that can be applied as a patch and those you don't contain enough information to be used as a patch. A (unified) diff on sourcode can be used as a patch. Git can also generate binary diffs that can be used as a patch.
You can as well use external diff tools or convert the whole file to a textual representation and to a diff on the text.
For example the plain text of a MS Word Doc could be extracted or the EXIF info of a jpeg-File.
Those diffs are useful when looking at commits but they can't be used to patch the original file.
The (delta) compression works on binary files as it works on text files. i search for patterns that are reused between different files and versions. But on already compressed files (like FCStd) i won't find anything useful,unless a subfile (e.g. a shape) is exactly the same, and the (zip/deflate) compression parameters are exactly the same.

[yorik]

Wow very interesting explanation. I'll definitely do some testing with fcstd files... Indeed if this can reduce the amount of data stored for each version, that can be tremendously interesting, imagine that with huge projects...