Reducing the size of AppImage

This subforum is specifically to discuss packaging issues on different platforms (windows, mac, linux), and using different packaging systems (conda, etc...)
triplus
Posts: 8888
Joined: Mon Dec 12, 2011 4:45 pm

Reducing the size of AppImage

Postby triplus » Fri Sep 06, 2019 6:08 pm

One of the tasks left to do is to investigate, if we can reduce the size of the AppImage, by removing some stuff not needed:

https://github.com/FreeCAD/FreeCAD-AppImage/issues/24

I decided to open a dedicated thread here on the forum. Anybody interested can poke around. Download the FreeCAD AppImage:

https://github.com/FreeCAD/FreeCAD/releases/tag/0.18.3

Use command:

Code: Select all

FreeCAD.AppImage --appimage-extract
And take a look. Look for any clues, try things out and ask or suggest after. Note that extracted AppImage can still be run, to test if removing something introduced a regression.
User avatar
sgrogan
Posts: 5547
Joined: Wed Oct 22, 2014 5:02 pm

Re: Reducing the size of AppImage

Postby sgrogan » Fri Sep 06, 2019 9:16 pm

triplus wrote:
Fri Sep 06, 2019 6:08 pm
I decided to open a dedicated thread here on the forum. Anybody interested can poke around. Download the FreeCAD AppImage:
Good thread.
I guess the first is the docs. I saw you other thread about the temp mount issue. The docs are 170MB binaries, so the compression doesn't help. Thanks for your effort on this, I understand better how the whole assistant works now because of your explaination.

As a point of reference the last Win conda .7z is 300 MB without docs. 7zip is around 30% smaller compression.

I can't test on linux now, but usr/cmake, usr/conda-meta, and any .h files should be safe to delete.
triplus
Posts: 8888
Joined: Mon Dec 12, 2011 4:45 pm

Re: Reducing the size of AppImage

Postby triplus » Fri Sep 06, 2019 9:23 pm

I am still working for a solution, regarding FreeCAD docs. I don't mind if it gets decided and removed after. But hopefully the solution i have in mind will pan out. Therefore the embedded FreeCAD docs to work as expected. As for docs of other libraries. I have seen some of that in the AppImage. I didn't remove them yet, as it wouldn't be fair, for me to do the easy part. ;)

Therefore FreeCAD docs are large indeed, but lets leave them in for now.
User avatar
sgrogan
Posts: 5547
Joined: Wed Oct 22, 2014 5:02 pm

Re: Reducing the size of AppImage

Postby sgrogan » Fri Sep 06, 2019 9:57 pm

triplus wrote:
Fri Sep 06, 2019 9:23 pm
Therefore FreeCAD docs are large indeed, but lets leave them in for now.
Fair enough. My main point is that the .AppImage (the .dmg is the same) isn't as bloated as it may appear. I created the Win conda packages by copying stuff out of the conda env until it didn't fail, so I don't think there is much extra. If you look at the difference, in size, between the Win installer and the portable package you will see a huge difference. The installer has small overhead, the majority of the difference is due to the compression.

For example FreeCAD_0.19.18024_x64_LP_12.1.2_PY3QT5-WinVS2015.7z is 271MB, FreeCAD_0.19.18024_x64_LP_12.1.2_PY3QT5-WinVS2015.zip is 440MB, same exact source directory only different compression. I downloaded the 0.18.3 .AppImage and extracted. Re-compressing with .zip is 564MB (vs 549 for the downloaded .AppImage) and .7z is 424MB. Based on all of this 40-50MB improvement is about the beat we can expect. Of course every little helps.

As you have mentioned before, finding a way to better use .AppImage.zsync probably offers more savings. How exactly to do this with tagged releases etc. I don't know :?
triplus
Posts: 8888
Joined: Mon Dec 12, 2011 4:45 pm

Re: Reducing the size of AppImage

Postby triplus » Fri Sep 06, 2019 10:35 pm

sgrogan wrote:
Fri Sep 06, 2019 9:57 pm
My main point is that the .AppImage (the .dmg is the same) isn't as bloated as it may appear.
I agree. There is FreeCAD, dependencies, documentation, popular (Python) libraries ... all included by default.
Based on all of this 40-50MB improvement is about the beat we can expect. Of course every little helps.
Yes, basically this effort could end up in realizing, nothing should be removed. That is OK too, but it needs to be explored.
The installer has small overhead, the majority of the difference is due to the compression.
Good point. Uncompressed folder, that likely can serve as a good reference. And when we had PPA based AppImage, it had some libraries included too. It was a bit smaller. I imagine due to Debian/Ubuntu packages being split up and only the necessary parts to be included. That is why i expect some size reduction will likely be possible.
As you have mentioned before, finding a way to better use .AppImage.zsync probably offers more savings. How exactly to do this with tagged releases etc. I don't know :?
This actually already works rather good. Whoever uses the updater, stable releases will always update to whatever is on the latest. Development releses won't automatically update from 0.19_pre to 1.0_pre, but that comes down to one full download, for each development cycle.
triplus
Posts: 8888
Joined: Mon Dec 12, 2011 4:45 pm

Re: Reducing the size of AppImage

Postby triplus » Fri Sep 06, 2019 11:17 pm

sgrogan wrote:
Fri Sep 06, 2019 9:16 pm
I can't test on linux now, but usr/cmake, usr/conda-meta, and any .h files should be safe to delete.

Code: Select all

du -sh squashfs-root/
1.5G	squashfs-root/

Code: Select all

du -sh usr/cmake/ usr/conda-meta/
8.0K	usr/cmake/
22M	usr/conda-meta/

Code: Select all

find . -name "*.h" | xargs du -sch
2.8M	total
It's a start!
User avatar
sgrogan
Posts: 5547
Joined: Wed Oct 22, 2014 5:02 pm

Re: Reducing the size of AppImage

Postby sgrogan » Sat Sep 07, 2019 2:41 pm

triplus wrote:
Fri Sep 06, 2019 11:17 pm
It's a start!
usr/doc/global
usr/lib/cmake
User avatar
Kunda1
Posts: 6223
Joined: Thu Jan 05, 2017 9:03 pm

Re: Reducing the size of AppImage

Postby Kunda1 » Sat Sep 28, 2019 3:55 pm

BTW, is this src/Mod/Ship code within the appimage ?
https://github.com/FreeCAD/freecad.ship/issues/8
Want to contribute back to FC? Checkout:
#lowhangingfruit | Use the Source, Luke. | How to Help FreeCAD | How to report FC bugs and features
vocx
Posts: 2450
Joined: Thu Oct 18, 2018 9:18 pm

Re: Reducing the size of AppImage

Postby vocx » Sat Sep 28, 2019 4:26 pm

Kunda1 wrote:
Sat Sep 28, 2019 3:55 pm
BTW, is this src/Mod/Ship code within the appimage ?
https://github.com/FreeCAD/freecad.ship/issues/8
vocx wrote: There doesn't seem to be any C++ code in the Ship workbench. If it is there still, it's because Werner and Yorik haven't made the decision to pull the plug and remove it.

If this is done finally, then some `CMakeLists.txt` files should be edited to remove the "compilation" of the workbench. In this case, compilation is just copying the files to the appropriate directories.
To support the documentation effort, and code development, your donation is appreciated: paypal.
User avatar
sgrogan
Posts: 5547
Joined: Wed Oct 22, 2014 5:02 pm

Re: Reducing the size of AppImage

Postby sgrogan » Sat Sep 28, 2019 4:32 pm

Kunda1 wrote:
Sat Sep 28, 2019 3:55 pm
BTW, is this src/Mod/Ship code within the appimage ?
Looks like it is. https://github.com/FreeCAD/FreeCAD_Cond ... d/build.sh

Ship is still built by default in FreeCAD master, it's also included in the PPA and Win builds.
Maybe as next step we could not build ship as default, and if there are no issues adding it with addons-manager, later we could remove the code entirely. I'll disable this in the next Win builds for testing. -DBUILD_SHIP=0