creating source releases

This subforum is specifically to discuss packaging issues on different platforms (windows, mac, linux), and using different packaging systems (conda, etc...)
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

creating source releases

Post by looo »

As I am running into difficulties with using git to fetch the source code of freecad, I would like to try a separate step to create daily source releases with an additional version file that contains the necessary git-information.

The github action to generate the version file and the compressed source code is already done. My question now is, how to use the version-file in the build process? And which information do we need? Currently the version file looks like this:

Code: Select all

rev_number: 26225
branch_name: master
commit_date: 2021/10/26 12:43:47
commit_hash: 4c754f7af7
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

Re: creating source releases

Post by looo »

Is it possible to create the version.hxx file within the GitHub action? I guess all information should be available. So the task is about inserting this information into the version.hxx file. But I am not sure how it is done currently. @wmayer can you briefly explain how this process is currently done with cake?
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

Re: creating source releases

Post by looo »

Code: Select all

// Version Number
#define FCVersionMajor "${PACKAGE_VERSION_MAJOR}"
#define FCVersionMinor "${PACKAGE_VERSION_MINOR}"
#define FCVersionName  "${PACKAGE_VERSION_NAME}"
// test: $Format:Hash (%H), Date: %ci$
#define FCRevision      "26232"      //Highest committed revision number
#define FCRevisionDate  "Hash: (cd8ce95f53), Date: 2021/10/27 19:26:26"     //Date of highest committed revision
#define FCRepositoryURL "${PACKAGE_WCURL}"      //Repository URL of the working copy
Not sure if this is correct for the version.h file.
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

Re: creating source releases

Post by looo »

resulting in this version info:

Code: Select all

OS: Ubuntu 18.04.6 LTS (ubuntu:GNOME/ubuntu)
Word size of FreeCAD: 64-bit
Version: 0.20.26232
Build type: Release
Branch: (HEAD detached at 349f90c)
Hash: 349f90c019f85ded8765929b2447ecf6d46fa7b7
Python version: 3.9.7
Qt version: 5.12.9
Coin version: 4.0.0
OCC version: 7.5.3
Locale: German/Germany (de_DE)
Attachments
Bildschirmfoto von 2021-10-27 21-57-10.png
Bildschirmfoto von 2021-10-27 21-57-10.png (45.31 KiB) Viewed 4697 times
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

Re: creating source releases

Post by looo »

this was the output before I switched to the prefilled git-information. (Just as a reminder for myself)

Code: Select all

OS: Ubuntu 18.04.6 LTS (ubuntu:GNOME/ubuntu)
Word size of FreeCAD: 64-bit
Version: 0.20.26155 (Git)
Build type: Release
Branch: master
Hash: 0926a4148bcff11249fd4f56bc30256102ffe105
Python version: 3.9.7
Qt version: 5.12.9
Coin version: 4.0.0
OCC version: 7.5.3
Locale: German/Germany (de_DE)
Attachments
Bildschirmfoto von 2021-10-29 21-47-16.png
Bildschirmfoto von 2021-10-29 21-47-16.png (39.81 KiB) Viewed 4589 times
User avatar
adrianinsaval
Veteran
Posts: 5541
Joined: Thu Apr 05, 2018 5:15 pm

Re: creating source releases

Post by adrianinsaval »

question about versioning in general, wouldn't it be better to explicitly name it 0.20.0.xxxxx? this way keeping track of what is newer can be more simple with future point releases (for example 0.20.27000 can be interpreted by software as a newer version than 0.20.1.27010 since 27000>>1)
wmayer
Founder
Posts: 20243
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: creating source releases

Post by wmayer »

The Version.h file will be created when running the build process by executing the Python script SubWCRev.py.

At the times when we used automake & friends there was a target make dist to create source tarballs. With the move to CMake the creation of clean source tarballs have become way more difficult because you need to maintain a blacklist of files you don't want to be added. And this is impossible to do reliably because you have no control of which files you have in the source directory.

Luckily with the move from svn to git we can use the fact that it will ignore all files that are no part of the repository and additionally with .gitattributes we can ignore certain files of the repository.

To create source tarballs this way I once wrote the script makedist.py and the CMake targets dist-git and distdfsg-git.

I have tried it right now and unfortunately it doesn't work any more at the moment. I guess it still needs to be ported to Py3.
User avatar
looo
Veteran
Posts: 3941
Joined: Mon Nov 11, 2013 5:29 pm

Re: creating source releases

Post by looo »

wmayer wrote: Sat Oct 30, 2021 8:44 am The Version.h file will be created when running the build process by executing the Python script SubWCRev.py.
I tried this approach and created a pyhon file that is called inside the github-action. Not sure about what else is needed to set the version correctly.
https://github.com/FreeCAD/FreeCAD-Bund ... on_file.py

so the github action pulls the FreeCAD repo with git, adds the git information to the Version.h.cmake, commits the changes and creates a tar.gz with git archive. Not sure if there are additional tasks that need to be part of this procedure.
wmayer wrote: Sat Oct 30, 2021 8:44 am Luckily with the move from svn to git we can use the fact that it will ignore all files that are no part of the repository and additionally with .gitattributes we can ignore certain files of the repository.

To create source tarballs this way I once wrote the script makedist.py and the CMake targets dist-git and distdfsg-git.

I have tried it right now and unfortunately it doesn't work any more at the moment. I guess it still needs to be ported to Py3.
Thanks for the information. I will have a look at it.
wmayer
Founder
Posts: 20243
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: creating source releases

Post by wmayer »

FYI, the target dist-git is defined in CreatePackagingTargets.cmake and looks like:

Code: Select all

    add_custom_target(dist-git
        COMMAND ${PYTHON_EXECUTABLE} ${CMAKE_SOURCE_DIR}/src/Tools/makedist.py
              --srcdir=${CMAKE_SOURCE_DIR} --bindir=${CMAKE_BINARY_DIR}
              WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
    )
and the generated build.make file has this block:

Code: Select all

CMakeFiles/dist-git:
	cd /home/user/Projects/FreeCAD && /usr/bin/python3.6 /home/user/Projects/FreeCAD/src/Tools/makedist.py --srcdir=/home/user/Projects/FreeCAD --bindir=/home/user/Projects/FreeCAD/build
And this is the error message I get:

Code: Select all

  File "/home/user/Projects/FreeCAD/src/Tools/makedist.py", line 107, in <module>
    main()
  File "/home/user/Projects/FreeCAD/src/Tools/makedist.py", line 80, in main
    tardata = os.popen("git archive {} --prefix={}/ HEAD".format(wta, DIRNAME)).read()
  File "/usr/lib/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb0 in position 445451: invalid start byte
CMakeFiles/dist-git.dir/build.make:57: recipe for target 'CMakeFiles/dist-git' failed
make[3]: *** [CMakeFiles/dist-git] Error 1
CMakeFiles/Makefile2:99: recipe for target 'CMakeFiles/dist-git.dir/all' failed
make[2]: *** [CMakeFiles/dist-git.dir/all] Error 2
CMakeFiles/Makefile2:106: recipe for target 'CMakeFiles/dist-git.dir/rule' failed
make[1]: *** [CMakeFiles/dist-git.dir/rule] Error 2
Makefile:177: recipe for target 'dist-git' failed
make: *** [dist-git] Error 2
wmayer
Founder
Posts: 20243
Joined: Thu Feb 19, 2009 10:32 am
Contact:

Re: creating source releases

Post by wmayer »

looo wrote: Sat Oct 30, 2021 9:41 am Thanks for the information. I will have a look at it.
git commit c5fb91170b
Post Reply