Travis build failure

[triplus]

Reference:

https://forum.freecadweb.org/viewtopic.php?f=4&t=39596

[triplus]

It turned out GCC/Py2/Qt4 build needed the least amount of effort. Therefore i have left it in the build matrix for now. If some trivial bug lands, that prevents the compiling, it still can get fixed for now. At any point in the future it can get removed from the build matrix.

And this is how it looks like in the real world:

https://forum.freecadweb.org/viewtopic. ... 20#p338341

Anyway i noticed Windows builds are currently failing due to:

No space left on device

This issue started to occur after:

https://travis-ci.org/FreeCAD/FreeCAD/builds/592550970

Therefore i wonder if we are really exceeding the space quota or a glitch is related to something else. Maybe cache should get cleared, as a first step, if the path related fix shouldn't be related.

P.S. I noticed Windows builds can sometimes get closely to 2 hrs 30 min limit. Somebody had to ask for more time in the past, and as nobody volunteered i might write an email in the following days, asking if 3 h would be possible. That would make us more future proof.

[triplus]

Good news, as FreeCAD project was granted an additional build time. Build time limit should now be set at 180 min. Lets try it out:

https://github.com/FreeCAD/FreeCAD/pull/2596

[triplus]

OK, it looks like things are working good again. I worry a bit, as GCC builds on Linux already take around 2 h 50 min, that is close to 3 h. It would be a shame if we would have to result to non-standard toolchain on Linux (Clang), due to GCC being slower ATM.

There is an issue with macOS builds, as i don't have any experience with macOS platform, therefore can't sort it out for now. It looks like some dependencies are not meet.

Most of Python developers don't care about Python 2 anymore. Should i turn off the Py2/Qt4 build from the matrix? Concurrent number of build jobs is limited and if we have less of them enabled, new PR should be scheduled to build faster. There is a Python: 3.7 related option in the matrix, failing all the time. Not sure what it is about. Best to turn it off?

On Linux side we build (Py3/Qt5) by using both Clang/GCC toolchain. As both are working good ATM i am wondering if it makes more sense to have both enabled, or would it make more sense to have only one, and to achieve faster PR schedule, due to less concurrent build jobs needed?

[wmayer]

Most of Python developers don't care about Python 2 anymore. Should i turn off the Py2/Qt4 build from the matrix? Concurrent number of build jobs is limited and if we have less of them enabled, new PR should be scheduled to build faster. There is a Python: 3.7 related option in the matrix, failing all the time. Not sure what it is about. Best to turn it off?

I have no strong opinion on this. We could still let it run until Py2 officially expires but then switch it off.

On Linux side we build (Py3/Qt5) by using both Clang/GCC toolchain. As both are working good ATM i am wondering if it makes more sense to have both enabled, or would it make more sense to have only one, and to achieve faster PR schedule, due to less concurrent build jobs needed?

I prefer to use both compilers as I consider it a good thing to improve the code quality. The one compiler finds a lot of issues the other doesn't find and vice-versa.

[yorik]

My opinion regarding py2 is also let it die peacefully when the time comes... Let's support it as long as it doesn't require too much extra work or prevent other things...

[sgrogan]

Good news, as FreeCAD project was granted an additional build time. Build time limit should now be set at 180 min. Lets try it out:

Thanks! Great! Did the Travis people give any feedback?

I vote strongly for disabling the "Allowed Failures" builds. We already did this for the OSX build that didn't allow a failure. Anyone that wants to work on this can re-enable them as part of their pull request that driggers a build.

[wmayer]

For some time the Windows builds fail on travis because apparently not enough space is left:

C:/Users/travis/build/FreeCAD/FreeCAD/build/src/Mod/TechDraw/Gui/Release/TechDrawGui.exp
TechDrawGui.vcxproj -> C:\Users\travis\build\FreeCAD\FreeCAD\build\Mod\TechDraw\TechDrawGui.pyd
Building Custom Rule C:/Users/travis/build/FreeCAD/FreeCAD/CMakeLists.txt
clcache statistics:
current cache dir : Disk cache at C:\Users\travis\clcache
cache size : 114,912,404 bytes
maximum cache size : 1,073,741,824 bytes
cache entries : 325
cache hits : 570
cache misses
total : 325
evicted : 0
header changed : 296
source changed : 29
passed to real compiler
called w/ invalid argument : 0
called for preprocessing : 0
called for linking : 0
called for external debug : 0
called w/o source : 0
called w/ multiple sources : 0
called w/ PCH : 30
Tests failed!

The command "if [ "${TRAVIS_OS_NAME}" == "windows" ]; then

# call msbuild using clcache
#cmd.exe /C '"C:\Program Files (x86)\Microsoft Visual Studio\2017\BuildTools\VC\Auxiliary\Build\vcvarsall.bat" amd64 && MSBuild.exe FreeCAD.sln /p:CLToolExe=clcache.exe /p:TrackFileAccess=false /p:CLToolPath=C:\Users\travis\build\FreeCAD\FreeCAD /m:2 /nologo /verbosity:minimal /p:Configuration=Release /p:Platform=x64'

cmd.exe /C 'C:\Users\travis\build\FreeCAD\FreeCAD\.travis\build.bat'

# ls -lahR
cp -a ../FreeCADLibs/bin/* bin/
# show clcache hit stats
cmd.exe /C 'C:\Users\travis\build\FreeCAD\FreeCAD\clcache.exe -s'
# run the tests
#cmd.exe /C 'cd C:\Users\travis\build\FreeCAD\FreeCAD\build\bin && FreeCADCmd.exe --run-test 0'
# Make build fail if ANY of the following fails
#set -ev
winpty.exe -Xallow-non-tty -Xplain /C/Users/travis/build/FreeCAD/FreeCAD/build/bin/FreeCADCmd.exe --run-test 0 | tee runlog.txt
grep -q FAILED runlog.txt ; [ $? == 1 ] && echo "Build succeeded and tests passed!" || ( echo "Tests failed!" && false )

else
# Stop compiling (GCC) after 2 hrs 15 min (2 hrs 30 min limit). Extended to 180 min.
# Preserves created ccache for the next build job.
# if [ "${TRAVIS_OS_NAME}" == "linux" ]; then sudo timeout -k 140m 135m make -j2 install || true; fi
# if [ "${TRAVIS_OS_NAME}" == "osx" ]; then sudo make -j2 install; fi
sudo make -j2 install
${INSTALLED_APP_PATH} --console --run-test 0
${INSTALLED_APP_PATH} --log-file /tmp/FreeCAD_installed.log &
sleep 10 && pkill FreeCAD
cat /tmp/FreeCAD_installed.log
grep --file=../.log_errors /tmp/FreeCAD_installed.log ; [ $? == 1 ] && echo "No errors from .log_errors file found in the log after start from /usr/local/bin" || ( echo "Error from .log_errors found!" && false )

fi

" exited with 1.
cache.2

store build cache

6.53scp: error writing 'bin/libs/libeay.lib': No space left on device
cp: error writing 'bin/libs/libeay_d.lib': No space left on device
cp: error writing 'bin/libs/liblzma.lib': No space left on device
cp: error writing 'bin/libs/liblzma_d.lib': No space left on device
cp: error writing 'bin/libs/python3.lib': No space left on device
cp: error writing 'bin/libs/python36.lib': No space left on device
cp: error writing 'bin/libs/python36_d.lib': No space left on device
cp: error writing 'bin/libs/python3stub.lib': No space left on device
cp: error writing 'bin/libs/python3_d.lib': No space left on device
...
cp: cannot create regular file 'bin/vtksys-8.2_d.dll': No space left on device
cp: cannot create regular file 'bin/vtktheora-8.2.dll': No space left on device
cp: cannot create regular file 'bin/vtktheora-8.2_d.dll': No space left on device
cp: cannot create regular file 'bin/vtktiff-8.2.dll': No space left on device
cp: cannot create regular file 'bin/vtktiff-8.2_d.dll': No space left on device
cp: cannot create regular file 'bin/vtkverdict-8.2.dll': No space left on device
cp: cannot create regular file 'bin/vtkverdict-8.2_d.dll': No space left on device
cp: cannot create regular file 'bin/vtkzlib-8.2.dll': No space left on device
cp: cannot create regular file 'bin/vtkzlib-8.2_d.dll': No space left on device
cp: cannot create regular file 'bin/windeployqt.exe': No space left on device
cp: cannot create regular file 'bin/wish86t.exe': No space left on device
cp: cannot create regular file 'bin/xerces-c_3_2.dll': No space left on device
cp: cannot create regular file 'bin/xerces-c_3_2D.dll': No space left on device
cp: cannot create regular file 'bin/xmdump2': No space left on device
cp: cannot create regular file 'bin/xmdump3': No space left on device
cp: cannot create regular file 'bin/xmlpatterns.exe': No space left on device
cp: cannot create regular file 'bin/xmlpatternsvalidator.exe': No space left on device
cp: cannot create regular file 'bin/zlib.dll': No space left on device
cp: cannot create regular file 'bin/zlib_D.dll': No space left on device
cp: cannot create regular file 'bin/zlibd.dll': No space left on device
tee: runlog.txt: No space left on device
Assertion failed!

Program: C:\program files\git\usr\bin\winpty.exe
File: src/libwinpty/winpty.cc, Line 924
Expression: ASSERT_CONDITION("wp != nullptr && cols > 0 && rows > 0")
grep: runlog.txt: No such file or directory
/c/Users/travis/.casher/bin/casher: line 3: cannot create temp file for here-document: No space left on device
/c/Users/travis/.casher/bin/casher: line 229: /c/Users/travis/.casher/cache-windows-1803-containers-8235d25d013123a8bb2b75ec5e47cbd26ef43f506d168f44e72b4afc3bddcb56--compiler-gpp.md5sums_before.sorted: No space left on device
/c/Users/travis/.casher/bin/casher: line 230: /c/Users/travis/.casher/cache-windows-1803-containers-8235d25d013123a8bb2b75ec5e47cbd26ef43f506d168f44e72b4afc3bddcb56--compiler-gpp.md5sums_after: No space left on device
nothing changed
Done. Your build exited with 1.
diff: /c/Users/travis/.casher/cache-windows-1803-containers-8235d25d013123a8bb2b75ec5e47cbd26ef43f506d168f44e72b4afc3bddcb56--compiler-gpp.md5sums_before.sorted: No such file or directory
diff: /c/Users/travis/.casher/cache-windows-1803-containers-8235d25d013123a8bb2b75ec5e47cbd26ef43f506d168f44e72b4afc3bddcb56--compiler-gpp.md5sums_after: No such file or directory

/c/Users/travis/.casher/bin/casher: line 232: /c/Users/travis/.casher/checksum_diff: No space left on device

Any idea by what exactly this is caused and how it can be fixed?

[sgrogan]

Any idea by what exactly this is caused and how it can be fixed?

Each pull request gets it's own cache, that appears to be a delta to master's cache. When there are too many pull requests we get this error on Windows (I suspect that the Travis Win workers have less resources available)

I deleted all repository caches and re-started the master build. This fixes it. All pull request build also need to be re-started, which I didn't do because we would be back in the same situation.

I noticed that most of the "delta" caches are nearly as large as the master cache. I've also noticed that when running incremental builds locally it seems that more has to be built than in the past. I know this is related to the code that has changed.

[wmayer]

Thanks!