Travis build failure

[triplus]

As for some additional future proofing (Windows builds and space) and as asked here:

https://github.com/FreeCAD/FreeCAD/pull/2656

I am not all that familiar with the Windows build procedure, but is this correct:

• Downloading and extracting FreeCADLibs.7z
• Using the downloaded LibPack as compile dependencies
• Copying (cp -a ../FreeCADLibs/bin/* bin/) step is needed to satisfy runtime dependencies

If that is correct, does anything get written to bin/ at compile stage? Or can /bin be deleted before copying the runtime dependencies? The reason i ask is moving is much more involved command, compared to copying command. And the easiest solution would be to delete the bin/ before moving.

[sgrogan]

If that is correct, does anything get written to bin/ at compile stage?

This is correct, at compile time the FreeCAD(cmd).exe and the FreeCAD .dll's/.pyd's (,so files on linux), (I think internal SMESH stuff as well)
See this closed pull request experimenthttps://github.com/FreeCAD/FreeCAD/pull/2661
It only copies the stuff from the libpack that is not already in bin. I think with a working cache it should copy nothing as the file already exist from the cache. If the cache has been deleted it should copy the necessary files.

[triplus]

This is correct, at compile time the FreeCAD(cmd).exe and the FreeCAD .dll's/.pyd's (,so files on linux), (I think internal SMESH stuff as well)

Thanks for confirming it. Therefore the bin/ folder is not empty and as such can't be deleted.

See this closed pull request experimenthttps://github.com/FreeCAD/FreeCAD/pull/2661

Note that it failed. If you take a look at the build job log file, -a option is unknown. In addition mv command likely has issues, bin/ folder not being empty. That is likely on why cp command got used in the first place.

It only copies the stuff from the libpack that is not already in bin. I think with a working cache it should copy nothing as the file already exist from the cache. If the cache has been deleted it should copy the necessary files.

The way i understand it $HOME/clcache directory gets uploaded, not sure about the rest. That is why i created a PR, to delete the LibPack. As LibPack and cache both weight around 400MB. Deleting the LibPack provided just enough space for restored cache. In the future for sure we will need a bit more. I guess best to test (in PR) on just how big bin/ folder is before and after the cp command completes.

P.S. If anybody wants to do that, go ahead, i will have more time in the following days.

[sgrogan]

Note that it failed. If you take a look at the build job log file, -a option is unknown. In addition mv command likely has issues, bin/ folder not being empty. That is likely on why cp command got used in the first place.

Yes, and the tests passed. So the necessary is already in the cache.

The way I think it works? Not sure I think...

clone FreeCAD
Download the Libpack
Extract the Libpack to ? https://github.com/FreeCAD/FreeCAD/blam ... s.yml#L226
Delete the .7z compressed archive
Set some Path's so CMake can find the Libpack at compile time.
Build FreeCAD
Copy the Libpack/bin to Build/bin
Run tests
Store to cache???

[triplus]

The way it's currently set up is nothing but the ccache/clcache folders get cached. That is, if compiler cache was created in previous build, it gets downloaded and used for the current build. Right before the build job finishes, the compiler cache will get compressed and uploaded (cached) again. Now, as for the available space, here is the situation before and after cp command:

Code: Select all

25M	bin/
2.3G	bin/

I used mv instead of cp command and it looks like no "Directory not empty" issues involved for now. Therefore deleting the LibPack archive and using mv instead of cp, we gained a few GB of space for Windows builds, and are now a bit more future proof. In addition i made the build job to fail, if unit tests don't complete successfully, as discussed:

https://forum.freecadweb.org/viewtopic.php?f=10&t=40324

But now both Py3 (Clang/GCC) build fail, when trying to run unit tests, as indeed there is:

Code: Select all

Program received signal SIGSEGV, Segmentation fault.

Involved. Not sure if toolchain related or an actual issue to fix in FreeCAD.

All of this is done in this PR:

https://github.com/FreeCAD/FreeCAD/pull/2729

P.S. I asked Travis support, on why GCC builds don't use compiler cache anymore. I suspect some setting on their side could be involved, as our .travis.yml looks OK, and compiler cache for GCC builds isn't used anymore, since they increased build job time limit to 180 min. Likely we will get an answer early in the next week.

[triplus]

Often, when a new compiler release is made (GCC and Clang), release notes mention benefits, such as more verbose and useful output for detected issues ... Anyway, we could use newer version of compiler(s) on Travis, done here:

https://github.com/FreeCAD/FreeCAD/pull/2743

Now, as for Clang 9, that works. GCC 9 currently spams the log, some Eigen related info. And as the log is too big, the build will fail. If it wouldn't fail, i am not sure if we have enough time on Travis, for GCC 9 build to complete in less than 180 min. As somehow compilers are getting slower and slower, with each new release.

The question i am asking. Is there any interest in having GCC 9 and Clang 9 enabled on Travis? GCC 9 build log does look like it is producing more verbose info (not just Eigen related).

[howetuft]

GCC 9 currently spams the log, some Eigen related info.

This is no spam! Indeed, Eigen breaks the famous "rule-of-3" in some parts of its code, as it explicitly declares some copy-assignment constructors with no equivalent explicit copy constructors and this is no good practice (see https://en.cppreference.com/w/cpp/langu ... e_of_three, e.g.), so GCC legitimately complains about it with a warning. Problem is the warning is raised each time the faulty classes are instanciated, which is quite frequent in FreeCAD (and, I admit, GCC warnings messages are quite detailed...).

Is there any interest in having GCC 9 and Clang 9 enabled on Travis?

Yes, I think this should not stop us in using most recent versions of compilers: they give us a more accurate insight of code quality and can avoid us some tricky bugs.
Concerning Eigen, specifically: maybe, in the short term, we could enclose faulty Eigen classes instanciation with pragmas disabling the itchy warning, and ask upstream for the warning resolution?
Concerning compilation time: maybe the solution should not be to get rid of the compiler, but to optimize the code (use pimpl idiom, optimize headers dependencies...)? This can be quite a large refactoring work, but quite a worthy one also...

[wmayer]

Yes, I think this should not stop us in using most recent versions of compilers: they give us a more accurate insight of code quality and can avoid us some tricky bugs.

But the most recent compiler version can be used on a local system where things get built much faster than on travis.

And it's not always said that the compiler warnings are correct. Just a few weeks ago I had the case that clang was recommending to use a std::move() and a later gcc version then was complaining about the std::move().

[triplus]

@howetuft

Thanks for taking a look and providing a more in-depth analysis. Yes, i guess this is exactly what the drive behind the PR was. GCC 9 therefore is able to detect such coding practices and as such can contribute to writing better code. The problem with Eigen could have already been addressed upstream, but on Bionic an older version of the library is available. Anyway, i turned off GCC 9, and only left Clang 9 enabled, for now. Lets see how it will behave. We can try to make GCC 9 work in the future (addressing the Eigen issue).

@wmayer

With GCC builds we are basically already hitting the 180 min limit on Travis. We tried to address this in the past, but due to "randomness" in using ccache, the solution didn't work reliable enough, too much full builds. Hopefully this will change:

https://github.com/FreeCAD/FreeCAD/pull/2743

I added some uniqueness to cache entries. Travis people didn't provide a hint this time, @sgrogan-CI did.

[wmayer]

For around a week we have again some tricky failures on travis.
The Linux versions seem to compile and run fine but then they stop with an error at the end

--------------------------------------------------------------------
Ran 367 tests in 25.925s
OK
double free or corruption (!prev)
/home/travis/.travis/functions: line 136: 2302 Aborted (core dumped) ${INSTALLED_APP_PATH} --console --run-test 0

The Qt4 Linux version seem to lack of pivy and aborts earlier.

The Windows build seems to suffer from a different problem:

======================================================================

ERROR: test_5_Flow1D_thermomech_analysis (femtest.app.test_ccxtools.TestCcxTools
)
----------------------------------------------------------------------
Traceback (most recent call last):
File "C:\Users\travis\build\FreeCAD\FreeCAD\build\Mod\Fem\femtest\app\test_ccx
tools.py", line 448, in test_5_Flow1D_thermomech_analysis
flow1d(self.active_doc, "ccxtools")
File "C:\Users\travis\build\FreeCAD\FreeCAD\build\Mod\Fem\femexamples\thermome
ch_flow1d.py", line 76, in setup
from Draft import makeWire
File "C:\Users\travis\build\FreeCAD\FreeCAD\build\Mod\Draft\Draft.py", line 83
, in <module>
import draftutils.gui_utils
File "C:\Users\travis\build\FreeCAD\FreeCAD\build\Mod\Draft\draftutils\gui_uti
ls.py", line 37, in <module>
import FreeCADGui
ImportError: DLL load failed: The specified module could not be found.
======================================================================
ERROR: TestPathApp (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: TestPathApp
Traceback (most recent call last):
File "C:\Users\travis\build\FreeCAD\FreeCAD\build\bin\lib\unittest\loader.py",
line 153, in loadTestsFromName
module = __import__(module_name)
File "C:\Users\travis\build\FreeCAD\FreeCAD\build\Mod\Path\TestPathApp.py", li
ne 34, in <module>
from PathTests.TestPathDepthParams import depthTestCases
File "C:\Users\travis\build\FreeCAD\FreeCAD\build\Mod\Path\PathTests\TestPathD
epthParams.py", line 25, in <module>
import PathScripts.PathUtils as PU
File "C:\Users\travis\build\FreeCAD\FreeCAD\build\Mod\Path\PathScripts\PathUti
ls.py", line 36, in <module>
from PathScripts import PathJob
File "C:\Users\travis\build\FreeCAD\FreeCAD\build\Mod\Path\PathScripts\PathJob
.py", line 25, in <module>
import ArchPanel
File "C:\Users\travis\build\FreeCAD\FreeCAD\build\Mod\Arch\ArchPanel.py", line
22, in <module>
import FreeCAD,Draft,ArchComponent,DraftVecUtils,ArchCommands,math, Part, Ar
chNesting
File "C:\Users\travis\build\FreeCAD\FreeCAD\build\Mod\Draft\Draft.py", line 83
, in <module>
import draftutils.gui_utils
File "C:\Users\travis\build\FreeCAD\FreeCAD\build\Mod\Draft\draftutils\gui_uti
ls.py", line 37, in <module>
import FreeCADGui
ImportError: DLL load failed: The specified module could not be found.
----------------------------------------------------------------------
Ran 231 tests in 28.095s
FAILED (errors=2)
QWaitCondition: Destroyed while threads are still waiting
Tests failed!

I guess it fails to load FreeCADGui.pyd because of an resolved dependency, i.e. a 3rd party dll it cannot find any more. Were there any recent changes that possibly removed some files from the libpack there?

Btw when I ran the gcc/clang builds locally everything works fine and I don't get that error message at the end. Is it maybe possible that something on the travis environment is wrong?

However, when I ran the tests on a Windows machine I get two failures. The tests are testInsertRows (from the Spreadsheet module) and testBalloonCase (from the TechDraw module).

Here is some test code to reproduce the issues:

Code: Select all

import Spreadsheet
doc=FreeCAD.newDocument()
sheet = doc.addObject('Spreadsheet::Sheet','Spreadsheet')
sheet.set('B1', '=B2')
sheet.set('B2', '124')
sheet.getContents("B1"),"=B2"
sheet.insertRows('2', 1)
sheet.getContents("B1"),"=B3"
sheet.getContents("B1"),"=B3" # the returned value is =B2, not =B3

Code: Select all

from FreeCAD import Units

path = FreeCAD.getHomePath() + 'Mod/TechDraw/TDTest'
templateFileSpec = path + '/TestTemplate.svg'

doc = FreeCAD.newDocument("TDBalloon")

box = doc.addObject("Part::Box","Box")
sphere = doc.addObject("Part::Sphere","Sphere")

page = doc.addObject('TechDraw::DrawPage','Page')
template = doc.addObject('TechDraw::DrawSVGTemplate','Template')
template.Template = templateFileSpec
page.Template = template
page.Scale = 5.0

view1 = doc.addObject('TechDraw::DrawViewPart','View')
doc.View.Source = [box]
rc = page.addView(view1)
view1.X = Units.Quantity(30.0,Units.Length) # raises an exception because it doesn't accept a quantity but an int or float

EDIT:
The local test failure on Windows were false alarm. After rebuilding everything from scratch it works without problems now.