Travis build failure

Having trouble installing or compiling FreeCAD? Get help here.
User avatar
Kunda1
Posts: 5915
Joined: Thu Jan 05, 2017 9:03 pm

Re: Travis build failure

Postby Kunda1 » Sat Oct 19, 2019 4:06 pm

So maybe to avoid this bottleneck have a soft recommendation that trivial PRs could [skip ci]?
Want to contribute back to FC? Checkout:
#lowhangingfruit | Use the Source, Luke. | How to Help FreeCAD | How to report FC bugs and features
wmayer
Site Admin
Posts: 14991
Joined: Thu Feb 19, 2009 10:32 am

Re: Travis build failure

Postby wmayer » Sat Oct 19, 2019 4:27 pm

Kunda1 wrote:
Sat Oct 19, 2019 4:06 pm
So maybe to avoid this bottleneck have a soft recommendation that trivial PRs could [skip ci]?
Yes, this makes sense to add it more often because there is no point in recompiling the sources up to 10+ times per day.
wmayer
Site Admin
Posts: 14991
Joined: Thu Feb 19, 2009 10:32 am

Re: Travis build failure

Postby wmayer » Sun Oct 20, 2019 8:52 am

I deleted all repository caches and re-started the master build. This fixes it. All pull request build also need to be re-started, which I didn't do because we would be back in the same situation.
Somehow the problem seems to be back. From the build log I saw that:
  1. all dlls of the libpack are copied to the build directory
  2. the libpack also contains the debug dlls (and possibly all the .pdb) which can take up to several GB
If it's possible to set the PATH environment to point to the libpack's bin directory then there is no need to copy over any dlls. Then I don't think we will ever need to build a debug version on travis so that a stripped down libpack should be used there.
User avatar
sgrogan
Posts: 5468
Joined: Wed Oct 22, 2014 5:02 pm

Re: Travis build failure

Postby sgrogan » Sun Oct 20, 2019 3:03 pm

wmayer wrote:
Sun Oct 20, 2019 8:52 am
Then I don't think we will ever need to build a debug version on travis so that a stripped down libpack should be used there.
Good idea, I will make a stripped down libpack.
User avatar
sgrogan
Posts: 5468
Joined: Wed Oct 22, 2014 5:02 pm

Re: Travis build failure

Postby sgrogan » Mon Oct 21, 2019 9:51 pm

wmayer wrote:
Sun Oct 20, 2019 8:52 am
the libpack also contains the debug dlls (and possibly all the .pdb) which can take up to several GB
Moved here: https://forum.freecadweb.org/viewtopic. ... 87#p342387
triplus
Posts: 8786
Joined: Mon Dec 12, 2011 4:45 pm

Re: Travis build failure

Postby triplus » Tue Oct 22, 2019 10:25 am

wmayer wrote:
Wed Oct 09, 2019 8:08 am
I have no strong opinion on this. We could still let it run until Py2 officially expires but then switch it off.
I prefer to use both compilers as I consider it a good thing to improve the code quality. The one compiler finds a lot of issues the other doesn't find and vice-versa.
yorik wrote:
Wed Oct 09, 2019 1:36 pm
My opinion regarding py2 is also let it die peacefully when the time comes... Let's support it as long as it doesn't require too much extra work or prevent other things...
OK.
sgrogan wrote:
Wed Oct 09, 2019 10:13 pm
Thanks! Great! Did the Travis people give any feedback?
Yes. They resolved the issue without any complications. There was a note on 180 min being a hard limit.
I vote strongly for disabling the "Allowed Failures" builds. We already did this for the OSX build that didn't allow a failure. Anyone that wants to work on this can re-enable them as part of their pull request that driggers a build.
I see this has already happened. Yes, basically we would need somebody experienced on macOS side, to take care of it.

As for Windows builds and space limit. If everything else fails, maybe i could ask if they can grant a bit more. Anyway, i noticed 2 additional issues. First one being Linux GCC builds don't use ccache anymore, at all. All look like full builds. @sgrogan, could you clear all cache you can
and lets see what happens after. And i noticed GCC builds can already exceed 180 min limit. We will likely need to do something about that, as the time is only increasing. I read about possibilities and as we use Boost (template heavy) and have a large code base, a lot of times it helps if precompiled headers are provided. Did anybody experiment in this area in the past already?
User avatar
sgrogan
Posts: 5468
Joined: Wed Oct 22, 2014 5:02 pm

Re: Travis build failure

Postby sgrogan » Tue Oct 22, 2019 10:07 pm

triplus wrote:
Tue Oct 22, 2019 10:25 am
First one being Linux GCC builds don't use ccache anymore, at all. All look like full builds.
I don't think I did this with my git commit e31f84f
There was already an osx brew build that was commented out. I didn't touch this.
There was an additional osx build, that allowed failures, that tried to use py2/qt5 that I deleted, as this will never work.
There was also a py3.7 xenial build, with allowed failures, that was trying to run some flake8 tests, I only commented this out because it may be able to make it work.
triplus wrote:
Tue Oct 22, 2019 10:25 am
@sgrogan, could you clear all cache you can
and lets see what happens after.
I did this earlier in this thread. I deleted all caches. The Win and clang builds passed and the Linux builds timed out. I restarted the builds and all builds passed. Quickly after, maybe 1-3 pull requests, the Win space error reappeared. (I did this after the time limit was increased, I think)
triplus wrote:
Tue Oct 22, 2019 10:25 am
I read about possibilities and as we use Boost (template heavy) and have a large code base, a lot of times it helps if pre-compiled headers are provided. Did anybody experiment in this area in the past already?
abdullah and wmayer implemented PCH, when abdullah migrated the win build from AppVeyor to Travis, on win at least it is the default. Locally I don't see a big difference when it's enabled or not. I think this is a windows only thing?

I'm looking into adding the Libpack to PATH: as wmayer pointed out.

Does anyone have a good link to clcache, what exactly is stored in the cache? I'm wondering if maybe we are adding the libpack/bin to the cache every time we copy it rather than overwriting it (time stamps or something).
wmayer
Site Admin
Posts: 14991
Joined: Thu Feb 19, 2009 10:32 am

Re: Travis build failure

Postby wmayer » Wed Oct 23, 2019 8:53 am

I think this is a windows only thing?
No, PCH is also supported by gcc.
triplus
Posts: 8786
Joined: Mon Dec 12, 2011 4:45 pm

Re: Travis build failure

Postby triplus » Wed Oct 23, 2019 11:15 am

sgrogan wrote:
Tue Oct 22, 2019 10:07 pm
I don't think I did this with my git commit e31f84f
No, that shouldn't have such effect. I suspect two things. First one being a "corrupted cache" and the second one might be Travis people changed some setting, when increasing time limit, and it's affecting ccache for GCC.
I did this earlier in this thread. I deleted all caches.
Could you stop all builds, delete all cache and don't trigger any rebuilds? After lets see what a few new PR/merges will behave like. I can ask Travis people for more assistance after, based on the results.

P.S. Don't know if i will have much time before Monday, by then the results should be known.
wmayer
Site Admin
Posts: 14991
Joined: Thu Feb 19, 2009 10:32 am

Re: Travis build failure

Postby wmayer » Wed Nov 13, 2019 7:34 am

Does anybody have a clue why on Windows the FEM unit tests often fail and sometimes don't. Could this be a caching problem?
When looking at the build log you can see that it copies the Python files to the target location but it never builds the binaries from the C++ code.