TestPathApp fails after CMake Install

[openBrain]

Hi all,

Not sure if this topic is better placed here or in Compile/Install.
Anyway.
I have an issue that TestPathApp fails after I build and 'cmake --install' FreeCAD, with following log :

Code: Select all

======================================================================
ERROR: TestPathApp (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: TestPathApp
Traceback (most recent call last):
  File "/usr/lib/python3.10/unittest/loader.py", line 154, in loadTestsFromName
    module = __import__(module_name)
  File "/usr/lib/python3/dist-packages/shiboken2/files.dir/shibokensupport/__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "/usr/local/Mod/Path/TestPathApp.py", line 25, in <module>
    from PathTests.TestPathAdaptive import TestPathAdaptive
  File "/usr/lib/python3/dist-packages/shiboken2/files.dir/shibokensupport/__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "/usr/local/Mod/Path/PathTests/TestPathAdaptive.py", line 27, in <module>
    import Path.Op.Adaptive as PathAdaptive
  File "/usr/lib/python3/dist-packages/shiboken2/files.dir/shibokensupport/__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
  File "/usr/local/Mod/Path/Path/__init__.py", line 1, in <module>
    from PathApp import *
  File "/usr/lib/python3/dist-packages/shiboken2/files.dir/shibokensupport/__feature__.py", line 142, in _import
    return original_import(name, *args, **kwargs)
ImportError: Import.so: cannot open shared object file: No such file or directory

Notice it doesn't fail if I run the tests on the build directory directly.
Tests are run with 'FreeCADCmd -t 0'.

I don't have precise version information because I run this in a Docker runner with Ubuntu 22.04 and latest master.

Thanks for any help.

[mlampert]

What happens when you run FC with UI and then

Code: Select all

import TestPathApp

in the console?

[wmayer]

I have an issue that TestPathApp fails after I build and 'cmake --install' FreeCAD, with following log :

Do you use a self-compiled 3rd party library, e.g. OCCT? If yes you should set LD_LIBRARY_PATH to point to the path of the shared libraries.

[openBrain]

What happens when you run FC with UI and then

Code: Select all

import TestPathApp

in the console?

Not so easy to get UI interaction in a Docker but I'l see what I can test.

Do you use a self-compiled 3rd party library, e.g. OCCT? If yes you should set LD_LIBRARY_PATH to point to the path of the shared libraries.

No, it's built with the basic receipt to build FC on Linux. From my first tests, it seems it's specific to Ubuntu 22.04 as same receipt works well on Ubuntu 20.04.
I'll confirm that with further testing.

Does one know which file 'from PathApp import *' is targeting ? Is it "PathApp.so" ?

[openBrain]

Continuing the tests it actually not seems to be linked to OS but to compiler.
Indeed error only appears when GCC is used and not with Clang. I'm continuing to investigate and will report.

Edit : GCC fails equally on Ubuntu 20.04 and Ubuntu 22.04

[openBrain]

What happens when you run FC with UI and then

Code: Select all

import TestPathApp

in the console?

OK, found some time to dig this.

If I run this line, I get a similar error :

Code: Select all

>>> import TestPathApp
Traceback (most recent call last):
  File "<input>", line 1, in <module>
  File "/usr/local/Mod/Path/TestPathApp.py", line 25, in <module>
    from PathTests.TestPathAdaptive import TestPathAdaptive
  File "/usr/local/Mod/Path/PathTests/TestPathAdaptive.py", line 27, in <module>
    import Path.Op.Adaptive as PathAdaptive
  File "/usr/local/Mod/Path/Path/__init__.py", line 1, in <module>
    from PathApp import *
ImportError: Import.so: cannot open shared object file: No such file or directory

I found the problem (not the root cause ) to be that 'PathApp.so' is installed in '/usr/local/lib/', which isn't in the system search path.

If I run

Code: Select all

LD_LIBRARY_PATH=/usr/local/lib FreeCAD

to run FreeCAD, then I can import PathAppTest without any issue.

I will run another VM to confirm everything is fine when building with Clang and try to compare things.

I you have an idea, feel free to suggest.

[wmayer]

So, /usr/local/lib isn't listed in ld.so.conf but /usr/lib should be. So an easy workaround is to set CMAKE_INSTALL_PREFIX=/usr The default value is /usr/local

[openBrain]

So, /usr/local/lib isn't listed in ld.so.conf but /usr/lib should be. So an easy workaround is to set CMAKE_INSTALL_PREFIX=/usr The default value is /usr/local

Hmmm. Maybe. Keeping into /usr/local/ would be a good idea though to not mix files with packaged softwares.

Anyway, it doesn't explain why everything works correctly when compiled with Clang.
I just confirmed that with exact same VM.

For the record, it seems '/usr/local/lib/' is actually (indirectly) listed in 'ld.so.conf' in my Clang VM.

Here is the 'ld.so.conf' of a vanilla 20.04 install :

Code: Select all

include /etc/ld.so.conf.d/*.conf

And in the folder there is 'x86_64-linux-gnu.conf' with :

Code: Select all

# Multiarch support
/usr/local/lib/x86_64-linux-gnu
/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu

And especially 'libc.conf' with :

Code: Select all

# libc default configuration
/usr/local/lib

I will now check if this latest file also exists in the GCC VM.

[openBrain]

OK, I can confirm my GCC VM has exactly the same 'ld.so.conf' structure and content.

[wmayer]

Hmmm. Maybe. Keeping into /usr/local/ would be a good idea though to not mix files with packaged softwares.

But we are talking about the VM for the CI and there I guess not much additional software is installed. Beyond that on Debian-based systems FreeCAD's binaries are installed into a sub-directory of /usr/lib (https://packages.debian.org/bullseye/am ... 9/filelist)

I found the problem (not the root cause ) to be that 'PathApp.so'

The root cause is the fact what the linker does when building a shared library and when installing it. When you run make to build the binaries then the linker adds the RUNPATH entry that contains a list of the paths to the shared libraries the binary depends on. This explains why you can directly run an application without having to set LD_LIBRARY_PATH fiirst.

When installing the binaries then the linker does some re-linking and cleans up the RUNPATH section again of the target file.

Perform this command after make and after make install

Code: Select all

objdump -x Mod/Import/Import.so | grep RUNPATH

See the difference.