macOS Qt5 plan and status

Here's the place for discussion related to coding in FreeCAD, C++ or Python. Design, interfaces and structures.
Forum rules
Be nice to others! Respect the FreeCAD code of conduct!
blacey
Posts: 370
Joined: Tue Dec 08, 2015 11:28 pm

Re: macOS Qt5 plan and status

Post by blacey »

And the good news is that with proper formulae in place for shiboken, pyside and pyside-tools from qt.io, the standard cmake with the additional -DBUILD_QT5 finds all the dependent packages making it much easier to build:

Code: Select all

$ cmake -DBUILD_FEM_NETGEN=ON -DFREECAD_USE_EXTERNAL_KDL=ON -DFREECAD_CREATE_MAC_APP=ON -DCMAKE_BUILD_TYPE=DEBUG -DBUILD_QT5=ON ..
-- The C compiler identification is AppleClang 6.1.0.6020053
-- The CXX compiler identification is AppleClang 6.1.0.6020053
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for GL/gl.h
-- Looking for GL/gl.h - not found
-- Looking for C++ include istream
-- Looking for C++ include istream - found
-- Looking for C++ include ostream
-- Looking for C++ include ostream - found
-- Looking for C++ include fstream
-- Looking for C++ include fstream - found
-- Looking for C++ include sstream
-- Looking for C++ include sstream - found
-- Looking for C++ include ios
-- Looking for C++ include ios - found
-- Looking for C++ include iostream
-- Looking for C++ include iostream - found
-- Looking for C++ include iomanip
-- Looking for C++ include iomanip - found
-- Looking for C++ include iostream
-- Looking for C++ include iostream - found
-- Check for STD namespace
-- Check for STD namespace - found
-- prefix: /usr/local
-- datadir: data
-- docdir: doc
-- includedir: include
-- libdir: /usr/local/lib
-- Detected Homebrew install at /usr/local
-- Found PythonInterp: /usr/local/bin/python (found version "2.7.13") 
-- Found PythonLibs: /usr/local/opt/python/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib (Required is exact version "2.7.13") 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE  
-- Boost version: 1.63.0
-- Found the following Boost libraries:
--   filesystem
--   program_options
--   regex
--   signals
--   system
--   thread
--   chrono
--   date_time
--   atomic
-- Found Xerces-C: /usr/local/lib/libxerces-c.dylib
-- Found ZLIB: /usr/lib/libz.dylib (found version "1.2.5") 
-- PyCXX found:
--   Headers:  /Users/blacey/FreeCAD-MacOS-CI/src
--   Sources:  /Users/blacey/FreeCAD-MacOS-CI/src/CXX
-- Found OCC: /usr/local/include/opencascade (found version "7.0.0") 
-- -- Found OCE/OpenCASCADE version: 7.0.0
-- -- OCE/OpenCASCADE include directory: /usr/local/include/opencascade
-- -- OCE/OpenCASCADE shared libraries directory: /usr/local/lib
-- VTK components: not found or used
-- HDF5: Using hdf5 compiler wrapper to determine C configuration
-- Found HDF5: /usr/local/lib/libhdf5.dylib;/usr/local/lib/libsz.dylib;/usr/lib/libz.dylib;/usr/lib/libdl.dylib;/usr/lib/libm.dylib (found version "1.8.18")  
-- Check for medfile (libmed and libmedc) ...
-- Found MEDFile: /usr/local/include  
-- Found NETGEN version 5.3, calculated: 328448
-- Found SWIG: /usr/local/bin/swig (found version "3.0.11") 
-- Found Eigen3: /usr/local/include/eigen3 (Required is at least version "2.91.0") 
-- Found Freetype: /usr/local/lib/libfreetype.dylib (found version "2.7.1") 
-- Found OpenGL: /System/Library/Frameworks/OpenGL.framework  
-- Found OpenGLU: /System/Library/Frameworks/OpenGL.framework
-- Could NOT find Spnav (missing:  SPNAV_LIBRARY SPNAV_INCLUDE_DIR) 
-- libshiboken built for Release
-- Found PySide2 tools: /usr/local/bin/pyside2-uic, /usr/local/bin/pyside2-rcc
-- -- matplotlib-1.5.1 has been found.
-- Platform is 64-bit, set -D_OCC64
-- Build type: DEBUG
Built successfully using the qt5 branch, albeit against qt5.5. All FreeCAD tests pass with ./bin/FreeCAD --run-test 0.

However,

Code: Select all

make install/fast
still does something really quirky and wrong. Here is an excerpt, in hopes that @peterl94 or @ian.rees know the root cause.

Code: Select all

-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.fseventsd/000000000629d4aa
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.fseventsd/00000000062a0f4b
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.fseventsd/00000000062ad594
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.fseventsd/fseventsd-uuid
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.Spotlight-V100
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.Spotlight-V100/Store-V1
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.Spotlight-V100/Store-V1/VolumeConfig.plist
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.Spotlight-V100/Store-V2
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.Spotlight-V100/Store-V2/643A3640-86AD-461D-AC14-8F14FDC4DD4B
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.Spotlight-V100/Store-V2/643A3640-86AD-461D-AC14-8F14FDC4DD4B/.store.db
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.Spotlight-V100/Store-V2/643A3640-86AD-461D-AC14-8F14FDC4DD4B/0.directoryStoreFile
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.Spotlight-V100/Store-V2/643A3640-86AD-461D-AC14-8F14FDC4DD4B/0.directoryStoreFile.shadow
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.Spotlight-V100/Store-V2/643A3640-86AD-461D-AC14-8F14FDC4DD4B/0.indexArrays
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.Spotlight-V100/Store-V2/643A3640-86AD-461D-AC14-8F14FDC4DD4B/tmp.spotlight.state
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.Spotlight-V100/VolumeConfiguration.plist
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.Trashes
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.vol
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///.VolumeIcon.icns
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///Applications
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///Applications/.DS_Store
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///Applications/.localized
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///Applications/App Store.app
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///Applications/App Store.app/Contents
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///Applications/App Store.app/Contents/_CodeSignature
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///Applications/App Store.app/Contents/_CodeSignature/CodeResources
-- Installing: /usr/local/FreeCAD.app/Contents/lib/qtplugins///Applications/App Store.app/Contents/Info.plist
For some reason, make install/fast seems to be tracing back to some the root volume and copying everything from that volume into qtplugins...

Time's up for me so I'm offline for a bit...
blacey
Posts: 370
Joined: Tue Dec 08, 2015 11:28 pm

Re: macOS Qt5 plan and status

Post by blacey »

Ok, so the qtplugin mystery is solved but we still can't build a proper Mac App Bundle with Qt5... It turns out that src/MacAppBundle/CMakeLists.txt needed to be changed to handle -DBUILD_QT5 because the plugin location changes from Qt4 to Qt5. I have made the changes and will push them to my FreeCAD fork under bblacey/FreeCAD-MacOS-CI as a qt5 branch but the Python script that makes the bundle relocatable fails because QtCore references Xcode's XCTest library as an @rpath and the python script isn't designed to handle that case (i.e. libraries in an SDK).

I instrumented the script and it fails when trying to process library '@rpath/XCTest.framework..."

Code: Select all

Looking for library: XCTest in install_path: @rpath/XCTest.framework/Versions/A
Traceback (most recent call last):
  File "/Users/blacey/FreeCAD-MacOS-CI/src/Tools/MakeMacBundleRelocatable.py", line 335, in <module>
    main()
  File "/Users/blacey/FreeCAD-MacOS-CI/src/Tools/MakeMacBundleRelocatable.py", line 329, in main
    build_deps_graph(graph, bundle_path, dir_filter, search_paths)
  File "/Users/blacey/FreeCAD-MacOS-CI/src/Tools/MakeMacBundleRelocatable.py", line 236, in build_deps_graph
    deps = create_dep_nodes(list_install_names(k2), s_paths)
  File "/Users/blacey/FreeCAD-MacOS-CI/src/Tools/MakeMacBundleRelocatable.py", line 162, in create_dep_nodes
    raise LibraryNotFound(lib_name + " not found in given paths")__main__.LibraryNotFound: XCTest not found in given paths

--Source Library---
/usr/local/opt/qt@5.5/lib/QtTest.framework/Versions/5/QtTest:
	/usr/local/opt/qt@5.5/lib/QtTest.framework/Versions/5/QtTest (compatibility version 5.5.0, current version 5.5.1)
	@rpath/XCTest.framework/Versions/A/XCTest (compatibility version 1.0.0, current version 9530.0.0)
	/System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 57337.20.43)
	/System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices (compatibility version 1.0.0, current version 48.0.0)
	/System/Library/Frameworks/IOKit.framework/Versions/A/IOKit (compatibility version 1.0.0, current version 275.0.0)
	/System/Library/Frameworks/Foundation.framework/Versions/C/Foundation (compatibility version 300.0.0, current version 1256.1.0)
	/usr/local/Cellar/qt@5.5/5.5.1/lib/QtCore.framework/Versions/5/QtCore (compatibility version 5.5.0, current version 5.5.1)
	/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration (compatibility version 1.0.0, current version 1.0.0)
	/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.1.0)
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1226.10.1)
	/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 1256.14.0)
	/usr/lib/libobjc.A.dylib (compatibility version 1.0.0, current version 228.0.0)

--Failed Dependency--
@rpath/XCTest.framework/Versions/A/XCTest
/usr/local/Cellar/qt@5.5/5.5.1/lib/QtCore.framework/Versions/5/QtCore
Where, oh where is our friend @peterl94 when you need him ;)
peterl94
Veteran
Posts: 1001
Joined: Thu May 23, 2013 7:31 pm
Location: United States

Re: macOS Qt5 plan and status

Post by peterl94 »

Just found this thread today and have been getting things set up for qt5. :)

Well that is a bother, they set the install name, but didn't provide an rpath. A way to work around it would be to ignore the fact it is a framework and add /path/to/XCTest.framework/Versions/A after ${HOMEBREW_PREFIX}${MACPORTS_PREFIX}/lib in MacAppBundle/CMakeLists.txt

My first thought is to make the relocatable script intelligent about the framework path even if it didn't keep the structure when copying. Something like this (likely needs more code - haven't thought what happens afterward):

Code: Select all

def create_dep_nodes(install_names, search_paths):
    ...
    for lib in install_names:
        install_path = os.path.dirname(lib)
        lib_name = os.path.basename(lib)
         ...
        path = get_path(lib_name, search_paths)
        
        if lib[0] == "@":
            #also could be done with regex
            path_components = lib.split('/')
            if path_components[1].endswith('.framework'):
                framework_path = os.path.join(*path_components[1:])
                #find location of framework
                path = get_path(framework_path, search_paths)
                if path:
                    #full path to library in framework
                    path = os.path.join(path, os.path.dirname(framework_path))
        ...
Or should we copy XCTest into the bundle at all? I don't know if or how QtTest is used, so I was thinking we could just require xcode to be installed to use it (and add the sdk path to the rpaths somehow).
ian.rees
Posts: 696
Joined: Sun Jun 15, 2014 3:28 am
Contact:

Re: macOS Qt5 plan and status

Post by ian.rees »

peterl94 wrote: Or should we copy XCTest into the bundle at all?
I think probably not - for one, there are likely licensing issues with distributing binaries from XCode. In general, it seems to me that the bundles are really meant more for <handwave>release builds</handwave>, so there's not as much utility in providing test tools.

I'm a bit behind you guys - got distracted by trying to setup a MacOS 10.12 VM, but will try to put some time in over the next few evenings. -Ian-
blacey
Posts: 370
Joined: Tue Dec 08, 2015 11:28 pm

Re: macOS Qt5 plan and status

Post by blacey »

@ian.rees and @peterl94, the following branches are available to support our Qt 5 collab.

Qt 5.x - Cmake updates to create an App bundle on MacOS with Qt 5.x (needed to build FreeCAD after Qt ports have been installed)
https://github.com/bblacey/FreeCAD-MacOS-CI/tree/qt5

Qt 5.5 ports - All dependent formulae developed, builds locally and all tests pass
homebrew tap with Qt 5.5 formulae: https://github.com/FreeCAD/homebrew-freecad/tree/qt5
NOTE: Shiboken, pyside and pyside-tools formula pinned to older SHAs to improve Qt 5.5 stability

Qt 5.6 ports - Adds a Qt 5.6 formula and updates dependencies to use 5.6
homebrew tap with Qt 5.6 formulae: https://github.com/FreeCAD/homebrew-freecad/tree/qt56
NOTE: Shiboken, pyside and pyside-tools formulae support --HEAD builds for experimentation

Bottom-line: Qt 5.5 looks pretty good right now but Qt 5.6 is very early and needs more work (pyside fails to build). My cycles are really limited right now so I will spend them trying to bring up Qt 5.5 builds on my fork so we can solicit volunteers to test Qt 5.5 on macOS. If those builds prove "good enough", then we can move the Travis builds to Qt 5.5 until we have Qt 5.6 ready.

I would welcome help from you guys with two things to keep the Qt 5 train moving: 1) Wringing out issues with Qt 5.6 and 2) Fixing the MakeMacBundleLocatable script to deal with the XCTest framework living within Xcode. Right now, I have used @peterl94's work-around for #2 and I know that @ian.rees has a "working" Qt 5.6 formula. I will get to these items eventually but it will just take a while due to other commitments.
blacey
Posts: 370
Joined: Tue Dec 08, 2015 11:28 pm

Re: macOS Qt5 plan and status

Post by blacey »

Travis just cranked out our first ever Qt5 build for macOS. I made the changes to the Travis config as a Qt 5 path-finder on my FreeCAD fork and interested parties can download the builds from the 0.17 releases section.

Code: Select all

OS: Mac OS X
Word size of OS: 64-bit
Word size of FreeCAD: 64-bit
Version: 0.17.9687 (Git)
Build type: Release
Branch: (detached from f6dd2e6)
Hash: f6dd2e6791fad3548d17ea14598a59a300a48224
Python version: 2.7.13
Qt version: 5.5.1
Coin version: 3.1.3
OCC version: 7.0.0
I have developed an integration script that monitors FreeCAD master and will automagically integrate changes from FreeCAD master into the Qt5 branch on my fork and kick off a build so the builds should track the main FreeCAD repo fairly well.

That said, Qt5 is still early. The idea of these builds is to get broader testing by the community to see if Qt5 in it's current state is stable enough to at least switch the macOS builds on the main repo over to Qt5.

For now, the current Qt5 builds are built using Qt5.5.1 and compatible versions of PySide2 (shiboken, pyside and pyside-tools) that are numerous commits behind head on qt.io.

In parallel to these Qt5.5 CI builds, @ian.rees is trying to move Qt5.6 down the field. Qt5.7, at least for the time being, is out of the question because PySide2 simply does not support the C++11 syntax used in Qt5.7 so we either wait or revisit alternative Python bindings for Qt again...

This is far from done so any macOS developers that want to jump in and build Qt5 locally for macOS to help with Qt5 bug fixing, the pertinent info is documented in this thread and GitHub Qt5 Issue

Please post any feedback and/or test results here.
triplus
Veteran
Posts: 9471
Joined: Mon Dec 12, 2011 4:45 pm

Re: macOS Qt5 plan and status

Post by triplus »

Hi @blacey.

To me it looks like some issues will need to be resolved upstream (Qt/PySide2) first. As your effort doesn’t come down just to "packaging" issues anymore. Therefore if it isn't resolvable for now i would wait for likely a couple of months for upstream to catch up?

P.S. If you are able to provide some sort of testing build for now that should do. To see if we need to start caring about QOpenGLWidget and things like that.
blacey
Posts: 370
Joined: Tue Dec 08, 2015 11:28 pm

Re: macOS Qt5 plan and status

Post by blacey »

triplus wrote:P.S. If you are able to provide some sort of testing build for now that should do. To see if we need to start caring about QOpenGLWidget and things like that.
@triplus, I provided the download link for the test builds in my last post. The test builds automatically merge changes from FreeCAD/master into the Qt5 branch on my fork so they are very current (in fact, Travis just cranked out build 9746 a short while ago and is already working on 9747 as I type - the last merge to FreeCAD/FreeCAD/master).
triplus
Veteran
Posts: 9471
Joined: Mon Dec 12, 2011 4:45 pm

Re: macOS Qt5 plan and status

Post by triplus »

I didn't notice there is a link to Qt 5.5 build available already. I mainly focused on Qt 5.7 issue as that doesn’t look too good. On the other hand Qt 5.5 sounds really good for me for now.
chrisb wrote:The current osx version does not use QT5 and has - expected - still the described problem. I will retest as soon as QT5 is out.
It's out!
User avatar
sgrogan
Veteran
Posts: 6499
Joined: Wed Oct 22, 2014 5:02 pm

Re: macOS Qt5 plan and status

Post by sgrogan »

blacey wrote: In parallel to these Qt5.5 CI builds, @ian.rees is trying to move Qt5.6 down the field. Qt5.7, at least for the time being, is out of the question because PySide2 simply does not support the C++11 syntax used in Qt5.7 so we either wait or revisit alternative Python bindings for Qt again...
As a point of reference:
Ubuntu 14.04 LTS QT5.2
Ubuntu 16.04 LTS QT5.5
Ubuntu 16.10 QT5.6
Ubuntu 17.04 (not yet releases) QT5.7
On Windows wmayer has been using QT5.6 so that's what I will be targeting, but I haven't even started yet.
"fight the good fight"
Post Reply