#1404 closed task (fixed)
2.0 moduleset updates
Reported by: | Antoine Martin | Owned by: | Smo |
---|---|---|---|
Priority: | major | Milestone: | 2.0 |
Component: | platforms | Version: | trunk |
Keywords: | osx | Cc: |
Description
Now that we've dropped support for older build systems, we can update the modulesets with upstream changes again:
- r14737: gettext, autoconf (safe for 1.x?)
- r14738: websocket_client (will apply to 1.x)
- r14740: libxml2 (safe for 1.x?), gettext (safe for 1.x), gtk-mac-integration (not safe), glib (not safe for 1.x), libffi (safe?), gobject-introspection (not safe), pango (safe?), gdk-pixbuf (not safe?), gtk+-3.0 (not used), pixman (safe?)
Change History (32)
comment:1 Changed 4 years ago by
Owner: | changed from Antoine Martin to Smo |
---|
comment:2 Changed 4 years ago by
Building from scratch ("jhbuild bootstrap" then "jhbuild build") works fine, just takes longer.
But trying to build the xpra package then fails with:
calling 'gtk-mac-bundler Xpra.bundle' in /Users/osx/Xpra-svn/trunk/osx Traceback (most recent call last): File "/Users/osx/.local/bin/gtk-mac-bundler", line 6, in <module> bundler.main.main(sys.argv[1:]) File "/Users/osx/gtk/source/gtk-mac-bundler-0.7.3/bundler/main.py", line 18, in main bundler.run() File "/Users/osx/gtk/source/gtk-mac-bundler-0.7.3/bundler/bundler.py", line 611, in run self.copy_binaries(self.project.get_binaries()) File "/Users/osx/gtk/source/gtk-mac-bundler-0.7.3/bundler/bundler.py", line 207, in copy_binaries dest = self.copy_path(path) File "/Users/osx/gtk/source/gtk-mac-bundler-0.7.3/bundler/bundler.py", line 224, in copy_path source = self.project.evaluate_path(Path.source) File "/Users/osx/gtk/source/gtk-mac-bundler-0.7.3/bundler/project.py", line 288, in evaluate_path path = utils.evaluate_pkgconfig_variables(path) File "/Users/osx/gtk/source/gtk-mac-bundler-0.7.3/bundler/utils.py", line 27, in evaluate_pkgconfig_variables raise Exception("pkg-config variable '%s %s' is undefined" % (key, module)) Exception: pkg-config variable 'pango_module_version pango' is undefined ERROR: gtk-mac-bundler failed
Following the advice from Make pango modules related code optional to fix build with pango 1.38, r14746 removes pango from the bundle file.
We still need the patch to be applied to gtk-mac-bundler (it applies cleanly - skipping the examples), this needs to be added to the moduleset.
comment:3 Changed 4 years ago by
comment:4 Changed 4 years ago by
comment:5 Changed 4 years ago by
comment:6 Changed 4 years ago by
Owner: | changed from Smo to Antoine Martin |
---|
comment:7 Changed 4 years ago by
Owner: | changed from Antoine Martin to Smo |
---|
Works for me too, closed ticket:840#comment:37.
Can you comment on which libraries are safe to apply to 1.0? (see ticket description).
My guess: gettext and autoconf only? gdk-pixbuf?
comment:8 Changed 4 years ago by
Sorry for the delay.
This is what I think
shoud be safe
gettext, autoconf, libxml2, gettext
maybe
pango, pixman, gdk-pixbuf
not safe
gtk-mac-integration, libffi
comment:9 Changed 4 years ago by
Resolution: | → fixed |
---|---|
Status: | new → closed |
Notes:
jhbuild build
is not enough - you may get errors like in comment:1, best to start again from the top:jhbuild bootstrap -f;jhbuild build -f
- rsvg was moaning:
Making all in . CCLD rsvg-view-3 ld: library not found for -lgtk-3 collect2: ld returned 1 exit status make[2]: *** [rsvg-view-3] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2
Fixed by: jhbuild buildone -f gtk+-3.0
- libtheora got stuck on SDL detection (odd - don't care)
comment:11 Changed 4 years ago by
r14910 also updates the bundler and properly fixes the issues with the new pango version
comment:12 Changed 4 years ago by
For the record: my VM got hosed by a system update.. (wouldn't boot no matter what bootloader or boot CD I tried to use..) so I reverted back to an earlier snapshot.. and jhbuild just fell over itself too many times trying to update + ugprade (libintl.8.dylib and more) so I decided to start again from scratch - a bad idea (tm).
That didn't work until I completely wiped the gtk directory. And even then it failed to build jpeg because nasm was the default system one (too old).
Got a few SSL certificate problems along the way too: cairographics.org, gstreamer.freedesktop.org, etc.. PITA
python-cryptography needs openssl, but somehow that wasn't built by then.. and then I found out that the current openssl version in the moduleset had been removed from the download location we use... so I bumped openssl in r14944. (we need permanent urls for everything instead)
I'll then forced a full rebuild (jhbuild build -f
) at the end to ensure that everything is linked against the library version we ship. (ie: our newer openssl for python, etc)
comment:13 Changed 4 years ago by
More in r15055:
- ffmpeg 3.2.4
- cryptography 1.7.2
- python-xxhash 1.0.0
comment:14 Changed 4 years ago by
Lots more updates in r15138 (copying changelog message):
- sshpass 1.06
- opus 1.1.4
- setuptools 34.2.0
- pycparser 2.17
- cffi 1.9.1
- curl 7.53.0
- pyasn1 0.2.2
- py2app 0.12
- numpy 1.12
- macholib 1.8
- modulegraph 0.14
- altgraph 0.13
- ipaddress 1.0.18
- idna 2.2
- enum34 1.1.6
- scons 2.5.1
- subversion 1.9.5
Some of those should be backported sooner rather than later: sshpass, opus - and maybe also numpy.
comment:15 Changed 4 years ago by
Resolution: | fixed |
---|---|
Status: | closed → reopened |
Two of those update are problematic:
- setuptools-34.0 fails with 'ImportError: No module named packaging.version'
- cffi seems to break cryptography packaging - could be: Support for PyInstaller, py2exe, Cx_Freeze and similar tools
I'd rather fix this rather than downgrade again, but can't spend too much time on this either.
comment:16 Changed 4 years ago by
Owner: | changed from Smo to Antoine Martin |
---|---|
Status: | reopened → new |
comment:17 Changed 4 years ago by
r15143 changes setup tools to 33.1.1. There seems to be an issue with 34 which will take some more work to fix.
comment:18 Changed 4 years ago by
Owner: | changed from Antoine Martin to Smo |
---|
- setuptools was downgraded to version 33.1.0 in r15143 to fix the import error from comment:15, but we are actually locked on the much older version 19.2 to fix pyobjc problems, done in r15170
- py2app seems to be a bit borken: it tries to check things (something it didn't do before), and it tries too hard, hitting this bug: Importing ctypes.wintypes on Linux gives a ValueError instead of an ImportError.
Then it also seems to get confused about which platform it is running on and blows up some other win32 modules we don't care about. (removing them from the source tree helps, temporarily at least)
Downgrading to 0.11 still shows problems: it fails during its qt5 import recipes (missing 'sip' dependency)... but we don't even use qt, what are you doing there py2app?
So r15165 downgrades py2app from version 0.12 to 0.10, also adds a warning message in the moduleset file.
- cffi: tried a few versions and only versions up to 1.5.2 allow us to package python-cryptography properly, so r15168 downgrades it and adds a note.
@smo: just a FYI, things should be packaged OK again. If you had rebuilt python-cryptography with a newer cffi, you may need to run
jhbuild buildone -f python-cryptography
comment:19 Changed 4 years ago by
Everything builds fine. Even forced rebuild of cffi and python-cryptography but i'm seeing this unit test fail now.
====================================================================== ERROR: test_perf (__main__.TestCrypto) ---------------------------------------------------------------------- Traceback (most recent call last): File "/Users/build/xpra-trunk/src/unittests/unit/net/crypto_test.py", line 31, in setUp pycryptography_backend.init() File "/Users/build/gtk/inst/lib/python2.7/site-packages/xpra/net/pycryptography_backend.py", line 52, in init backend = default_backend() File "/Users/build/gtk/inst/lib/python2.7/site-packages/cryptography/hazmat/backends/__init__.py", line 71, in default_backend _default_backend = MultiBackend(_available_backends()) File "/Users/build/gtk/inst/lib/python2.7/site-packages/cryptography/hazmat/backends/multibackend.py", line 36, in __init__ "Multibackend cannot be initialized with no backends. If you " ValueError: Multibackend cannot be initialized with no backends. If you are seeing this error when trying to use default_backend() please try uninstalling and reinstalling cryptography. ---------------------------------------------------------------------- Ran 2 tests in 5.384s
Maybe I need to do something more to get this to work properly again?
comment:20 Changed 4 years ago by
The error in comment:19 is definitely the error that made me downgrade cffi in r15168.
You may need to ensure you don't have an old version laying around (remove all and re-install it). Then you may need to zap the python-cryptography source directory too.
If that still doesn't help, try zapping all the other downgrades (setuptools, etc).
comment:21 Changed 4 years ago by
After deleting all directories having to do with setuptools, cffi and python-cryptography I was able to rebuild them then the unittests pass.
Must have been some issue with the downgrades.
I should note I also deleted stuff from
~/gtk/inst/lib/python2.7/site-packages
cffi*, cryptography* and setuptools*
comment:22 Changed 4 years ago by
Even on newer versions of osx we still need the package tools to make the pkg.
Making Xpra-x86_64-2.0-r15188.pkg 229163 blocks 4 blocks mkbom: invalid option -- u Usage: mkbom [-h] [-s] directory bom mkbom [-s] -i filelist bom PKG Signing skipped (no keyname) 33 ./image/Xpra-x86_64-2.0-r15188.pkg Done PKG
I built them with this
jhbuild build meta-osx-xpra-pkgtools
comment:23 Changed 4 years ago by
Ran into this issue and it appears it is a problem with openssl 1.1. This might take some work to fix. Not sure at the moment. Going to check if we really need this on newer versions of osx.
checking for openssl/evp.h... yes checking for OpenSSL_add_all_ciphers in -lcrypto... no configure: error: Cannot build without libcrypto (OpenSSL) *** Error during phase configure of xar: ########## Error running ./configure --prefix /Users/build/gtk/inst *** [1/4] *** module meta-osx-xpra-pkgtools not built due to non buildable xar *** [4/4]
comment:24 Changed 4 years ago by
Making Xpra-x86_64-2.0-r15188.pkg 229148 blocks 4 blocks PKG Signing skipped (no keyname) 34 ./image/Xpra-x86_64-2.0-r15188.pkg Done PKG
r15189 comments out xar and adds a note
comment:25 Changed 4 years ago by
The pygobject flags patch had gone MIA: I have removed it in the r15149 cleanup because it was not referenced anywhere since r13713, but that was a mistake: we need it to avoid the warnings (see ticket:1444#comment:2).
Re-added in r15191. I wasn't just seeing things then.
comment:26 Changed 4 years ago by
We should do something about the git checkouts: either use a specific git id, or take a snapshot (version it ourselves). ie: libepoxy and others.
And since we also want to be able to apply patches.. maybe snapshots is the way to go?
comment:27 Changed 4 years ago by
r15254 adds grep and pcre which is needed for grep
Added these because I needed them to rule out issues with cmake and configuring libyuv. The system grep doesn't support the '-P' option so it errors when running cmake for libyuv.
This didn't fix the issue with libyuv but I figure I should leave them in there in case we need them later.
As for the stuff from git goes if we need to apply any patches with the jhbuild system that doesn't work for us. The only way to do this is with tarball snapshots.
comment:28 Changed 4 years ago by
I was updating my builds today and for some reason the dependency for setuptools_scm is being ignored for some reason I see that it was added for lz4 which is why I needed it.
It looks right in xpra.modules but it isn't doing what we want.
comment:29 Changed 4 years ago by
Added libyuv now so we can build it within the moduleset and added it to the meta module so it should automatically get built.
details in #1280
comment:30 Changed 4 years ago by
Follow up for 2.1: #1501
Please close unless something still needs doing?
comment:31 Changed 4 years ago by
Resolution: | → fixed |
---|---|
Status: | new → closed |
comment:32 Changed 3 months ago by
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1404
I encountered some build problems on "yelp-xsl", "pango" and "gdk-pixbuf":
Which clearly we're not the first ones to hit: dyld: Library not loaded: /usr/local/libodep/lib/libintl.8.dylib (and many more hits)
Fixed by rebuilding libcroco. Hit a similar thing later on "gtk+", solved by rebuilding "atk".
Then it gets wedged with gobject-introspection:
Fixed by re-installing flex. ("make install" from the source dir - odd that "jhbuild buildone flex" didn't work here, oh well)
Maybe I should have just ran everything from the top ("jhbuild bootstrap").
@smo: any changes to the list of safe/unsafe candidates for applying to the 1.x branch?