Started work on this here is the first start. Hoping these can be added to svn somewhere.
Still needs work for dependencies however they all seem to build nicely. Only one patch was needed for "nasm" to support make install DESTDIR= or jhbuild complains.
I'm attaching the first files to this ticket.
Xpra Moduleset
Added these in r5829
After doing
jhbuild bootstrap jhbuild build jhbuild buildone gstreamer jhbuild buildone gst-plugins-base jhbuild buildone gst-plugins-good
You should be able to now cd to trunk/osx/jhbuild
Run jhbuild -m ./xpra.modules build meta-osx-xpra-deps
This should build and install everything else necessary to build xpra..
Currently it is missing (py)gtkglext which is the next step to be worked on.
Optional if you want subversion updated and installed you can try this as well
jhbuild -m ./xpra.modules build meta-xpra-subversion
Very good stuff, what a timesaver overall! I've updated the winswitch osx build page to refer to this moduleset.
For reference here's a link to the jhbuild directory
Only found a few minor things that we want to tweak / note:
Pillow
(2.3.1) and lz4
(0.6.1) in r5882
https
for the download links and/or add checksums for the downloads
libpng
, libtiff
and libjpeg
: see #544
PyOpenGL
should be bumped to 3.1 beta (see #531), using the same download or build arguments as setuptools
if needed (in the setup.cfg
?), I've just recorded it as I did the upgrade:
Downloading https://pypi.python.org/packages/source/P/PyOpenGL/PyOpenGL-3.1.0b1.zip#md5=98cf868fac68e57d1712bc8b52bc8a4b Processing PyOpenGL-3.1.0b1.zip Writing /tmp/easy_install-18IUtE/PyOpenGL-3.1.0b1/setup.cfg Running PyOpenGL-3.1.0b1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-18IUtE/PyOpenGL-3.1.0b1/egg-dist-tmp-L7SbV6 (...) Downloading https://pypi.python.org/packages/source/P/PyOpenGL-accelerate/PyOpenGL-accelerate-3.1.0b1.zip#md5=8db7b69ba0553ca5b927fe2ddbf2424e Processing PyOpenGL-accelerate-3.1.0b1.zip Writing /tmp/easy_install-33_Ewg/PyOpenGL-accelerate-3.1.0b1/setup.cfg Running PyOpenGL-accelerate-3.1.0b1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-33_Ewg/PyOpenGL-accelerate-3.1.0b1/egg-dist-tmp-7MQaJK
pypi.douban.com
from here (today?) and even downloading from my UK servers it was pretty slow too at times (5KB/s at some point), which made building the python bits really tedious. Not sure if there's a better mirror we could/should be using? (easy_install
must be using a different download domain, because it didn't have any problems)
pypi.douban.com
for things like Cython
(and probably others too) which have known good upstream download locations
easy_install
will automagically grab the latest release for us, whereas the moduleset will require updating everytime one of the dependencies is updated. I'm OK with that, it will make the upgrades more obvious (we'll have a commit reference). The old macosx build page has links to most of the dependencies to make it easier to check for new releases, we should probably do the same for the python dependencies, maybe here: wiki/Building or in a new wiki page
ffmpeg
build is the winswitch one, it should be the xpra minimal one instead, ideally we could have both in the module somehow (unfortunately, I don't think that jhbuild does "virtual packages" ala portage)
chgrp
and chmod
at the end..)
--insecure
download flag at all to bootstrap the process, but I can't think of an easy way..
meta-osx-image-libs
for easier building of the 3 libs
I've had mixed results with not using --insecure or -k with curl in the end I ended up having to add the flag to the tarball.py within jhbuild to get by some issues.
New updates:
For ffmpeg, I am now using a better, more limited configure line (also on win32):
./configure --prefix=${JHBUILD_PREFIX} \ --cpu=i686 --enable-runtime-cpudetect --enable-pic --enable-memalign-hack \ --enable-static --enable-shared --enable-gpl \ --disable-everything \ --enable-swscale --enable-libx264 --enable-decoder=h264 \ --enable-libvpx --enable-decoder=vp8 --enable-decoder=vp9 --enable-decoder=hevc \ --disable-protocol=tcp --disable-protocol=rtp \ --disable-filter=aformat --disable-filter=crop --disable-filter=setpts \ --disable-filter=anull --disable-filter=format --disable-filter=trim \ --disable-filter=atrim --disable-filter=null \ --disable-programs --disable-avfilter
It took a lot of fiddling and trial and error:
--disable-all
sounds like what we want... but it doesn't work: nothing gets installed
Added patch for gtkglext in r6096
Can use it by doing something like this
git clone git://git.gnome.org/gtkglext cd gtkglext curl -O http://xpra.org/svn/Xpra/trunk/osx/jhbuild/xpra-gtkglext.patch patch -p1 <xpra-gtkglext.patch autoreconf -fiv ./configure --prefix=${JHBUILD_PREFIX} make && make install
This isn't ideal but jhbuild doesn't allow me to apply a patch when pulling source from git only when using tarballs
Next is pygtkglext.
Added
python-macholib python-modulegraph python-altgraph
in r6098
As I was unable to build without these perhaps easy_install pulls these in for us normally?
After compiling and installing these I was able to build and package xpra.
Installing pygtkglext is still quite a hack as the installing portion usually fails and requires you to install them manually into site-packages/
New ones:
ORC
0.4.19 is out and is worth upgrading to as it fixes memory leaks. download link
libjpeg-turbo
1.3.1 is out. download link
numpy
1.8.1 is out
These are going to keep accumulating until this ticket is closed..
As for the question above: macholib
, modulegraph
and altgraph
are normally pulled in automatically when installing py2app
via easy_install
.
patch for building git version of pygtkglext
The patch above builds pygtkglext
for me using:
PYTHONPATH=$JHBUILD_PREFIX/share/pygobject/2.0/:$PYTHONPATH python2.7 \ ./setup.py build
We cannot use setup.py
for installing (see #226 for details), so we install it with rsync
(ugly):
rsync -rpvlgto build/lib.macosx*/gtk/* $JHBUILD_PREFIX/lib/python2.7/site-packages/gtk-2.0/gtk/
Note: be aware of ticket:563#comment:14 when choosing which version to use (git tree vs 1.1 snapshots)
1.2 snapshot for osx found here: http://lists.gnu.org/archive/html/bug-gnubg/2011-04/msg00006.html
Newer versions should be updated in the moduleset:
Note to smo: work is going to keep piling on until you close this ticket...
updated versions in r6479
Compiling this whole moduleset to make sure its still working. I will use the patch that I made against gtkglext this time but should I be using this new hack that you attached? Have you tried this?
The 1.2.0 patched version I posted is a newer version with some osx patches already applied, in particular the gtk>=2.20 compatibility patches. I haven't found any regressions, so I think we should use this rather than patching an older version ourselves.
New:
And more updates will continue to accumulate until this ticket is closed..
Today:
PyOpenGL version update in r6691 Python 2.7.7 I also changed the url in .jhbuildrc-custom and recompiled everything with no issues.
I have some modifications that work well for this as well that should be updated in the winswitch osx dev page.
Can we host gtkglext and the patch for pygtkglext somewhere on xpra.org so that I can put those into the moduleset as well?
Was your pygtkglext patch for the tarball?
Can we host gtkglext and the patch for pygtkglext somewhere on xpra.org so that I can put those into the moduleset as well?
Sure, point me to them and I'll upload them.
Was your pygtkglext patch for the tarball?
Which one?
They were both attachments to this ticket that you posted before however as I see now the pygtkglext patch is for git which won't work with jhbuild.
We can however put the hacked copy of gtkglext in the moduleset and that removes one more thing.
Done: http://xpra.org/src/gtkglext-1.2.0.osx-hack.tar.gz
Added gtkglext-1.2.0.osx-hack.tar.gz in r6720
had to configure with --enable-gtk-doc
or make install
failed with missing html doc file
I've had to pull the 0.12.4 OSX build because pygtkglext is now completely broken (not that I could easily have noticed since I don't have GL in my macosx vms...), rebuilding it against the latest gtk+ update does not seem sufficient to make it run, see #593. You can easily get a crash just by trying to import gtkgl:
python -c "from gtk.gtkgl import *"
The error seems to be the one mentioned here Fatal Python error: can't initialize module gtk.gtkgl but I can't see anything wrong, this same source code used to build and run fine...
I've just reviewed the GTK+ changes since 2.24.16, and we want quite a few of those (many are also relevant to win32 rebuilds discussed in ticket:263#comment:10):
r6897 updates many libraries
I had issues trying to rebuild after this as gst-plugins-ugly links against x264
jhbuild -m ./xpra.modules uninstall gst-plugins-ugly-xpra jhbuild -m ./xpra.modules -f buildone gst-plugins-ugly-xpra
Should be enough to get it building properly again.
It's taken a while but I think we will soon be able to close this ticket!
Please update as discussed:
Finally, you can find three patches which allow us to build (py)gtkglext from git which I have added to svn in r6940 + r6941:
gtkglext
: gtkglext-osx-quartztagfix.diff
git clone git://git.gnome.org/gtkglext cd gtkglext curl -O "http://xpra.org/trac/browser/xpra/trunk/osx/jhbuild/xpra-gtkglext.patch?rev=6940" patch -p1 < xpra-gtkglext.patch autoreconf -fiv curl -O "http://xpra.org/trac/browser/xpra/trunk/osx/jhbuild/gtkglext-osx-quartztagfix.diff?rev=6940" patch -p1 < gtkglext-osx-quartztagfix.patch ./configure --prefix=${JHBUILD_PREFIX} --with-gdktarget=quartz make && make install
pygtkglext
: pygtkglext-osx-v4.patch
git clone git://git.gnome.org/pygtkglext cd pygtkglext curl -O "http://xpra.org/trac/browser/xpra/trunk/osx/jhbuild/pygtkglext-osx-v4.patch?rev=6940" patch -p1 < pygtkglext-osx-v4.patch ./autogen.sh --prefix ${JHBUILD_PREFIX} make && make install
If needed for the moduleset, we can host the patches in a more canonical location, or even make a patched source archive with the patches already applied.
Note: the pygtkglext patch is not so different from the previous version (which may still work?), and I've seen weird things happen as I rebuilt the libraries. I believe jhbuild leaves some crud behind and we must forcibly remove the build directories before trying to rebuild a library. Something to keep in mind.. since I don't see a way of forcing jhbuild to completely remove a build directory.
It may be worthwhile adding a switch somewhere so we can get debug builds for all those libraries. Not sure exactly how to add -g
to the CFLAGS
however.
Once that's all done and tested, please re-assign to me so I can close this.
Dammit, looks like we're not the first ones to hit this "invalid drawable
" bug (recorded in ticket:563#no4 - which is closed, so moving here):
But for gtkgl on osx, the best I found is this (which has a dead link in it - it should point here instead: GtkGLExt OS X Quartz hack patch):
I think I see it just by running gl_check
which should help in testing a fix.
Running the examples in gtkglext
works without major problems, but the examples in pygtkglext
show the invalid drawable
problem.
For the record, we can also build pygtkglext-1.1.0
with the patch above, ignoring the changes to autogen.sh
and then using:
CFLAGS="-I/usr/X11/include" ./configure --prefix=${JHBUILD_PREFIX}
But the bug remains..
We can also build gtkglext
with debug enabled:
--enable-debug=[no/minimum/yes]
Edited comment:25 with instructions I have tested repeatedly from scratch, this seem to fix the "invalid drawable
" bug from ticket:563#no4, now tracked in ticket:593#comment:4 (at least it works on my 10.5 and 10.6 test machines so far)
Let's try to close this ticket whilst things work OK! (so raising to blocker)
If jhbuild doesn't allow us to apply the quartztagfix
patch after the autoreconf
step, we can just generate our own archive from git and stick it in svn.
gtkglext git snapshot
pygtkglext git snaptshot
I've updated most of the stuff from comment:25
I've had to make a copy of the modulesets-stable to our source control so that I can modify them to update a few modules. We can contribute this upstream when we are satisfied everything is working.
r7184 brings in all the necessary changes to our moduleset r7186 updates many libraries to current versions
I still have to build gtkglext and pygtkglext by hand but the instructions work nicely along with the patches. We may have to make source tarballs for these to apply the patches.
I've tried to find a nicer way to fix autoconf for gtkglext so we don't have to do the second patch but didn't figure it out yet.
I've tested building all of these on an mac mini running osx 10.6.8 64bit hence all the build switches I commited for 32bit builds
I am now in the process of building this all again on a mac mini running 10.5.8 32bit but I assume it will work even easier than the other machine.
So again we are very close to closing this ticket but still some work needs to be done.
This is what my .jhbuildrc-custom looks like at the moment on the 32bit machine
module_autogenargs['cairo'] = '--enable-ft=no' setup_sdk(target="10.5", sdk_version="10.5", architectures=["i386"]) os.environ["CC"] = "/usr/bin/gcc-4.2" os.environ["DYLD_LIBRARY_PATH"] = "" build_policy = "updated-deps" modules = [ "python", "meta-gtk-osx-bootstrap", "meta-gtk-osx-core", "libcroco", "librsvg", "meta-gtk-osx-python", "meta-gtk-osx-themes", "gtk-quartz-engine", "gtk-mac-integration-python" ] #skip iconv and add some switches to turn off stuff that was breaking skip.append("libiconv") append_autogenargs('gstreamer', '--enable-introspection=no') append_autogenargs('gst-plugins-base', '--enable-introspection=no') append_autogenargs('gst-plugins-good', '--enable-introspection=no --disable-deinterlace') #change moduleset moduleset="http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/gtk-osx.modules"
More problems it doesn't use bootstrap.modules from that location it only uses it from local ~/Source/jhbuild/modulesets/bootstrap.modules
Which is troublesome as we need to update gtk-doc so that we can build new glib if memory serves right I will also need another package to build new gtk-doc
I checked bootstrap.modules
against the one locally and the only change was the change I made so its safe to use this one to edit for now.
Unfortunatly I need to just copy over the one that exists with ours to do this not ideal
cd ~/Source/jhbuild/modulesets && curl -O http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/bootstrap.modules
Then run jhbuild bootstrap
Ran into another problem
gtk-doc requires itstool but itstool requires python
configure: error: itstool not found *** Error during phase configure of gtk-doc: ########## Error running ./configure --prefix /Users/blank/gtk/inst --libdir '/Users/blank/gtk/inst/lib' --disable-scrollkeeper --with-xml-catalog=$JHBUILD_PREFIX/etc/xml/catalog *** [16/19]
jhbuild buildone python jhbuild buildone libxml2 jhbuild buildone itstool jhbuild bootstrap
lets me get past this point but I guess python, libxml2 and itstool need to move to bootstrap? Or build gtk-doc without itstool?
After this I get
*** success *** [19/19]
That is enough to move on to the next steps
Please see also:
AppKit
from ticket:627#comment:4
I'll add the md5sums to the missing packages. Looks like we still need a patch for GTK for dual monitor support I found that out today.
As for (py)gtkglext I really think we should probably just make snapshots of the repos and host our own .tar.bz2 for those 2 so we can use them with jhbuild.
What would be a good place to stick these?
I found a way to have jhbuild pull our patch and run it after autoreconf however I can't seem to build from the gtkglext git snapshot in comment:33
I always end up with a build failure like this
i686-apple-darwin9-gcc-4.2.1: .libs/gdkglversion.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglinit.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglquery.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglconfig.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglcontext.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkgldrawable.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglpixmap.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglwindow.o: No such file or directory i686-apple-darwin9-gcc-4.2.1: .libs/gdkglglext.o: No such file or directory make[4]: *** [libgdkglext-quartz-1.0.la] Error 1 make[3]: *** [all-recursive] Error 1 make[2]: *** [all] Error 2 make[1]: *** [all-recursive] Error 1
I followed the instructions from comment:25 just to be sure and it works no problem so i'm thinking maybe that snapshot is bad?
I doubt the snapshot is bad, but it's possible.
Those errors are about object files, which are meant to be generated during the build, so a bad snapshot shouldn't be causing this unless it messed with the build scripts.
https://pypi.python.org/pypi/Pillow/2.5.3 low priority: only CVE entries for things we don't use.
gtkglext snapshot from git works for building osx with patches
r7472 adds gtkglext and pygtkglext
The source tarball for gtkglext is one I attached to this ticket so this should probably change.
I made this one by doing this
git clone git://git.gnome.org/gtkglext && cd gtkglext git archive --prefix=gtkglext-1.2.0/ -o ../gtkglext-1.2.0.tar HEAD cd .. && bzip2 gtkglext-1.2.0.tar
Seems to build now with that snapshot i'm not sure what the difference is. Feel free to make a new one and just provide me with the url and I will update it.
I will now delete my ~/gtk
directory and start this all from scratch hopefully now it will just build everything we need.
I've uploaded your snapshot in the source dir: http://xpra.org/src/gtkglext-1.2.0.tar.bz2
Note: Cython should be updated to 0.21.x (I have reported fairly big bug, they may well do a point release before too long).
I have created a separate ticket for OSX GTK3: #680.
Many new changes seems to work quite well now. Still needs some polish but pretty happy with it now.
You'll need the .jhbuildrc-custom from r7651 which includes our modulesets from by url.
r7642 adds gtk-doc from bootstrap.modules to our moduleset as we need itstool and libxml2 which requires python to build it now.
r7648 adds a patch for liboil to fix autreconf from breaking found it here https://github.com/GNOME/frogr/tree/master/osx/jhbuild/patches
r7652 Cython 0.21
You need to overwrite ~/Source/jhbuild/modulesets/bootstrap.modules
with ours I haven't found any way to use ours instead.
If everything is in place I'm able to do
jhbuild bootstrap jhbuild build jhbuild build meta-subversion-xpra
bootstrap has 17 steps and build has 88
I had to modify ~/Source/jhbuild/jhbuild/versioncontrol/tarball.py
to add -k to curl as a lot a few https tarballs were failing since the certificates on this old mac are outdated.
My best guess at how to run this build from start to finish (took a few tries to get every step) - to be used as documentation template? There should be at least a README file telling you what to do:
--insecure
!):
curl --insecure -O https://git.gnome.org/browse/gtk-osx/plain/gtk-osx-build-setup.sh sh gtk-osx-build-setup.sh export PATH=$PATH:~/.local/bin/; jhbuild shell
svn co http://xpra.org/svn/Xpra
cp Xpra/trunk/osx/jhbuild/modulesets-stable/*.modules ~/Source/jhbuild/modulesets/
.jhbuildrc-custom
(not obvious - I would prefer appending to the existing one rather than duplicating some of it):
cp Xpra/trunk/osx/jhbuild/jhbuildrc-custom-xpra ~/.jhbuildrc-custom
jhbuild bootstrap jhbuild build jhbuild build meta-subversion-xpra
I got some errors, but I think they were due to the fact that I had originally missed some of the undocumented steps.
Build started at: Wed 17 Sep 09:42:47 ICT 2014
Build finished at: Wed Sep 17 16:46:12 ICT 2014
A fair amount of this time was spent downloading stuff again on my slow DSL line (as I had wiped the whole home directory to start clean), but also re-doing things, and fixing cert errors...
Comments:
gst-plugins-base
was wrong, fixed in r7664
gtk-mac-bundler
is missing altogether, means we can't run the xpra build (note: there is no "real" install step for it, you must keep the source directory around...)
--cacert
option?)
python-lzo
and python-yaml
rencode
: see ticket:683#comment:1
Others can wait for #680: I've added the info there.
It looks bad because of the amount of pending issues, but really it isn't. One more round and we should be able to close this... and move on to #680 and #640.
I've made a beta build (with some version updates on top, as per above) which works fine: http://xpra.org/beta/osx/x86/
not fully tested moduleset updates
Applied your patch uploaded the modulesets to a test webserver and tested them. Just a couple issues.
https://github.com/atgreen/libffi/issues/128
--with-internal-glib
to configure or you get
configure: error: Either a previously installed pkg-config or "glib-2.0 >= 2.16" could not be found. Please set GLIB_CFLAGS and GLIB_LIBS to the correct values or pass --with-internal-glib to configure to use the bundled copy.
things that were updated in r7693
I tested this like you did by updating a few versions then I recompiled the whole lot.
Looks good the client will require some testing to make sure nothing breaks as this is quite a big update.
Next I will add the winswitch stuff that is left i'm just not sure if I want to make a winswitch.modules and include it or just add it to the current moduleset.
I will write out some simplified instructions and attact them to this ticket. Your instructions from comment:43 are roughly correct except you don't have to copy all the .modules files they are included via urls
Notes:
I'm having issues trying to add the winswitch stuff can't link with this nxcomp library
ld: in /Users/user/gtk/inst/lib/libXcomp.so, can't link with bundle (MH_BUNDLE) only dylibs (MH_DYLIB)
I've been following the steps http://winswitch.org/dev/macosx.html for the winswitch stuff.
I will try a few more things.
I needed to use the autotools module to do some hacks to download a 2nd tarball and extract it plus make a link.
This might not be the smartest way to do this perhaps I can just have it download a simply makefile that does all this stuff instead of hacking it into jhbuild that way satisfying both problems.
I will add in gtk-mac-bundler soon as well as it doesn't seem to matter where it is actually extracted to it simply makes a helper script in ~/.local/bin pointing to it. With a working patch this should install cleanly.
More updates needed (I've already done most of them outside the moduleset):
more?
gdb is still problematic to add to the moduleset since it requires one extra step to make it work after installing
New serf uses scons build system which is problematic to adding to moduleset as well.
I believe I came across this before so unless we absolutely need the latest version I don't think we should update.
the changeset was r8250.
FWIW: I'm still on ffmpeg 2.4 (2.4.4 as of this writing), not switching to 2.5 yet.
r8653 brings in many changes from upstream git repo
https://git.gnome.org/browse/gtk-osx/tree/modulesets-stable
Few were left behind if we were using newer versions like for gmp and python 2.7.9
Still needs testing which I'm rebuilding right now will update with instructions if successful
r8668 Newer versions of gtk-mac-integration will not build on osx < 10.6
r8671 Host patch for gtk-mac-integration as the old one seems to be missing
r8673 removes 2 modules we had put in that are now bootstrap modules
r8676 adds python-nose as we didn't have this and is needed now by lz4
r8678 we don't use scons for building serf
r8683 add gtk-mac-bundler to dependencies for meta-osx-xpra-deps this install works now
r8684 added proper old catched patch for gtk-mac-integration
Will update soon with new jhbuildrc-custom and a few tweaks.
Haven't found a way to override bootstrap.modules yet so I typically copy over it with ours.
Also I typically have to also make a change in jhbuild/versioncontrol/tarball.py to make curl use the '-k' option as certificate checking is a bit old and often fails when downloading files.
Another bug I discovered is that you want to build everything outside of jhbuild shell so that you use the system python for calling jhbuild. If you do not the ssl module in python 2.7.9 now checks certificates by default and this also causes problems as they use urllib2 to download and cache patches.
Here are some rough instructions should be good for updating the winswitch osx dev page if this all works.
curl -k -O https://git.gnome.org/browse/gtk-osx/plain/gtk-osx-build-setup.sh sh gtk-osx-build-setup.sh export PATH=$PATH:~/.local/bin/ curl -k -o ~/Source/jhbuild/modulesets/bootstrap.modules http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/bootstrap.modules sed -i -e 's/^setup_sdk/#setup_sdk/g' .jhbuildrc-custom cat << EOF >> .jhbuildrc-custom setup_sdk(target="10.5", sdk_version="10.5", architectures=["i386"]) os.environ["CC"] = "/usr/bin/gcc-4.2" os.environ["DYLD_LIBRARY_PATH"] = "" build_policy = "updated-deps" modules = [ "python", "libxml2", "itstool", "gtk-doc", "meta-gtk-osx-bootstrap", "meta-gtk-osx-core", "libcroco", "librsvg","meta-gtk-osx-python", "meta-gtk-osx-themes", "gtk-quartz-engine","gtk-mac-integration-python", "gstreamer", "gst-plugins-base", "gst-plugins-good", "meta-osx-xpra-deps"] #skip iconv and add some switches to turn off stuff that was breaking skip.append("libiconv") append_autogenargs('gstreamer', '--enable-introspection=no') append_autogenargs('gst-plugins-base', '--enable-introspection=no') append_autogenargs('gst-plugins-good', '--enable-introspection=no --disable-deinterlace') #change moduleset moduleset="http://xpra.org/svn/Xpra/trunk/osx/jhbuild/modulesets-stable/gtk-osx.modules" EOF jhbuild bootstrap jhbuild build jhbuild build meta-subversion-xpra jhbuild shell
Another day another fun time with osx.
Found that I was missing libz.1.2.8.dylib not because of de-duplication but because new jhbuild modulesets build it instead of using the system one.
After wasting my type trying to debug and fix the de-duplication part of the make-app.sh script I found that I just needed to add this to Xpra.bundle so that it was actually included.
Now I've run into further problems with libjpeg-turbo and PIL which I will have to solve now.
2015-03-19 10:58:21,680 draw error Traceback (most recent call last): File "/Users/test/gtk/inst/lib/python2.7/site-packages/xpra/client/ui_client_base.py", line 1961, in _do_draw File "/Users/test/gtk/inst/lib/python2.7/site-packages/xpra/client/client_window_base.py", line 423, in draw_region File "/Users/test/gtk/inst/lib/python2.7/site-packages/xpra/client/window_backing_base.py", line 476, in draw_region File "/Users/test/gtk/inst/lib/python2.7/site-packages/xpra/client/gtk2/window_backing.py", line 35, in paint_image File "/Users/test/gtk/inst/lib/python2.7/site-packages/xpra/client/window_backing_base.py", line 227, in paint_image File "PIL/Image.pyc", line 644, in tobytes one of :py:attr:`PIL.Image.NEAREST` (use nearest neighbour), File "PIL/ImageFile.pyc", line 191, in load File "PIL/Image.pyc", line 413, in _getdecoder the image. IOError: decoder jpeg not available
Could be that PIL wasn't built with jpeg support as I can see the library is packaged but I am looking for any advice you can give me to debug this.
Here is output from forcing PIL to rebuild. Looks like jpeg is compiled in so perhaps just missing something.
-------------------------------------------------------------------- PIL SETUP SUMMARY -------------------------------------------------------------------- version Pillow 2.5.3 platform darwin 2.7.9 (default, Feb 19 2015, 11:53:52) [GCC 4.2.1 (Apple Inc. build 5577)] -------------------------------------------------------------------- --- TKINTER support available --- JPEG support available *** OPENJPEG (JPEG2000) support not available --- ZLIB (PNG/ZIP) support available --- LIBTIFF support available --- FREETYPE2 support available *** LITTLECMS2 support not available --- WEBP support available *** WEBPMUX support not available --------------------------------------------------------------------
To build pycups on OSX 10.5.x, I had to add the following missing defines to cupsmodule.c
before building:
#define CUPS_FORMAT_AUTO "application/octet-stream" #define CUPS_FORMAT_COMMAND "application/vnd.cups-command" #define CUPS_FORMAT_PDF "application/pdf" #define CUPS_FORMAT_POSTSCRIPT "application/postscript" #define CUPS_FORMAT_RAW "application/vnd.cups-raw" #define CUPS_FORMAT_TEXT "text/plain" #define CUPS_GET_DOCUMENT 0x4027
It fails at runtime on 10.5. (I think that the version of cups in 10.5.x is just too old. I guess we could try to build a newer version of cups and bundle that too.. but at that point, we might as well make our own OS - so we just won't support 10.5 for printing)
$ python -c "import cups" Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: dlopen(/Users/MacAdmin/gtk/inst/lib/python2.7/site-packages/cups.so, 2): Symbol not found: _cupsCreateJob Referenced from: /Users/MacAdmin/gtk/inst/lib/python2.7/site-packages/cups.so Expected in: dynamic lookup
It does run with 10.6 onwards though, see ticket:598#comment:34.
PS: in 0.15, you also need rencode as a separate module.
r8828 added configure switch for libjpeg-turbo to turn on v8 compatibility which now works with python-pillow
PyObjcTools? seems to be missing __init__.py
in ~/gtk/inst/lib/python2.7/site-packages/PyObjCTools
which causes py2app to not bundle it.
You can see the file here though which should be there https://bitbucket.org/ronaldoussoren/pyobjc/src/f4334b2f9f2d0b4a940eb9d0a6f1be98d5db9d02/pyobjc-framework-Cocoa/Lib/PyObjCTools/__init__.py?at=default
Not sure why this is missing will have to look into this some more. Once I added this file everything is packaged again.
My previous builds were always missing this stuff and I believe it may have been causing issues.
r8835 adds patch for pygobject to get rid of annoying messages as noted here
http://winswitch.org/dev/macosx-manual.html
That is the last of the cleanup and everything seems to be running smoothly now
Can't add patches for disutils stuff in jhbuild so I may have to add pycups some other way if I want to patch it to include these constants.
Builds fine when I add them same as you.
I have gone through the steps in comment:58.
I've hit the following https certificate issues (this is a 10.5.x VM), worked around by adding --insecure
to the curl download command from a shell:
(which means that it took me a lot longer than 3 hours to build because of the manual steps)
Also, the versions of many packages is way too old: netifaces, Pillow, Cython, numpy, etc.. (and probably many others too), see comment:43 (which must be out of date by now, but more up to date than the moduleset). Please go through them all as I did before and ensure we use the most up to date version, unless there is a good reason (Twisted had problems with winswitch - but we'll need the latest version to get the ssh fix anyway, so the fix will have to be in winswitch for that one)
I've seen the missing pyobjc warning, PIL is having problems too (module has no attribute 'Image'), and obviously pycups is also missing.
If this helps, I've uploaded this beta osx x86 build as Xpra-0.15.0-r8825-MODULESET.dmg
- beware: because of the issues above, it can't be used for anything useful.
I can go through and add update all packages to their latest version that is not a problem.
The libjpeg problem I think I see the issue is that libjpeg.*.dylib needs to be added to Xpra.bundle and possible libz.*.dylib
Svn is marking this file as a binary file for some reason so I missed it when I was doing a diff.
Can you try adding those 2 and making another build that will probably fix up the PIL issue.
As I said in comment:62 pyobjctools seems to be missing init.py in site-packages after it is installed.
r8873:r8889 updates versions in this moduleset
I will have to rebuild and test it all again to make sure everything is fine.
For the record (and for my own reference), I have updated my OSX build machine with the latest:
libwebp.lib
and libwebp.dll
)
Not updating pyopengl - will wait for the 3.1.1 proper.
The win32 build VM needed most of the same updates (added difficulty: both the python 2.7 and python 3.4 environments), done them too.
After that I uploaded a new beta build made from those environments, this may help in comparing with what you're getting (or with tickets such as #598).
FWIW: the OSX "moduleset" based builds are not usable at this point. So I've uploaded both a "MODULESET" build and a "normal" one in the osx beta area. PS: Some NS Autorelease Pool errors (in both builds now), somewhat similar to __NSAutoreleaseNoPool error … - just leaking. I haven't seen those before, or at least not recently, so this may be related to the update of pyobjc. But I will try the fix suggested in that link.
r8903 rolled back pyobjc stuff to older versions however still seeing these kinds of errors
_NSAutoreleaseNoPool(): Object 0x1803a660 of class NSCFDictionary autoreleased with no pool in place - just leaking
I figured it would be an updated version causing this problem. the link in comment:68 said something about matplotlib but we don't use this. Could the problem be with numpy?
r8904 rolls back numpy to 1.8.1 and still I get the same error messages as comment:69
Not sure where to look from here maybe we do need to fix this properly?
Tried my client with --speaker=off
No more NSAutoreleaseNoPool messages. Must be something new in sound that causes this?
my attempts at getting rid of the warnings
The patch above doesn't help... As per Working with threads, it sounds like this is a threading issue but we already process all the packets in the main thread... not sure what to do (see ticket:669#comment:24).
New update: libvpx 1.4.0, see #832. (download location has changed)
I don't see libvorbis anywhere in the moduleset, but this one has been bumped to 1.3.5 last month.
Pillow 2.8.1 has been released...
@smo: FWIW, I am giving this a go on a clean OSX 10.9 (aka Mavericks) + Xcode 6.2 and the command line tools.
Using these changes to .jhbuildrc-custom
:
setup_sdk(target="10.9", sdk_version="10.9", architectures=["x86_64"]) os.environ["CC"] = "/usr/bin/gcc"
It's going to take a while... but so far so good! (the bootstrap stage finished without problems - yay!)
blatant bug caught by newer versions of gcc / clang
Hurdles (not mentioning the dozens of packages which spit out thousands of warnings because this is using clang instead of gcc):
*** Building gtk-quartz-engine *** [44/97]
:
quartz-style.c:1573:45: error: variable 'height' is uninitialized when used here [-Werror,-Wuninitialized] item_rect = CGRectMake (x1, y, x2-x1, height); ^~~~~~
This is fixed by applying attachment/ticket/533/quartz-style-fix.patch. Which we should apply in any case, using an uninitialized variable like this is a bug. It is probably fixed upstream already, so maybe we just need a newer version of the library.
*** Configuring yasm *** [49/97]
:
configure: error: C preprocessor "/lib/cpp" fails sanity check
That's because the configure line looks like this:
./configure --prefix ${JHBUILD_PREFIX} --libdir ${JHBUILD_PREFIX}/lib --build=i386-darwin CC="gcc -arch i386" CXX="g++ -arch i386"
Removing the CC and CXX definitions and using --build=x86_64-darwin
allows it to build (completely removing --build=
might also work).
@smo: do we really need those overrides for the i386 case? Why? If so, how can we make this conditional?
*** Building libtheora *** [53/97]
Similar problem, solved by just adding --build=x86_64-darwin
to the configure line.
Then you hit this problem: https://trac.macports.org/browser/trunk/dports/multimedia/libtheora/files/patch-configure.diff?rev=118032.
*** Configuring libvpx *** [63/97]
Same problem.
Since I am building on 10.9, I used --target=x86_64-darwin13-gcc
.
That's libvpx 1.3, we should try 1.4 to see if that builds ok.
*** Configuring x264 *** [64/97]
Same problem.
*** Configuring ffmpeg *** [66/97]
Same problem. --cpu=i686
should be --cpu=x86_64
*** Building sdl *** [67/97]
:
./include/SDL_syswm.h:58:10: fatal error: 'X11/Xlib.h' file not found
Lots of answers: Can't install PIL after Mac OS X 10.9, but what I did was simply:
ln -sf /System/Library/Frameworks/Tk.framework/Versions/8.5/Headers/X11 ${JHBUILD_PREFIX}/include/
Which is nice because it doesn't require root. But the build then fails with:
./src/video/quartz/SDL_QuartzVideo.h:94:5: error: unknown type name 'CGDirectPaletteRef'
Which is fixed using this: http://hg.libsdl.org/SDL/rev/e9466ead70e5.
This looks like a cleaner fix: http://www.emaculation.com/doku.php/compiling_sheepshaver_basilisk#tidbits. (not tested).
Then it will moan about missing X11/Xmd.h
, and we don't care about this X11 video stuff so just add this to configure: --disable-video-x11
.
*** Configuring gmplib *** [71/97]
Same arch problem.
And then a new obscure one after changing arch to x86_64
:
tmp-invert_limb_table.s:46:1: error: unknown directive .hidden __gmpn_invert_limb_table ^ make[2]: *** [invert_limb_table.lo] Error 1
I tried the latest development version to see if that fixes the problem, sadly not. The solution turns out to be quite simple: just take out the whole arch thing and let the configure script guess what to do.
*** Building mpfr *** [72/97]
:
Note: I didn't wait to see if it was going to build, because it's wrong to build for i386... Same thing: just removed arch completely to build it.
*** Building python-pyobjc-core *** [91/97]
:
Obscure errors:
error: missing ',' between enumerators error: missing ',' between enumerators In file included from Modules/objc/block_support.m:11: In file included from Modules/objc/pyobjc.h:14: In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/Foundation.framework/Headers/Foundation.h:52: In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/Foundation.framework/Headers/NSOperation.h:6: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/usr/include/sys/qos.h:128:4: error: redefinition of enumerator '__AVAILABILITY_INTERNAL__MAC_10_10' __QOS_CLASS_AVAILABLE_STARTING(__MAC_10_10, __IPHONE_8_0) = 0x19, ^ ...
I give up on this one, and the ones that depend on it (pyobjc cocoa, pyobjc quartz). We need it so I will have to find a fix.
*** Building pygtkglext *** [95/97]
:
gdkglext.override:36:10: fatal error: 'GL/gl.h' file not found #include <GL/gl.h>
Solved in a similar way to the X11 header problem:
ln -sf /System/Library/Frameworks/OpenGL.framework/Versions/A/Headers ${JHBUILD_PREFIX}/include/GL
@smo: a lot of the problems are because of the configure arguments, can't we use append_autogenargs
for this? Then have one custom file for i386 and one for x86_64?
Small nitpick: when installing gtk-mac-bundler
, it doesn't really install anything but links back to where the source is. Which means that when cleaning gtk/source
... we break it. It would be worth copying to gtk/inst/lib
and running install from there.
Important: we do need some of those updates. Pillow 2.8 for example.
I installed pycups using easy_install, because I didn't see in the moduleset.. Then I hit disk space issues... made some space. Then I hit a PIL.Image packaging issue, needed to add libz to the bundle list, fixed in r9089. I am moving the 64-bit only issues to #840 (not many at all - at least so far), link to a beta 64-bit build there. Then I bumped a bunch of libraries: r9091 + r9092.
@smo: I am happy with this and I think we can finally close the ticket. I'm not sure if I will be using it for the 0.15.x release, but going forward I will definitely be using it for 64-bit builds (#840). Either close and follow up there, or re-assign to me if you have more fixes / updates.
r9116 Many version updates from upstream r9119 libffi version downgrade r9120 + r9121 gtk-mac-integration version downgrade r9122 ffmpeg downgrade to latest in 2.5 branch since 2.6 requires newer libvpx and this won't build on osx 10.5
r9123 remove i386 build options since they are only needed when compiling 32bit on 64bit systems.
I've tested these being removed on a 32bit system and all is fine without them.
This should help us moving to ticket #840. There are some version of libs that we can build on newer systems that we cannot for 32bit builds on 10.5 so we may have to split this up a bit going forward.
I tried to keep it as easy to update from upstream as possible so we can help test their stable builds as well.
Pycups requires a patch on osx 10.5 which is not possible with distutils instructions through jhbuild I may be able to hack this up though.
r9124 commited the patch you put here for gtk-quartz-engine can't hurt r9125 add that patch for the moduleset
I'm going to rebuild the lot from scratch again and if I don't have any serious issues with the 32bit builds I will close this and move on to the 64bit ticket.
@smo:
the downgrade to apr and apr-util in r9116 wasn't intentional I guess I missed that one when I was looking at it. This is probably something that should be fixed as apr and apr-util are now in bootstrap so they aren't necessary in xpra.modules. I will look into this further.
I will have to try ffmpeg 2.6.1 again and see what the error was it was related to vp9.
I'm also in the process of cleaning this up a bit. Made some mess when I was dealing with patches.
My bad about ffmpeg 2.6.1 those were warnings and jhbuild wasn't printing out the actual error which was
LD libavformat/libavformat.56.dylib HTML doc/ffmpeg-utils.html HTML doc/ffmpeg-scaler.html HTML doc/ffmpeg-resampler.html Unknown open() mode ':encoding(utf-8-strict)' at /usr/bin/texi2html line 11111. make: *** [doc/ffmpeg-utils.html] Error 255 make: *** Waiting for unfinished jobs.... Unknown open() mode ':encoding(utf-8-strict)' at /usr/bin/texi2html line 11111. make: *** [doc/ffmpeg-scaler.html] Error 255 Unknown open() mode ':encoding(utf-8-strict)' at /usr/bin/texi2html line 11111. make: *** [doc/ffmpeg-resampler.html] Error 255
adding --disable-doc to the configure line seems to fix this issue so I will commit those fixes
Still a couple things to do here.
(just setting milestone)
Tested with a completely clean user account on my 10.5.8 VM and got this error:
configure: error: Oops, mp_limb_t is 32 bits, but the assembler code in this configuration expects 64 bits. You appear to have set $CFLAGS, perhaps you also need to tell GMP the intended ABI, see "ABI and ISA" in the manual. *** Error during phase configure of gmplib: ########## \ Error running ./configure --prefix /Users/osx/gtk/inst --libdir '/Users/osx/gtk/inst/lib' *** [71/97]
Fixed by re-adding --build=i386-darwin
. That's odd, is anything else building for 64 bit??
@smo: we may have to re-add this one and workaround it for 64-bit.
Another really annoying problem is that the apr downloads seem to disappear with new releases (see http://www.eu.apache.org/dist/apr/), which is a really bad idea for those building from source like us, maybe we should mirror those? Otherwise we run the risk of not being able to build things in the future. I've used apr 1.5.2 for building and updated the moduleset, also updated subversion to 1.8.13
I have just uploaded an osx beta build (tagged with r9198) made from this moduleset environment. @afarr: testing it wouldn't hurt, making sure there are no regressions from previous "non moduleset" builds.
PS: unless problems are reported against these builds, from now on all the 0.15 builds (and later) will be made from this "moduleset" environment.
Just noticed an important update missing: gtk 2.24.27, see:
There are some crashes fixed in these releases.
r9279 updates gtk to 2.24.27 builds fine
I'm going to close this now while I can and move on to 64bit builds.
@smo: FYI r10091 bumped x264 to a newer stable snapshot. This caused hard to decipher problems at package time because some dylibs still referred to the old one. (libx264.142.dylib
vs libx264.146.dylib
)
Fixed by rebuilding gstreamer with:
jhbuild buildone -f
I wished jhbuild had the equivalent of revdep-rebuild under Gentoo. There might be a way of cooking up a check script using otool to verify all the dylibs found in gtk/inst/lib can be loaded. Until then, we'll fix things when they break and hope the breakage manifests itself at package time rather than at runtime..
Similar dependency problem in #971.
Follow up for 64-bit in #840.
@smo: FYI lots of updates in r11631.
Note: the pygtkglext patch should be updated to support "get_depth" which we commented out: #1443
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/533