Xpra: Ticket #970: switch to gstreamer 1.x on osx

Follow up from #849.

This should not require any real code changes, mostly build env installation and configuration, and of course: packaging..

We probably want to support both 0.10 and 1.x for a while, we could even ship both so that we can switch at runtime.

Related to #678, #533.



Thu, 17 Sep 2015 17:52:07 GMT - Smo:

I will look into how difficult to add this to the moduleset some of it might even be there we would just be missing plugins-ugly I believe.

As for Win32 I will look into what is involved here. I would prefer if we can build these ourselves instead of downloading another old binary dist.


Fri, 18 Sep 2015 06:50:07 GMT - Antoine Martin: priority changed

win32: it's a bit confusing isn't it?

The important thing for us is how well this is going to play with building other things from source. (#678?) We also need pygobject3 ("gi" bindings) to access gstreamer 1.x, no idea how to build that or what dependencies are needed (see below for osx).


OSX:

After that, I can run things against gstreamer 1.x (not bothering with "ugly" just yet, vorbis is ok anyway):

$ XPRA_GSTREAMER1=1 ./xpra/sound/gstreamer_util.py
(gst-plugin-scanner:64798): GStreamer-WARNING **: Failed to load plugin '/Users/osx/gtk/inst/lib/gstreamer-0.10/libgstfieldanalysis.so': dlopen(/Users/osx/gtk/inst/lib/gstreamer-0.10/libgstfieldanalysis.so, 2): Library not loaded: /Users/osx/gtk/inst/lib/liborc-0.4.0.dylib
  Referenced from: /Users/osx/gtk/inst/lib/gstreamer-0.10/libgstfieldanalysis.so
  Reason: Incompatible library version: libgstfieldanalysis.so requires version 24.0.0 or later, but liborc-0.4.0.dylib provides version 23.0.0
(gst-plugin-scanner:64798): GStreamer-WARNING **: Failed to load plugin '/Users/osx/gtk/inst/lib/gstreamer-0.10/libgstvideofiltersbad.so': dlopen(/Users/osx/gtk/inst/lib/gstreamer-0.10/libgstvideofiltersbad.so, 2): Library not loaded: /Users/osx/gtk/inst/lib/liborc-0.4.0.dylib
  Referenced from: /Users/osx/gtk/inst/lib/gstreamer-0.10/libgstvideofiltersbad.so
  Reason: Incompatible library version: libgstvideofiltersbad.so requires version 24.0.0 or later, but liborc-0.4.0.dylib provides version 23.0.0
Loaded Python GStreamer version 1.4.3 for Python 2.7
GStreamer plugins found: (...)
GStreamer version: 1.4.3.0
PyGStreamer version: 3.14.0
encoders supported: vorbis, wav, wavpack, speex
decoders supported: vorbis, wav, wavpack, speex

@smo: if you can reproduce this and tweak the modulesets as per the comments above (in particular - fixing orc..), then I'll take a look at packaging and changing the default to 1.x for OSX. Raising the priority as this should be complete for the 0.16 release.


Thu, 08 Oct 2015 21:45:16 GMT - Smo:

I found the issue with orc the id for orc in gtk-osx-gstreamer.modules is 'liborc' while ours is just 'orc'

Perhaps I should just remove ours and update it to use liborc instead and update the version on that one to 0.4.24


Fri, 09 Oct 2015 10:12:34 GMT - Antoine Martin:

I found the issue with orc the id for orc in gtk-osx-gstreamer.modules is 'liborc' while ours is just 'orc' Perhaps I should just remove ours and update it to use liborc instead and update the version on that one to 0.4.24


Ah! Right, let's use the same name as upstream and update to 0.4.24


Thu, 15 Oct 2015 19:49:39 GMT - Smo:

Changed orc to liborc in r10855 which is what it is called in the moduleset since ours is loaded last this should override it


Fri, 23 Oct 2015 06:41:20 GMT - Antoine Martin:

Blocker for #987


Tue, 27 Oct 2015 22:20:39 GMT - Smo:

update for pygobject3 3.16.2 in r10857 builds no problem


Tue, 27 Oct 2015 22:23:09 GMT - Smo:

Version updates for gstreamer all built successfully

r11062 gstreamer-1.0 1.4.5 r11063 gst-plugins-base-1.0 1.4.5 r11064 gst-plugins-good-1.0 1.4.5 r11065 gst-plugins-bad-1.0 1.4.5


Tue, 27 Oct 2015 22:27:02 GMT - Smo:

r11066 gst-plugins-ugly-1.0 1.4.5


Tue, 27 Oct 2015 22:29:44 GMT - Smo:

PYTHONPATH=./ XPRA_GSTREAMER1=1 ./xpra/sound/gstreamer_util.py
Loaded Python GStreamer version 1.4.5 for Python 2.7
GStreamer plugins found: 3gppmux, aacparse, ac3parse, accurip, adder, adpcmdec, adpcmenc, agingtv, aiffmux, aiffparse, alawdec, alawenc, alpha, alphacolor, amrparse, apedemux, appsink, appsrc, asfdemux, asfmux, asfparse, aspectratiocrop, asteriskh263, audioamplify, audiochannelmix, audiochebband, audiocheblimit, audioconvert, audiodynamic, audioecho, audiofirfilter, audioiirfilter, audioinvert, audiokaraoke, audiomixer, audiopanorama, audioparse, audiorate, audioresample, audiosegmentclip, audiotestsrc, audiowsincband, audiowsinclimit, auparse, autoaudiosink, autoaudiosrc, autoconvert, autovideoconvert, autovideosink, autovideosrc, avidemux, avimux, avisubtitle, bayer2rgb, bin, breakmydata, bulge, burn, bz2dec, bz2enc, cairooverlay, camerabin, capsfilter, capssetter, checksumsink, chopmydata, chromahold, chromium, circle, clockoverlay, coloreffects, combdetect, compare, compositor, cpureport, cutter, dashdemux, dataurisrc, dcaparse, debugspy, decklinksink, decklinksrc, decodebin, deinterlace, deinterleave, dicetv, diffuse, dilate, diracparse, dodge, downloadbuffer, dtmfsrc, dvbsuboverlay, dvdlpcmdec, dvdspu, dvdsubdec, dvdsubparse, dynudpsink, edgetv, encodebin, equalizer-10bands, equalizer-3bands, equalizer-nbands, exclusion, faac, faad, fakesink, fakesrc, fdsink, fdsrc, festival, fieldanalysis, filesink, filesrc, fisheye, flacdec, flacenc, flacparse, flactag, flvdemux, flvmux, flxdec, fpsdisplaysink, freeverb, funnel, gamma, gaussianblur, gdkpixbufdec, gdkpixbufoverlay, gdkpixbufsink, gdpdepay, gdppay, giosink, giosrc, giostreamsink, giostreamsrc, goom, goom2k1, h263parse, h264parse, h265parse, icydemux, id3demux, id3mux, identity, imagefreeze, input-selector, interaudiosink, interaudiosrc, interlace, interleave, intersubsink, intersubsrc, intervideosink, intervideosrc, irtspparse, ismlmux, ivfparse, ivtc, jifmux, jp2kdecimator, jpegdec, jpegenc, jpegparse, kaleidoscope, lamemp3enc, level, liveadder, mad, marble, matroskademux, matroskamux, matroskaparse, midiparse, mirror, mj2mux, mp4mux, mpeg4videoparse, mpegaudioparse, mpegpsdemux, mpegpsmux, mpegtsmux, mpegvideoparse, mssdemux, mulawdec, mulawenc, multifdsink, multifilesink, multifilesrc, multipartdemux, multipartmux, multiqueue, multisocketsink, multiudpsink, mxfdemux, navigationtest, navseek, oggaviparse, oggdemux, oggmux, oggparse, ogmaudioparse, ogmtextparse, ogmvideoparse, optv, oss4sink, oss4src, osxaudiosink, osxaudiosrc, output-selector, pcapparse, perspective, pinch, pipeline, playbin, playsink, pngdec, pngenc, pngparse, pnmdec, pnmenc, pnmsrc, progressreport, pushfilesrc, qtdemux, qtmoovrecover, qtmux, quarktv, queue, queue2, rademux, radioactv, rdtdepay, rdtmanager, removesilence, revtv, rfbsrc, rganalysis, rgb2bayer, rglimiter, rgvolume, rippletv, rmdemux, rndbuffersize, rotate, rsvgdec, rsvgoverlay, rtpL16depay, rtpL16pay, rtpL24depay, rtpL24pay, rtpac3depay, rtpac3pay, rtpamrdepay, rtpamrpay, rtpasfdepay, rtpasfpay, rtpbin, rtpbvdepay, rtpbvpay, rtpceltdepay, rtpceltpay, rtpdec, rtpdtmfdepay, rtpdtmfmux, rtpdtmfsrc, rtpdvdepay, rtpdvpay, rtpg722depay, rtpg722pay, rtpg723depay, rtpg723pay, rtpg726depay, rtpg726pay, rtpg729depay, rtpg729pay, rtpgsmdepay, rtpgsmpay, rtpgstdepay, rtpgstpay, rtph263depay, rtph263pay, rtph263pdepay, rtph263ppay, rtph264depay, rtph264pay, rtpilbcdepay, rtpilbcpay, rtpj2kdepay, rtpj2kpay, rtpjitterbuffer, rtpjpegdepay, rtpjpegpay, rtpmp1sdepay, rtpmp2tdepay, rtpmp2tpay, rtpmp4adepay, rtpmp4apay, rtpmp4gdepay, rtpmp4gpay, rtpmp4vdepay, rtpmp4vpay, rtpmpadepay, rtpmpapay, rtpmparobustdepay, rtpmpvdepay, rtpmpvpay, rtpmux, rtppcmadepay, rtppcmapay, rtppcmudepay, rtppcmupay, rtpptdemux, rtpqcelpdepay, rtpqdm2depay, rtprtxqueue, rtprtxreceive, rtprtxsend, rtpsbcdepay, rtpsbcpay, rtpsession, rtpsirendepay, rtpsirenpay, rtpspeexdepay, rtpspeexpay, rtpssrcdemux, rtpstreamdepay, rtpstreampay, rtpsv3vdepay, rtptheoradepay, rtptheorapay, rtpvorbisdepay, rtpvorbispay, rtpvp8depay, rtpvp8pay, rtpvrawdepay, rtpvrawpay, rtpxqtdepay, rtspreal, rtspsrc, rtspwms, sbcparse, scaletempo, scenechange, sdpdemux, shagadelictv, shapewipe, shmsink, shmsrc, simplevideomark, simplevideomarkdetect, sirendec, sirenenc, smooth, smpte, smptealpha, solarize, spacescope, spectrascope, spectrum, speed, speexdec, speexenc, sphere, splitfilesrc, square, srtenc, ssaparse, stereo, streaktv, streamsynchronizer, stretch, subparse, subtitleoverlay, synaescope, taginject, tcpclientsink, tcpclientsrc, tcpserversink, tcpserversrc, tee, testsink, textoverlay, textrender, theoradec, theoraenc, theoraparse, timeoverlay, tsdemux, tsparse, tunnel, twirl, twolamemp2enc, typefind, udpsink, udpsrc, uridecodebin, valve, vc1parse, vertigotv, videoanalyse, videobalance, videobox, videoconvert, videocrop, videodiff, videoflip, videomedian, videomixer, videoparse, videorate, videoscale, videosegmentclip, videotestsrc, viewfinderbin, vmncdec, volume, vorbisdec, vorbisenc, vorbisparse, vorbistag, vp8dec, vp8enc, vp9dec, vp9enc, warptv, watchdog, waterripple, wavenc, wavescope, wavpackdec, wavpackenc, wavpackparse, wavparse, webmmux, webpdec, webvttenc, wrappercamerabinsrc, x264enc, xingmux, y4mdec, y4menc, yadif, zebrastripe
GStreamer version: 1.4.5.0
PyGStreamer version: 3.16.2
encoders supported: vorbis, mp3, wav, wavpack, speex
decoders supported: vorbis, mp3, wav, wavpack, speex

Tue, 27 Oct 2015 22:30:24 GMT - Smo: owner changed


Wed, 28 Oct 2015 16:00:41 GMT - Antoine Martin: status changed

Mostly done in r11074 + r11075. Why or why does everything take so long on OSX... (this changes how we load gstreamer, I have checked for regressions on win32 and Linux)

Issues:

I tried disabling everything I could find in an attempt to prevent gstpbutils from linking with the libgstvideo dylib:

./configure --prefix $JHBUILD_PREFIX --libdir $JHBUILD_PREFIX/lib \
    --disable-x --disable-xvideo --disable-videoconvert --disable-videotestsrc \
    --disable-videorate --disable-videoscale --disable-xshm --disable-libvisual \
    --disable-theora --disable-subparse --disable-freetypetest --disable-adder \
    --disable-encoding --disable-cdparanoia --disable-alsa --disable-pango \
    --disable-playback --disable-typefind --disable-gio --disable-tcp --disable-examples

But the dependency is hardcoded in the source... I even tried patching it out in the module to remove the video subfolder, which worked... but then I would have needed to rebuild all the other gstreamer modules too, so I gave up on that and we ship the 1.0 version of the video library.


For testing, you still need to explicitly enable gstreamer 1.0 support using:

XPRA_GSTREAMER1=1 ./Desktop/Xpra.app/Contents/MacOS/Xpra attach ...

This is an edited (removing noise) summary of what I get with:

$ XPRA_SOUND_DEBUG=1 XPRA_GSTREAMER1=1 ./Desktop/Xpra.app/Contents/Helpers/GStreamer_info
trying to import GStreamer 1.0 using <function import_gst1 at 0x1830c70>
Python GStreamer version 1.4.5 for Python 2.7
GStreamer version: 1.4.5.0
PyGStreamer version: 3.14.0
initialized sound codecs:
* vorbis     : vorbisenc   , gdppay      , vorbisdec   , gdpdepay
* wav        : wavenc      , None        , None        , wavparse
* wavpack    : wavpackenc  , None        , wavpackdec  , wavpackparse
* speex      : speexenc    , oggmux      , speexdec    , oggdemux

@smo: if that works for you and you can solve most of the problems, please re-assign to afarr / pvenkateswaralu for testing. Hopefully this helps with #987.


Wed, 28 Oct 2015 16:48:59 GMT - Antoine Martin: owner, status changed

@smo: ready for you to have a go... (see above)


Thu, 29 Oct 2015 18:38:26 GMT - Smo:

Mp3 missing is usually something to do with gst-plugins-ugly. I'm going to fix these few warnings and pass this on to test.

I'll also add the flag for gtk-mac-integration-python


Fri, 30 Oct 2015 02:40:46 GMT - Antoine Martin:

Please also take a look at the gst_audio_ring_buffer_open_device issue. Sound did not run on machines other than the build machine when I tried - a sign that something is missing.


Mon, 09 Nov 2015 09:38:08 GMT - Antoine Martin: priority changed


Tue, 17 Nov 2015 02:52:23 GMT - Antoine Martin:

r11253 makes gstreamer the default on OSX, in part to fix #987 I'm not seeing the warnings from comment:15 any more either, but I haven't tested this very thoroughly (all the codecs, connecting to older servers, etc)


Mon, 23 Nov 2015 17:14:21 GMT - Antoine Martin: summary changed

@smo: not much was happening with this ticket, so I assume that OSX works for you? Did you see or investigate the warnings from comment@15? It is probably safe to close this ticket.


I attempted to get gstreamer 1.x bundled with win32. There are recent pygobjectwin32 installers (3.18.2 rev2 as of last week), which may also help with #640. And if I install the Gst gi bindings, I can load gstreamer 1.x.

The bit problem is that despite some build tweaks (r11313 + r11314 + r11315 + r11316 which will still be helpful for #1041), this doesn't play well with the gstreamer 0.10 bindings: they use different versions of gobject and glib libraries.. but with the same DLL name! (sigh) So we cannot easily bundle them together. I tried modified the gstreamer loader to inject a per-version DLL path into the environment before loading, but that doesn't seem to be honoured by the loader which uses the path of the executable before anything else.. It also seems to look in Xpra/lib/gstreamer-1.0, no matter what I set for GST_PLUGIN_PATH...

(see also the ugly code in the bundle BAT file which overwrites the old ossbuild gstreamer dlls with the more up to date ones from GTK... which makes it harder to split gstreamer 0.10 into its own directory)

We can't use the same hack that we do on OSX using symlinks.. because win32 doesn't support "1980s innovation" in any meaningful way.

So I am moving win32 support to #1041


Tue, 15 Dec 2015 00:54:11 GMT - alas:

I suppose this would be the place to point out that the later 0.16.0 osx builds in the xpra/beta repos (r11261 & r11304) have no mp3 codec? (Meanwhile the 0.16.0 r11206, and presumably several earlier builds, do have mp3 available.)


Tue, 15 Dec 2015 23:45:52 GMT - Smo: owner changed

Can't seem to reproduce the gst_audio_ring_buffer_open_device you've been seeing but here is what I have found out.

If I mount the dmg and run XPRA_SOUND_DEBUG=1 XPRA_GSTREAMER1=1 ./Contents/Helpers/GStreamer_info

On my build system it looks mostly okay looks like its still missing something.

skipping flac with GStreamer 1.x to avoid obscure 'not-neogtiated' errors I do not have time for
missing ['mp3parse'] from ('lamemp3enc', None, 'mad', 'mp3parse')
missing ['opusenc', 'opusdec'] from ('opusenc', 'oggmux', 'opusdec', 'oggdemux')
initialized sound codecs:
* vorbis     : vorbisenc   , gdppay      , vorbisdec   , gdpdepay
* mp3        : lamemp3enc  , None        , mad         , mpegaudioparse
* wav        : wavenc      , None        , None        , wavparse
* wavpack    : wavpackenc  , None        , wavpackdec  , wavpackparse
* speex      : speexenc    , oggmux      , speexdec    , oggdemux

However with 2 other people running them on osx el capitan. This is the output they get.

skipping flac with GStreamer 1.x to avoid obscure 'not-neogtiated' errors I do not have time for
missing ['lamemp3enc', 'mad', 'mp3parse'] from ('lamemp3enc', None, 'mad', 'mp3parse')
missing ['lamemp3enc', 'mad'] from ('lamemp3enc', None, 'mad', 'mpegaudioparse')
missing ['opusenc', 'opusdec'] from ('opusenc', 'oggmux', 'opusdec', 'oggdemux')
initialized sound codecs:
* vorbis     : vorbisenc   , gdppay      , vorbisdec   , gdpdepay
* wav        : wavenc      , None        , None        , wavparse
* wavpack    : wavpackenc  , None        , wavpackdec  , wavpackparse
* speex      : speexenc    , oggmux      , speexdec    , oggdemux
encoders supported: vorbis, wav, wavpack, speex
decoders supported: vorbis, wav, wavpack, speex

Any suggestions?


Wed, 16 Dec 2015 01:16:09 GMT - Antoine Martin: owner changed

you should be able to test by moving your /gtk/ installation out of the way.

Note: XPRA_GSTREAMER1=1 is now the default already on all platforms except win32.


Fri, 18 Dec 2015 00:03:23 GMT - Smo:

Found the missing dylibs looks like we need libgstlame.so and libgstmad.so just like in 0.10

This has been tested and it works still have messages about missing stuff however

GStreamer version: 1.4.5.0
PyGStreamer version: 3.16.2
skipping flac with GStreamer 1.x to avoid obscure 'not-neogtiated' errors I do not have time for
missing ['mp3parse'] from ('lamemp3enc', None, 'mad', 'mp3parse')
missing ['opusenc', 'opusdec'] from ('opusenc', 'oggmux', 'opusdec', 'oggdemux')
initialized sound codecs:
* vorbis     : vorbisenc   , gdppay      , vorbisdec   , gdpdepay
* mp3        : lamemp3enc  , None        , mad         , mpegaudioparse
* wav        : wavenc      , None        , None        , wavparse
* wavpack    : wavpackenc  , None        , wavpackdec  , wavpackparse
* speex      : speexenc    , oggmux      , speexdec    , oggdemux

I would like to add opus which is from plugins-bad. It will require adding opus to the moduleset so maybe we open a new ticket for this?


Fri, 18 Dec 2015 00:03:51 GMT - Smo: attachment set


Fri, 18 Dec 2015 00:10:44 GMT - Smo: owner changed


Fri, 18 Dec 2015 03:16:08 GMT - Antoine Martin: owner changed

I was explicitly skipping those two plugins because of the problems I was seeing but rebuilding with jhbuild build root-gst-plugins-ugly-1.0 then making a new build seems to work fine (tested on the build machine as another user, then on a clean test VM). So I have applied this patch in trunk in r11426, and then in 0.16.x in r11427.

As for the other missing plugins:

@afarr: just a FYI, please close, all 0.16.x and trunk OSX builds should now have most codecs:

initialized sound codecs:
* vorbis     : vorbisenc   , gdppay      , vorbisdec   , gdpdepay
* mp3        : lamemp3enc  , None        , mad         , mpegaudioparse
* wav        : wavenc      , None        , None        , wavparse
* wavpack    : wavpackenc  , None        , wavpackdec  , wavpackparse
* speex      : speexenc    , oggmux      , speexdec    , oggdemux

Mon, 04 Jan 2016 18:04:15 GMT - J. Max Mena: status changed; resolution set

Noted. Closing.

(also double checked codecs in latest OSX beta build, and confirmed that I see these codecs)


Tue, 05 Jan 2016 09:10:19 GMT - Antoine Martin:

for opus support on osx, see #1074 and ticket:1204#comment:1


Sat, 23 Jan 2021 05:11:13 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/970