xpra icon
Bug tracker and wiki

Opened 19 months ago

Closed 18 months ago

Last modified 12 months ago

#1194 closed enhancement (fixed)

mpeg4 audio support

Reported by: Antoine Martin Owned by: alas
Priority: major Milestone: 1.0
Component: sound Version: trunk
Keywords: Cc:

Description

Based on mp4mux and qtdemux, we should be able to forward sound in an mpeg container.

Example pipelines found here: https://github.com/pedwo/gst-sh-mobile/blob/master/examples/examples.sh:

# Encode microphone input as mp4 AAC file.
gst-launch -e \
 alsasrc ! audio/x-raw-int, rate=16000, channels=2 \
 ! queue ! audioconvert ! faac profile=LC bitrate=192000 \
 ! queue ! mp4mux ! filesink location=./test.mp4
# mp4 AAC file playback
gst-launch \
 filesrc location=./test.mp4 \
 ! qtdemux ! queue ! faad ! audioconvert ! audioresample ! autoaudiosink

Attachments (1)

add-aac-mpeg4.patch (3.5 KB) - added by Antoine Martin 19 months ago.
work in progress patch: try to add aac codec and mpeg4 muxer

Download all attachments as: .zip

Change History (11)

Changed 19 months ago by Antoine Martin

Attachment: add-aac-mpeg4.patch added

work in progress patch: try to add aac codec and mpeg4 muxer

comment:1 Changed 19 months ago by Antoine Martin

Owner: changed from Antoine Martin to alas

Implemented in r12565: we now have aac+gdp and aac+mpeg4.
Both work OK on Fedora if you have all the required plugins installed (if not, run xpra/sound/gstreamer-util.py -v for diagnostics, then find the required rpm packages to install.

This is what I get:

$ xpra attach --speaker-codec=help
speaker codecs available: opus+gdp, opus, vorbis, vorbis+mka, flac+gdp, mp3, aac+gdp, aac+mpeg4, wav, wavpack, speex+gdp, speex

Since this seems to work well, I've ranked it higher than wav and wavpack, just after mp3.
r12574 adds support for those new formats to the win32 and osx clients and servers.
Minor improvement follow up in #1196.

Last edited 19 months ago by Antoine Martin (previous) (diff)

comment:2 Changed 18 months ago by alas

Owner: changed from alas to Antoine Martin

Well, I'm getting the same response client side (using the 0.18.0 r12587 osx client in your repo), but I can't seem to find the rpms I need for a fedora 23 vm 0.18.0 server.

Google has directed me to rpmfusion (gstremaer-plugins-good/bad/ugly), gstreamer1-libav, gstreamer-ffmpeg, gstreamer-plugins-bad-nonfree, gstreamer1-plugins-bad-free-extras, vlc, & rygel... but I continue to get this message:

missing ['mp3parse'] from ('lamemp3enc', None, 'mp3parse ! mad', None)
missing ['faac', 'faad'] from ('faac', 'gdppay', 'faad', 'gdpdepay')
missing ['avenc_aac', 'avdec_aac'] from ('avenc_aac', 'gdppay', 'avdec_aac', 'gdpdepay')
missing ['faac', 'faad'] from ('faac', 'mp4mux', 'faad', 'qtdemux')
missing ['avenc_aac', 'avdec_aac'] from ('avenc_aac', 'mp4mux', 'avdec_aac', 'qtdemux')

I even tried a sudo dnf install gstreamer* --exclude=*devel*,*doc* - still no luck.

I presume that I'm just not finding the right things to search for... so I'll assign this back to you and leave you to decide to close (and I can take some more stabs at finding the right rpms when I get the chance) or perhaps provide a hint where to try next.

comment:3 Changed 18 months ago by Antoine Martin

Owner: changed from Antoine Martin to alas

Odd, the aac codec is part of libav (see: https://www.rpmfind.net/linux/rpm2html/search.php?query=gstreamer1(element-avdec_aac)), which should have been installed from your gstreamer* glob, try explicitly:

sudo dnf install gstreamer1-libav

Then verify with

gst-inspect-1.0 avenc_aac
gst-inspect-1.0 avdec_aac

comment:4 Changed 18 months ago by alas

I feel a little less like I'm just missing something... or maybe I'm missing something unbelievably obvious?

sudo dnf install gstreamer1-libav
Last metadata expiration check: 0:34:17 ago on Thu May 19 15:48:42 2016.
Package gstreamer1-libav-1.6.3-1.fc23.x86_64 is already installed, skipping.
Dependencies resolved.
Nothing to do.
Complete!

...

[jimador@jimador bin]$ gst-inspect-1.0 avenc_aac
No such element or plugin 'avenc_aac'
[jimador@jimador bin]$ gst-inspect-1.0 avdec_aac
No such element or plugin 'avdec_aac'

Uhhh?

comment:5 Changed 18 months ago by Antoine Martin

NOT obvious at all.
Trying again in a clean virtual machine after installing rpmfusion and gstreamer1-libav:

gst-inspect-1.0 /usr/lib64/gstreamer-1.0/libgstlibav.so | grep aac

Shows the aac plugins, but they aren't present with a plain gst-inspect-1.0 | grep aac.

So then I wrote a script (run as root) to delete all the plugins and then re-add them one by one to find out which one is preventing us from loading the libav plugin:

#!/usr/bin/bash
#stash them away:
mkdir /usr/lib64/gstreamer-1.0.bak/
mv /usr/lib64/gstreamer-1.0/* /usr/lib64/gstreamer-1.0.bak/
#restore the libav one:
cp /usr/lib64/gstreamer-1.0.bak/*libav* /usr/lib64/gstreamer-1.0/
for x in `ls /usr/lib64/gstreamer-1.0.bak/`; do
        mv /usr/lib64/gstreamer-1.0.bak/$x /usr/lib64/gstreamer-1.0/
        #check if aac is still found:
        gst-inspect-1.0 avenc_aac >& /dev/null
        if [ "$?" != "0" ]; then
                echo "aac missing after adding $x"
                exit 1
        fi
        echo "* $x"
done

And to my surprise, it ran without a hitch.
After running this script, the aac plugins are found without problems every time.
There must be something broken with the gstreamer libav plugin packaging, or with the plugin probing.
You may be able to fix your installation with just:

touch /usr/lib64/gstreamer-1.0/*

If not, run the script. If that fixes things for you, please file an upstream ticket for this.

Last edited 18 months ago by Antoine Martin (previous) (diff)

comment:6 Changed 18 months ago by alas

Touch, unfortunately, seems to fail...

[jimador@jimador bin]$ sudo touch /usr/lib64/gstreamer-1.0/*
[jimador@jimador bin]$ gst-inspect-1.0 avenc_aac
sys:1: PyGIWarning: Gst was imported without specifying a version first. Use gi.require_version('Gst', '1.0') before import to ensure that the right version gets loaded.

(gst-plugin-scanner:10366): GStreamer-WARNING **: Failed to load plugin '/usr/lib64/gstreamer-1.0/libgstlibav.so': libx264.so.148: cannot open shared object file: No such file or directory
DtsGetHWFeatures: Create File Failed
sys:1: Warning: cannot register existing type 'GstSmooth'
sys:1: Warning: g_once_init_leave: assertion 'result != 0' failed

(gst-plugin-scanner:10366): GStreamer-CRITICAL **: gst_element_register: assertion 'g_type_is_a (type, GST_TYPE_ELEMENT)' failed
No such element or plugin 'avenc_aac'

Running the script... I get aac missing after adding libgst1394.so ... but trying to run gst-inspect again, I get more No such element or plugin 'avenc_aac'.

I'll see about filing a ticket upstream about the libgst1394.so... after a quick tweak to the script to be sure that's not the first library and all of them are giving me issues.

comment:7 Changed 18 months ago by Antoine Martin

This is an upstream bug in rpmfusion:

(gst-plugin-scanner:10366): GStreamer-WARNING **: Failed to load plugin '/usr/lib64/gstreamer-1.0/libgstlibav.so': libx264.so.148: cannot open shared object file: No such file or directory

You need to install x264-libs, which rpmfusion should include as a dependency of the gstreamer-libav package, as libav will not load without it.

Maybe aac will work after that, or maybe you'll need to run touch again, or run the script again.
libgst1394 is probably a false positive, as the libav and its aac codecs will not work until x264-libs is installed.

comment:8 Changed 18 months ago by alas

Resolution: fixed
Status: newclosed

Ok... looks like sudo dnf install x264-libs was "all" I needed to do (though, it occurs to me I've done all of the above on the same vm, so not sure that it was indeed all that was needed).

Went ahead and grabbed a clean vm to confirm the mess of steps:

  • Install rpmfusion.
  • Install gstreamer1-libav
  • Install x264-libs
  • touch /usr/lib64/gstreamer-1.0/*

And that's "all" I had to do to get something when inspecting.

Closing.

comment:9 Changed 17 months ago by Antoine Martin

Milestone: 0.181.0

Milestone renamed

comment:10 Changed 13 months ago by Antoine Martin

r14401 adds "mp3+mpeg4" which may help with #1341, r14410 + r14415 tries to make the stream more "compatible".
The html5 client should now use mpeg4, see ticket:1341#comment:6 and #845.

Last edited 12 months ago by Antoine Martin (previous) (diff)
Note: See TracTickets for help on using tickets.