Xpra: Ticket #246: support vorbis and flac sound encoding

At the moment xpra-0.8.x has incomplete "vorbis" and "flac" audio forwarding support leaving only "mp3" as working alternative (I couldn't make "aac" work on client despite having the same packages installed).

MP3 is just retarded comparing to Vorbis in regards to bandwidth and audio quality (not to mention support for more than 2 channels).

AAC is not-so-free?

Flac is great license- and quality-wise (loseless) so IMHO the best order for codecs would be Vorbis > Flac > MP3 > AAC.

Please support Vorbis and Flac.

Mon, 04 Feb 2013 07:35:34 GMT - Antoine Martin: status, milestone changed; owner set

owner set to Antoine Martin
status changed from new to accepted
milestone changed from 0.8 to 0.9

Yes, this is planned, in fact the code already supports specifying the codecs, the gstreamer support code is mostly there, etc - only needs a little bit more work. But there was not enough time to get this tested properly for 0.8, hopefully this can be completed in time for 0.9. There are other, more important issues to do with sound though: buffering and latency being the main problem.

As can be seen here: browser/tags/v0.8.x/src/xpra/sound/gstreamer_util.py, the codecs that are meant to be supported (eventually) are exactly what you ask for, namely: vorbis, flac, aac and mp3.

Sun, 17 Feb 2013 14:08:01 GMT - Antoine Martin: summary, description, version, milestone changed

summary changed from 0.8.1: please support vorbis and flac encoding. to support vorbis and flac sound encoding
description modified (diff)
version changed from trunk to 0.8.x
milestone changed from 0.9 to 0.8

Wed, 27 Feb 2013 15:54:11 GMT - Antoine Martin:

r2833 adds support for wav and flac. vorbis and aac just stubbornly refuse to work.

Here's how you can test the code from the command line. This is easier than starting a session with a pulseaudio server and all the codec arguments and then starting a media player, etc:

record whatever is playing from your soundcard at the time into "test.mp3":
```
XPRA_SOUND_DEBUG=1 ./xpra/sound/src.py test.mp3
```
verify that the file exists and has valid mp3 data in it:
```
mplayer test.mp3
```

play it back using xpra's code:

XPRA_SOUND_DEBUG=1 ./xpra/sound/sink.py test.mp3

then test with a real session and a pulseaudio server (longer - see wiki/Sound for details), ie:
```
xpra attach --speaker-codec=flac
(...)
using audio codec: FLAC
```

You can repeat this with the other supported file extensions: ".flac" and ".wav".

Now, to add support for more formats (say vorbis, ac3 or aac for example - or maybe even some more obscure ones like amr, gsm or dca?), here is what is needed:

find a pipeline that records a valid file in that format (inc container format), ie:

gst-launch-0.10 pulsesrc device=alsa_output.pci-0000_00_14.2.analog-surround-51.monitor ! \
    vorbisenc ! oggmux ! filesink location=test.vorbis

verify that the file is valid (look for container error messages - the file needs to play "cleanly"):
```
mplayer test.vorbis
```

find the corresponding play pipeline:

gst-launch filesrc location=test.vorbis ! oggdemux ! vorbisdec ! \
    audioconvert ! audioresample ! autoaudiosink

modify CODECS and CODEC_ORDER definitions in xpra.sound.gstreamer_util accordingly, ie:

#format: encoder, formatter, decoder, parser
CODECS = {
            VORBIS : ("vorbisenc", "oggmux", "vorbisdec", "oggdemux"),
...
CODEC_ORDER = [MP3, FLAC, WAV, VORBIS]

as per the comments above, verify that the src.py and sink.py work as expected. The pipeline should be (almost) identical to the one used with gst-launch.

When it does not, like the "vorbis" example above, weep.

Notes:

the workflow above definitely works as this is what I have used to add support for opus, speex and wavpack in r2834
If we need a completely different pipeline to support other formats, we can do that too - it's not too late for that as the code is still fairly small and self-contained. But hopefully we don't have to.
The container format is important, I've used "oggmux" / "oggdemux" for most of the formats because it is one of the most widely used container format and well supported. But there are others... Worth a try. Especially when we start dropping some buffers because of network congestion, the container format will become more important in dealing with those "broken" streams.
The list of gstreamer plugins that you can use can be found here (note that you may need to install extra packages to support all the plugins)
vorbis is a good format, it would be good to find out what is wrong with it..

Thu, 21 Mar 2013 03:02:07 GMT - Antoine Martin: status changed; resolution set

status changed from accepted to closed
resolution set to fixed

Not going to bother with vorbis and aac since there are better alternatives.

As of now, we support:

flac
mp3
wav
opus (*)
speex
wavpack

win32 needs a newer build of gstreamer to support "opus", maybe #299 will help

Sat, 23 Jan 2021 04:49:30 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/246