Picture Encodings

This page is strongly related to WindowRefresh, the client rendering is also relevant.

Introduction

Xpra supports a number of picture encodings, provided you have the required libraries installed. Even then, the features of each encoding may vary based on the version of the libraries and other dependencies, both client and server side.
Here is the list as of v0.10

• lossless encodings:
  • rgb24 (+zlib)
  • png (24/32-bit colour, 8-bit grayscale and 8-bit colour)
  • webp (lossless mode)
• lossy encodings:
  • jpeg
  • webp (lossy mode)
  • vpx
  • x264

Choosing an Encoding

The best thing to do is to try them all and choose the one that provides the best results. Here are some rough guidelines:

• on LANs with 100MBit/s or higher: rgb24 + zlib should give you the best latency possible (whilst consuming quite a lot of bandwidth in the process..)
• to save a little bit of bandwidth at the expense of framerate and latency, use fast lossless encodings: webp or png
• otherwise, choose x264 and tune the speed/quality to suit your needs (see below)

The other encodings are somewhat less useful:

• vpx is similar to x264 but it does not support speed and quality tuning
• webp (lossy mode) is a single image subset of vpx, and therefore lacks ​intra-frame compression - but latency is decent
• jpeg gives lower size/quality than other lossy encodings

---

x264 specifics

(see ​wikipedia H.264) x264 is the encoding that supports the most options and tunings. Quality/speed and minimum quality/speed can be set via the command line (and the latter can also be changed at runtime via the tray applet) Note that even when using the x264 encoding, some small screen updates may get sent as png or rgb24 to save time/bandwidth (not encoding a full frame).

Profiles

(see ​wikipedia h264 profiles) x264 supports the following profiles: baseline, main, high, high10, high422 and high444.

At present, we use high or better by default unless the client overrides it.

Quality Option

What this does should be obvious (in x264 speak, this controls the rc.f_rf_constant parameter), but it is more complicated than you think:

• we support 3 different colourspace modes: YUV420, YUV422 and YUV444 and this affects the quality of the picture too. (YUV420 is used for lowest quality settings). YUV444 uses roughly twice as much bandwidth as YUV420. Switching from one mode to another is expensive, as we then need to send a new key frame and re-initialize both the encoder and the decoder - so the thresholds for going up to the next mode are not the same as the thresholds for going down to the next mode (prevents a yoyo effect).
• not all modes are supported by all profiles, so we need to switch to a different profile to support YUV422 (high422 or high444) and YUV444 (high444 only).
• some builds against older versions of libav/ffmpeg only support YUV420, see x264-limited-csc.patch

Speed Option

This option, shown as "latency" via the tray menu, controls how hard the encoder is going to work at compressing the picture. Working harder means lower bandwidth, but also higher latency. x264 supports the following speed settings: ultrafast, superfast, veryfast, faster, fast, medium, slow, slower, veryslow and placebo. xpra maps the 100% to 0% speed option from ultrafast to slower only. veryslow and placebo are not particularly useful (diminishing returns: much much slower and without sufficient savings for real-time use). ultrafast is only available when setting the encoder speed to 100% manually (via the command line or UI) - note that this setting has side-effects which prevents other settings from behaving as they should if the option is later changed.

---

Tuning

The best way to choose the right options is through wiki/Testing. Though you may find some ​good illustrations online to give you an idea of the trade offs. (ie: ​fps vs noise, ​fps vs size) Be aware that the lossless auto-refresh will trigger a lossless frame encoded using png or webp. If the delay is too low, it may negate the benefits of using efficient compression.

Notes

When comparing performance, make sure that you use the right metrics... The number of updates per second is not always a good one (if there are more small regions, this can be a good or a bad thing), more examples here: Misleading Statistics