xpra icon
Bug tracker and wiki

This bug tracker and wiki are being discontinued
please use https://github.com/Xpra-org/xpra instead.

Version 35 (modified by Antoine Martin, 2 years ago) (diff)


Packet Encoding and Compression

See also:

  • Data Flow: an overview of the data that flows over the network connection
  • Network for the network connection options
  • Network Protocol for the actual contents of the packets

Wire Format

Each message may be split into multiple network packets as large binary chunks (ie: compressed pixel data, large clipboard data, etc) will be sent in their own packet, skipping the generic compression layer.

The compression command line option only affects regular packets, not binary chunks. It is rarely a good idea to change it.

In most cases, the packets are so small that compression is not even attempted. The data that does need to be compressed (ie: pixel data) uses other controls (ie: speed and quality options).

Each chunk is preceded by an 8 byte header containing:

  • the magic value "P" (char value 80 in decimal, 0x50 in hexadecimal)
  • one byte for protocol flags (stream encoder used, encryption, etc)
  • one byte for compression level hint (0 for uncompressed)
  • one byte for the chunk index (0 for the main chunk)
  • one long (4 bytes) for the size of the data that follows

The main chunk containing the message uses index 0 and it is encoded (see bencode / rencode below), and sent last. Each message consists of a list of values, the first item is the packet type (see wiki/NetworkProtocol).

The other chunks replace the items found at the specified index in the main chunk. ie:

  • chunk 1: some png pixel data
  • chunk 0 (the main packet): [draw, ..., placeholder for the png data, ...]

Bencode vs Rencode

The main chunk is encoded using one of those stream encoders:

  • The old bencode mode (see Bencode on wikipedia): a modified bencoder which supports non string keys in dictionaries, non ordered keys, and support for unicode values
  • The newer/faster rencode mode, based on this version with minor tweaks/fixes (packaging info: #683)
  • YAML (see r6934) which is much slower, but more widely available

This allows various languages to implement the xpra protocol, specifying which encoder they want to use (bencode is more widely available than rencode). We also include a faster bencoder implemented in Cython, which is roughly twice as fast as the pure Python version and does make a difference to the overall performance of the system: Cython vs Python bencoder, but rencode is 3 times faster than that still.

The python client and server can specify which packet encoders should be enabled using the switch --packet-encoders= (or via the config file). For backwards compatibility, you should always enable bencode as some clients may use this encoding (ie: html5 client), at least initially.

Pixels and Icons

Note: all window pixels, icons and cursors are usually sent in their own chunk using dedicated picture encodings and bypass the stream encoder completely (for efficiency). Pixel compression is done in its own thread, to improve responsiveness.

The only exception to this rule is the RGB encoding, which will use the stream compression on the raw pixels (but it is still called from the pixel compression thread to preserve responsiveness).


There are 3 compressors supported:

  • zlib - which is always available but very inefficient
  • lz4 - used by default if present, fast! (more information here: #443, how lz4 works)
  • lzo not as efficient as lz4, but still much faster than zlib

All supported platforms have lz4 so zlib and lzo should not be used.

You can select which compressors are enabled using the --compressors= switch (or the equivalent entry in the configuration file).

The compression level can be specified via the command line ("-z LEVEL" or "--compress=LEVEL"). It is generally best to not modify this option unless you really understand the implications.

Here is what the LEVEL does:

  • 0 means no compression at all and is useful when bandwidth is no issue but latency is
  • 1 is the best trade-off and will compress everything without spending too much time on it
  • values of 2 and above increase the compression only slightly and should rarely be needed (CPU usage and latency will increase much more): the picture encoding used is much more important

lz4 vs zlib

There is a benchmark test you can run to measure the performance of lz4 and lzo compared to zlib. On average, lz4 is about 10 times faster than zlib on its fastest setting for regular packets, and even faster for RGB data (up to 60 times faster than zlib!) whilst also compressing better than zlib!
You can also find some more generic statistics here: lz4 vs zlib graphs

A more thorough comparison can be found here: lossless compression products benchmarked (pay particular attention to compression speed, which is the most important aspect for the network layer)


The default settings should end up using lz4 and rencode, since those are by far the best options.

The zlib and bencode options will remain for compatibility with clients which do not have support for them (ie: HTML5)

You can also change the current encoder/compressor at runtime:

  • switch to zlib: xpra control :DISPLAY compression zlib
  • switch to lz4: xpra control :DISPLAY compression lz4
  • switch to lz0: xpra control :DISPLAY compression lz0
  • switch to bencode: xpra control :DISPLAY encoder bencode
  • switch to rencode: xpra control :DISPLAY encoder rencode
  • switch to yaml: xpra control :DISPLAY encoder yaml

More info on xpra control here: #461. You can view the current connection options with:

xpra info | egrep "connection.encoder=|connection.compression="