At the moment, we only use delta compression for non video screen updates using the last suitable region we had. Multiple updates of differing sizes means we will constantly change the delta buffer, and never match anything.
We could let the client tell the server how many buffers it will accept, and of what size. Then the server can keep more delta regions in memory, keyed using their dimensions, and is more likely to make use of them.
This will help with things like unnecessary repaints (#670). The only difficulty will be with UDP and dropped packets #639: we will need a way of requesting a reset / refresh when we are missing a reference region.
Implemented in r8285, see commit message for details.
Remaining tasks:
frees buckets as needed - not sure this is worth doing
More improvements in r8315, r8311, r8309: we now re-stride the data, and directly into the target buffer in Cython, so now much faster. I don't think we want to apply the delta-free patch above: too much complexity and extra code in the hot path, all for small memory savings. If we want to constrain the client memory usage, a better approach would be to let the client specify its own limit for the size of delta buffers. That's on top of being able to choose the number of delta buffers already via:
XPRA_DELTA_BUCKETS=10 xpra attach ..
Ready for testing, #760 may be helpful here to visualize where we use delta regions (and see if we miss any).
Seems to be working as expected (aside from the windows client opengl paint box issue mentioned in #760).
I would note that setting XPRA_DELTA_BUCKETS=11
does seem to lead to increased blurriness, even on a small window on a 1080p display.
I would note that setting XPRA_DELTA_BUCKETS=11 does seem to lead to increased blurriness, even on a small window on a 1080p display.
It is not meant to.
I should explain: delta means that we substract the previous contents of a region from its current contents before compression. So if you end up showing more or less the same contents, you end up with something empty as delta - which compresses very well. Video compressors already do this by default (that's why they compress so well - and also why the size of the video cannot change without a full encoder re-initialization), we use delta compression on lossless picture encodings (rgb24/rgb32+lz4 and png) to benefit from the same trick.
Testing some more with 0.15.0 r8647 win32 client against 0.15.0 r8661 fedora 20 server... it looks like the blurriness when scrolling is resolving itself promptly (perhaps previous tests hit some extra latency... or maybe I was just using a monitor that periodically seems to actually be becoming blurry itself... not certain).
However... testing this I'm suddenly seeing that unicode error from #786 again (in this case 2015-02-20 17:31:10,731 unexpected data type <type 'unicode'> in logging packet: u'using audio codec: MPEG 1 Audio, Layer 3 (MP3)'
), and I'm also seeing another message that I'm also seeing in a number of other places (not sure it's related specifically to this, thought it was a webkit browser related error until just now): 2015-02-20 17:25:13,779 failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'
.
In case it is liable to make matters clear, launched the server with the following:
[jimador@zapopan ~]$ XPRA_OPENGL_PAINT_BOX=1 dbus-launch xpra --no-daemon --bind-tcp=0.0.0.0:1201 --start-child=xterm --start-child=xterm start :23 --remote-logging=on -d client
And the client with the following:
C:\Program Files (x86)\Xpra>SET XPRA_OPENGL_PAINT_BOX=1 C:\Program Files (x86)\Xpra>xpra_cmd.exe attach tcp:10.0.32.53:1201 2015-02-20 17:05:38,502 xpra client version 0.15.0 2015-02-20 17:05:38,986 OpenGL_accelerate module loaded 2015-02-20 17:05:38,986 Using accelerated ArrayDatatype 2015-02-20 17:05:39,095 detected keyboard: layout=us 2015-02-20 17:05:39,095 desktop size is 5120x2160 with 1 screen(s): 2015-02-20 17:05:39,095 '1\WinSta-Default' (1354x571 mm - DPI: 96x96) workarea : 5120x2120 2015-02-20 17:05:39,095 DISPLAY1 3840x2160 at 1280x0 (621x341 mm - DPI: 157x 160) workarea: 3840x2120 2015-02-20 17:05:39,095 DISPLAY2 1280x720 (597x336 mm - DPI: 54x54) workarea : 1280x680 2015-02-20 17:05:39,408 enabled remote logging, see server log file for output
And, with the client also logging to the server, the server initialization output this initially:
2015-02-20 17:05:47,941 Python/Gtk2 Microsoft Windows 8 client version 0.15.0 connected from 'hasenpfeffer' as 'hassenpfeffer' ('hassenpfeffer') 2015-02-20 17:05:47,942 using h264 as primary encoding, also available: vp8, png, png/P, png/L, webp, rgb24, jpeg, rgb32 2015-02-20 17:05:47,948 client root window size is 5120x2160 with 1 displays: 2015-02-20 17:05:47,949 '1\WinSta-Default' (1354x571 mm - DPI: 96x96) workarea: 5120x2120 2015-02-20 17:05:47,949 DISPLAY1 3840x2160 at 1280x0 (621x341 mm - DPI: 157x160) workarea: 3840x2120 2015-02-20 17:05:47,949 DISPLAY2 1280x720 (597x336 mm - DPI: 54x54) workarea: 1280x680 2015-02-20 17:05:48,098 server virtual display now set to 5120x2560 (best match for 5120x2160) 2015-02-20 17:05:48,100 setting key repeat rate from client: 500ms delay / 33ms interval 2015-02-20 17:05:48,101 setting keyboard layout to 'us' 2015-02-20 17:05:48,234 server: Linux Fedora 20 Heisenbug, Xpra version 0.15.0 (r8661) 2015-02-20 17:05:48,308 DPI set to 96 x 96 2015-02-20 17:05:48,310 sent updated screen size to 1 clients: 5120x256I 0 (max 8192x4096) 2015-02-20 17:05:48,313 Attached to tcp:10.0.32.53:1201 (press Control-C to detach) 2015-02-20 17:05:48,333 failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow' 2015-02-20 17:05:48,347 using Pulseaudio device 'Monitor of Null Output' 2015-02-20 17:05:48,361 starting sound stream capture using Pulseaudio source 2015-02-20 17:05:48,362 failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow' 2015-02-20 17:05:50,472 codec: MPEG-1 Layer 3 (MP3) 2015-02-20 17:05:50,494 using audio codec: MPEG 1 Audio, Layer 3 (MP3) 2015-02-20 17:05:50,495 unexpected data type <type 'unicode'> in logging packet: u'using audio codec: MPEG 1 Audio, Layer 3 (MP3)'
Using firefox to test for delta processing, I'm periodically (mostly when opening a new tab, but not exactly correspondingly) seeing the failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'
error repeating.
Not sure if it is a combination of the global variable settings that is triggering these messages or if it is just a matter of my now aging test versions... any insights where to start (I mention here just before we close this delta ticket, because the combination of the global variables and the client debug forwarding seems to have triggered those messages which I was no longer seeing when testing #786 without all the other variables and log forwarding in place)... before I try going through the bits and pieces randomly until I find a combination-culprit?
testing this I'm suddenly seeing that unicode error from #786 again
Tracked in #786.
failed to set group leader: 'module' object has no attribute 'SHGetPropertyStoreForWindow'
This is tracked in #799. If you can reproduce this problem with one of my later builds, please add to that ticket.
As for actual delta compression, I think we can close this ticket?
Testing again for #799, I couldn't reproduce the error(s).
Agreed regarding the actual delta compression.
Closing.
Whilst testing vp9 (#832) and having decoder problems (or maybe encoder problems even..), I think I've hit a bug related to multi-delta:
2015-04-05 19:49:05,883 paint_with_video_decoder: \ wid=1, vp9 decompression error on 44 bytes of picture data for 499x316 pixels using vpx.Decoder(vp9), \ options={'speed': 83, 'frame': 2, 'quality': 67, 'csc': 'YUV444P', 'encoding': 'vp9'} 2015-04-05 19:49:06,251 draw error Traceback (most recent call last): File "/usr/lib64/python2.7/site-packages/xpra/client/ui_client_base.py", line 1962, in _do_draw window.draw_region(x, y, width, height, coding, data, rowstride, packet_sequence, options, [record_decode_time]) File "/usr/lib64/python2.7/site-packages/xpra/client/client_window_base.py", line 423, in draw_region self._backing.draw_region(x, y, width, height, coding, img_data, rowstride, options, callbacks) File "/usr/lib64/python2.7/site-packages/xpra/client/window_backing_base.py", line 472, in draw_region self.paint_with_video_decoder(VIDEO_DECODERS.get(coding), coding, img_data, x, y, width, height, options, callbacks) File "/usr/lib64/python2.7/site-packages/xpra/client/window_backing_base.py", line 376, in paint_with_video_decoder img = self._video_decoder.decompress_image(img_data, options) File "xpra/codecs/vpx/decoder.pyx", line 300, in xpra.codecs.vpx.decoder.Decoder.decompress_image (xpra/codecs/vpx/decoder.c:4578) File "/usr/lib64/python2.7/site-packages/xpra/codecs/codec_constants.py", line 61, in get_subsampling_divs raise Exception("invalid pixel format: %s" % pixel_format) Exception: invalid pixel format: bucket
comment:8 can be ignored, the problem was a dangling pointer caused by a bug in the vpx decoder, fixed in r8940.
r9028 adds a new debug flag for delta compression: -d delta
.
r9063 ensures that we expire all delta buckets after 20 hits to prevent xor decay where the bucket just contains unhelpful noise. The number of hits can be configured using:
XPRA_MAX_DELTA_HITS=20 xpra start ...
This caused a bug: #861. (fixed in r9449)
Related: #2248 (more aggressive use of scroll encoding)
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/756