Xpra: Ticket #1320: lossless scrolling
Follow up from #1232: we could remove the need for auto-refresh packets when we do scrolling and the original data is lossless.
The same rectangle transformations could be applied to the list of regions needed a refresh.
Fri, 08 Feb 2019 06:00:49 GMT - Antoine Martin: status, milestone changed
- status
changed from new to assigned
- milestone
changed from future to 3.0
Thu, 21 Mar 2019 09:04:37 GMT - Antoine Martin: attachment set
- attachment
set to lossless-scrolling.patch
work in progress patch
Thu, 21 Mar 2019 09:09:45 GMT - Antoine Martin:
Updates:
- r22188 better rectangle hash: prevent collisions, already backported
- r22189 bug fix (almost impossible to trigger), already backported
- r22190 boost non-scroll quality, more so if we have a high match percentage (less to send) - could be backported but not strictly a bug fix
Those fixes combined with the patch above and scrolling already works a lot better than I expected. Giving smooth lossless scrolling in most cases.
Still TODO:
- fix race conditions: we modify the refresh list from a network thread..
- the list of rectangles could grow too big, and we traverse it twice (
O(N^2)
): give up past a certain size?
- maybe try harder to re-merge contiguous rectangles?
One unintended benefit of this ticket is that we should be able to see bugs in the scroll paint code more easily because those would not get refreshed. So far so good.
Fri, 22 Mar 2019 08:10:35 GMT - Antoine Martin: owner, status changed
- owner
changed from Antoine Martin to Jonathan Anthony
- status
changed from assigned to new
Changes:
- r22205 + r22206: reduce race condition window, try to only add to the refresh list until we consume it
- r22207 more correct pixel accounting when adding regions to the refresh list (side note: the cython rectangle code is much faster with python2.. not sure why)
- r22208 "lossless scrolling", the core changes
It works very well.
Caveats:
- it may still be possible for the server to lose track of some regions that need a refresh, but the window of opportunity for this race condition is very small and unlikely - adding locking to prevent this race condition would be too costly.
- a video may still look weird when scrolling since it runs in parallel, with its own scheduling.
- I trust the html5 client scroll paint code less than the python client (ie: #1637), though it seems to work fine when I tried.
For testing, I had to force the quality lower to ensure that the non-scroll areas would get sent using a lossy encoding and later refreshed correctly (ie: jpeg or webp in lossy mode is what we want).
To make sure that the server doesn't choose rgb for those, I configured my client to not use rgb at all: --encodings=webp,png,jpeg,h264
To be able to see what is being sent to the client:
- use
-d compress
on the server (and add -d refresh,scroll
for even more logging
- use paint boxes on the client: #760 (or the "paint" debug for the html5 client)
I can scroll up and down frantically using a browser as a client application and I never see any visual artifacts. And it's smooth too.
@encodedEntropy: this is a major improvement over the previous version of the scrolling code.
Thu, 01 Aug 2019 12:02:39 GMT - Smo: owner changed
- owner
changed from Jonathan Anthony to Smo
Wed, 07 Aug 2019 14:33:38 GMT - Smo: owner changed
- owner
changed from Smo to Antoine Martin
Tested with server and client 3.0-r23413
server
xpra --bind-tcp=0.0.0.0:14000 --start=xterm --no-daemon -d compress,refresh,scroll start :15
client
XPRA_OPENGL_PAINT_BOX=1 xpra --encodings=webp,png,jpeg,h264 attach tcp:localhost:14000
Also tested with google-chrome and the html5 client.
In both cases the scrolling was working as expected and was extremely smooth.
Very good!
Thu, 08 Aug 2019 05:39:29 GMT - Antoine Martin: status changed; resolution set
- status
changed from new to closed
- resolution
set to fixed
@smo: why override the encodings
?
Not having rgb in that list is going to hurt performance, why bother?
Sat, 23 Jan 2021 05:21:02 GMT - migration script:
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1320