Xpra: Ticket #2052: downscaling server side - not just for video
When the client is using a desktop-scaling
value lower than 1.0, we end up sending pixels that aren't all used client-side since we end up downscaling the picture to display it. (the render target is smaller than the picture the client receives)
It would be better if the client could tell the server about the scaling factor it is using then the server can:
- choose this same factor (or more) for video scaling - easy
- downscale picture data before compression: new capability flag, new client option, image down scaling pre-compression
Wed, 21 Nov 2018 10:18:01 GMT - Antoine Martin: status, summary changed
- status
changed from new to assigned
- summary
changed from downscaling attribute to downscaling server side - not just for video
Wed, 20 Mar 2019 05:06:15 GMT - Antoine Martin: milestone changed
- milestone
changed from 3.1 to 4.0
Milestone renamed
Sun, 20 Oct 2019 15:28:00 GMT - Antoine Martin:
See also #2458.
Thu, 30 Jan 2020 14:08:28 GMT - leo60228:
I'd argue that a major reason for this feature is shadowing. Currently, there isn't any way to use a shadowed desktop of a higher resolution than the client (at least in HTML5, the only client I use).
Sat, 15 Feb 2020 12:29:57 GMT - Antoine Martin: milestone changed
- milestone
changed from 4.0 to 4.1
Tue, 09 Jun 2020 16:41:09 GMT - Antoine Martin:
Mostly done in:
- r26652: client tells the server about its "render-size", the resolution it will actually render at - the video scaling code will
- r26656: downscale pixel data before compressing, and upscale it client-side
Still TODO:
- maybe support some kind of fast scaling so we can compress
webp
and jpeg
via the native encoders rather than using pillow? (meh: it is likely that the savings from the downscaling via pillow are enough to not have to bother)
- client side should be able to decode using any decoder and just upscale during the backend pixel paint (cairo / opengl) rather than being forced to use the pillow decoder
- drop delta compression? (not very useful and makes the code a lot more complicated)
... at least in HTML5, the only client I use ...
@leo60228: there is no support for any window scaling in the HTML5 client, feel free to create a new ticket for that.
Wed, 10 Jun 2020 09:15:55 GMT - Antoine Martin: owner, status changed
- owner
changed from Antoine Martin to stdedos
- status
changed from assigned to new
- r26663: remove delta support (+r26665 + r26666)
- r26664: honour gravity adjustments when resizing with opengl backend (could be backported with extra care: v4 does gravity adjust in
paint_rgb
, trunk now does it in do_paint_rgb
and gl_paint_planar
)
- r26668: handle scaling during the backing paint rather than the decompression stage, so all the decoders can use it and not just pillow
Notes:
- webp can decompress to a scaled image - not sure there are any benefits to doing that
- in non-opengl video mode, we could avoid scaling during the csc stage, and let the cairo paint handle it - meh
@stdedos: I am re-assigning this ticket to you because you will see huge improvements in bandwidth usage and performance when using desktop-scaling
values lower than 1.0
.
Feel free to just close.
(this requires 4.1-r26668 or later on both client and server)
Wed, 10 Jun 2020 10:37:34 GMT - stdedos:
I can keep it open, and I cannot test it (my OS update plan for the server is late June to August)
Fri, 31 Jul 2020 15:45:00 GMT - Antoine Martin: owner, status changed
- owner
changed from stdedos to Antoine Martin
- status
changed from new to assigned
This has caused a regression for normal windows when we resize them up: the render-size
does not get updated server side, and we end up downscaling when we don't need to..
Fri, 31 Jul 2020 16:13:09 GMT - Antoine Martin: status changed; resolution set
- status
changed from assigned to closed
- resolution
set to fixed
... the render-size does not get updated server side ..
Fixed in r27044.
Side effect: we now call update_encoding_selection
from set_client_properties
, even when the render-size is identical to the window size, which is a little bit of a waste.
stdedos: I'm going to close this ticket for 4.1 - try it when you get a chance.
Wed, 23 Sep 2020 04:46:06 GMT - Antoine Martin:
This causes problems with windows that don't fit the sizing constraints exactly: we end up downscaling them to the client render size (which is often only a few pixels off) and that makes it blurry. Also makes us use pillow (png) instead of the faster options (webp, jpeg).
r27525 fixes that using a threshold.
Sat, 23 Jan 2021 05:40:54 GMT - migration script:
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/2052