xpra icon
Bug tracker and wiki

Opened 20 months ago

Last modified 4 weeks ago

#2052 new enhancement

downscaling server side - not just for video

Reported by: Antoine Martin Owned by: stdedos
Priority: major Milestone: 4.1
Component: encodings Version: 2.4.x
Keywords: Cc:

Description

When the client is using a desktop-scaling value lower than 1.0, we end up sending pixels that aren't all used client-side since we end up downscaling the picture to display it. (the render target is smaller than the picture the client receives)

It would be better if the client could tell the server about the scaling factor it is using then the server can:

  • choose this same factor (or more) for video scaling - easy
  • downscale picture data before compression: new capability flag, new client option, image down scaling pre-compression

Change History (8)

comment:1 Changed 20 months ago by Antoine Martin

Status: newassigned
Summary: downscaling attributedownscaling server side - not just for video

comment:2 Changed 16 months ago by Antoine Martin

Milestone: 3.14.0

Milestone renamed

comment:3 Changed 9 months ago by Antoine Martin

See also #2458.

comment:4 Changed 5 months ago by leo60228

I'd argue that a major reason for this feature is shadowing. Currently, there isn't any way to use a shadowed desktop of a higher resolution than the client (at least in HTML5, the only client I use).

comment:5 Changed 5 months ago by Antoine Martin

Milestone: 4.04.1

comment:6 Changed 4 weeks ago by Antoine Martin

Mostly done in:

  • r26652: client tells the server about its "render-size", the resolution it will actually render at - the video scaling code will
  • r26656: downscale pixel data before compressing, and upscale it client-side

Still TODO:

  • maybe support some kind of fast scaling so we can compress webp and jpeg via the native encoders rather than using pillow? (meh: it is likely that the savings from the downscaling via pillow are enough to not have to bother)
  • client side should be able to decode using any decoder and just upscale during the backend pixel paint (cairo / opengl) rather than being forced to use the pillow decoder
  • drop delta compression? (not very useful and makes the code a lot more complicated)

... at least in HTML5, the only client I use ...

@leo60228: there is no support for any window scaling in the HTML5 client, feel free to create a new ticket for that.

Last edited 4 weeks ago by Antoine Martin (previous) (diff)

comment:7 Changed 4 weeks ago by Antoine Martin

Owner: changed from Antoine Martin to stdedos
Status: assignednew
  • r26663: remove delta support (+r26665 + r26666)
  • r26664: honour gravity adjustments when resizing with opengl backend (could be backported with extra care: v4 does gravity adjust in paint_rgb, trunk now does it in do_paint_rgb and gl_paint_planar)
  • r26668: handle scaling during the backing paint rather than the decompression stage, so all the decoders can use it and not just pillow

Notes:

  • webp can decompress to a scaled image - not sure there are any benefits to doing that
  • in non-opengl video mode, we could avoid scaling during the csc stage, and let the cairo paint handle it - meh

@stdedos: I am re-assigning this ticket to you because you will see huge improvements in bandwidth usage and performance when using desktop-scaling values lower than 1.0.
Feel free to just close.
(this requires 4.1-r26668 or later on both client and server)

comment:8 Changed 4 weeks ago by stdedos

I can keep it open, and I cannot test it (my OS update plan for the server is late June to August)

Note: See TracTickets for help on using tickets.