Xpra: Ticket #1134: fix websockify bottleneck

Apparently, websockify can become the bottleneck with the HTML5 client when stressed with packet storms (ie: lots of screen refreshes as happens when you scroll a window for example).

I will attach some patches to websockify which try to fix the most glaring issue I spotted: our use case may be slightly unusal in that we transfer some fairly large frames, and when dealing with those the existing packet handling code makes unnecessary copies.

There may well be other issues that need fixing / tuning. I'm not even sure this will make enough of a difference.. but let's try.

Wed, 24 Feb 2016 04:03:36 GMT - Antoine Martin: attachment set

first version: just never join the headers and packet data

Wed, 24 Feb 2016 04:04:13 GMT - Antoine Martin: attachment set

split the header formatting into its own function

Wed, 24 Feb 2016 04:05:05 GMT - Antoine Martin: attachment set

define a convenience send method that we can just re-use everywhere

Wed, 24 Feb 2016 04:05:34 GMT - Antoine Martin: attachment set

do merge the header and the packet data if the size is sufficiently small (4KB)

Wed, 24 Feb 2016 04:31:13 GMT - Antoine Martin: status changed

I am not convinced that this will fix the problem but maybe it will help a bit. Even if it did help a lot, maybe there's still another bug somewhere: surges should not cause the network traffic to stall, even if the code spends too much time aggregating header and data... It's also worth double-checking that the bottleneck really is in websockify and not somewhere else, the other candidates are: xpra's fdproxy (which now has a configurable buffer size, set to 64KB), or even the browser's websocket code. wiki/Profiling may help with that.

Looking at the rest of the websockify code:

Fri, 17 Jun 2016 02:55:54 GMT - Antoine Martin: milestone changed

Things are a lot better now that websockify runs in-process (#1136), but we should still verify that the patches above are worth carrying, or not.

The way forward:

Other related tickets:

Thu, 07 Jul 2016 07:58:19 GMT - Antoine Martin: priority changed

This ticket may have been made redundant by #1136, would be good to evaluate the benefits of the patches above still.

Tue, 12 Jul 2016 16:52:22 GMT - Antoine Martin: milestone changed

Milestone renamed

Sat, 06 Aug 2016 15:44:49 GMT - Antoine Martin:

The nojoin patch has been submitted upstream: https://github.com/kanaka/websockify/pull/244

Fri, 18 Nov 2016 12:01:37 GMT - Antoine Martin: milestone changed

Don't have time for this right now.

Thu, 16 Feb 2017 06:25:52 GMT - Antoine Martin: milestone changed

Mon, 22 May 2017 15:47:47 GMT - Antoine Martin:

Once the patches are merged, we can revert r15924

Sat, 01 Jul 2017 06:33:44 GMT - Antoine Martin: milestone changed

(re-scheduling - websockify upstream is still making changes)

Wed, 25 Oct 2017 10:09:33 GMT - Antoine Martin: milestone changed

still no new upstream release and they've closed the ticket, re-scheduling

Wed, 01 Aug 2018 17:58:04 GMT - Antoine Martin:

See also #1926.

Wed, 20 Mar 2019 05:00:18 GMT - Antoine Martin: status changed; resolution set

We're no longer using websockify: #2121

Sat, 23 Jan 2021 05:15:56 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1134