Xpra: Ticket #2792: xpra shadow server "not working"

---

Mon, 01 Jun 2020 11:06:48 GMT - stdedos: attachment set

• attachment set to redact-display-_0-$TIMESTAMP.log

---

Mon, 01 Jun 2020 14:49:40 GMT - Antoine Martin: owner changed

• owner changed from Antoine Martin to stdedos

I assume this is a regression? A client regression? Does it work with a Linux client? 4.0 client? Can you post the client log?

---

Mon, 01 Jun 2020 18:17:54 GMT - stdedos:

Replying to Antoine Martin:

I assume this is a regression?

Yes

A client regression?

Idk, that is my guess

Does it work with a Linux client? 4.0 client?

This makes no sense to my workflow, and I don't have a system to test with that configuration. I also have been having #2617 - this could be the most recent/severe version of it (I also have used different client versions, so I decided to post a new bug, just to be sure).

Can you post the client log?

Equivalent to

"Xpra-Python3-x86_64_4.1-r26503\xpra_cmd" attach ssh://user@ip/200 --ssh="plink -ssh -agent" --modal-windows=no --title="@title@ on @@/@server-display@" --opengl=no --bandwidth-limit=6Mbps
2020-06-01 09:44:34,754 Xpra GTK3 client version 4.1-r26503 64-bit
2020-06-01 09:44:34,756  running on Microsoft Windows 10
2020-06-01 09:44:34,825 Warning: failed to import opencv:
2020-06-01 09:44:34,825  No module named 'cv2'
2020-06-01 09:44:34,826  webcam forwarding is disabled
2020-06-01 09:44:35,422 GStreamer version 1.16.2 for Python 3.8.3 64-bit
2020-06-01 09:44:35,789 keyboard layout code 0x409
2020-06-01 09:44:35,789 identified as 'United States - English' : us
2020-06-01 09:44:35,956  keyboard settings: layout=us
2020-06-01 09:44:35,959  desktop size is 4160x1440 with 1 screen:
2020-06-01 09:44:35,960   Default (1100x381 mm - DPI: 96x96) workarea: 4160x1400
2020-06-01 09:44:35,960     Generic PnP Monitor 1600x900 at 0x534 (309x174 mm - DPI: 131x131) workarea: 1600x860
2020-06-01 09:44:35,960     C32JG5x 2560x1440 at 1600x0 (697x392 mm - DPI: 93x93) workarea: 2560x1400
2020-06-01 09:44:57,331 enabled remote logging
2020-06-01 09:44:57,333 Xpra GTK3 X11 server version 3.0.9-r26132 64-bit
2020-06-01 09:44:57,335  running on Linux Ubuntu 16.04 xenial

With these flags: shadow ssh://user@ip/0 --ssh="plink -ssh -agent" --opengl=no --desktop-scaling=0.75 --min-speed=70 --webcam=no --speaker=off --microphone=off --pulseaudio=no --exit-with-client=no

---

Sun, 05 Jul 2020 14:39:32 GMT - Antoine Martin:

It's not clear to me if the bug is server-side or client side. Did this go away when you downgraded to 4.0.x?

---

Sun, 05 Jul 2020 14:57:36 GMT - stdedos:

Now that I think of it, actually yeah. I managed to use 2/3 shadow monitors with ease (given the latency ofc)

---

Tue, 06 Oct 2020 13:05:51 GMT - Antoine Martin:

I am strongly suspecting bandwidth issues. 6Mbps is low. You have 3 monitors (at least 1080p each), that's roughly 20MB worth of pixels to send. Uncompressed, that would be over 160Mbps of bandwidth per screen update. Compression helps, but doesn't do miracles.

Now that I think of it, actually yeah. I managed to use 2/3 shadow monitors with ease (given the latency ofc)

So this is a 4.1 regression? Do you have any idea which revision broke?

---

Tue, 06 Oct 2020 17:27:45 GMT - stdedos:

Replying to Antoine Martin:

I am strongly suspecting bandwidth issues. 6Mbps is low.

I know, but it used to work. Also, I have had much more bw, but something-something and you asked me to use this flag and it probably kind-of helped - so it's there now.

You have 3 monitors (at least 1080p each), that's roughly 20MB worth of pixels to send. Uncompressed, that would be over 160Mbps of bandwidth per screen update.

Is this "per screen update" === per frame? per second? is it per min(max(1/25,update-dt),1) sec? While on theory that is the original size, I don't think that this is remotely to the actual size. Otherwise, if xpra was really throttling <6mbps, that means less than 1 FPS for all of the monitors combined (!), and I don't see that

Compression helps, but doesn't do miracles.

I think we have discussed numerous optimizations that exist on the background that would help. And I'll keep asking you: Is there a way to expose all of that?

Something like: | Window Name | Quality | Speed | FPS | MPixels/MB | BW (calculated or actual)

Now that I think of it, actually yeah. I managed to use 2/3 shadow monitors with ease (given the latency ofc)

So this is a 4.1 regression? Do you have any idea which revision broke?

At this time, I honestly don't remember - and I have been adjusted to keep avoiding shadow server unless necessary. I would say, at worst, it would be between r25898 and whatever was the latest released beta 2 months from that day - but that is already too old, you've made a lot of changes even since r25898.

However, if it's any consolation, I have the same "issue"/meta-stable condition that I can use 2/3 monitors (2x1080p) with low FPS, and significant delay. I'll try to remember to post tomorrow what is the XPRA_ env set.

---

Wed, 07 Oct 2020 07:17:11 GMT - Antoine Martin:

Uncompressed, that would be over 160Mbps of bandwidth per screen update.

Is this "per screen update" === per frame? per second? is it per min(max(1/25,update-dt),1) sec?

That's how much bandwidth it would take to send one full screen update. Every pixel of every window takes 3 bytes. Multiply that by the number of pixels. And you have to send all of that at least once, first when you show the windows for the first time.

While on theory that is the original size, I don't think that this is remotely to the actual size.

Like I said, this is the actual uncompressed size. Depending on the content, it may compress well or not. For lossless, you can expect a reduction by a factor of 10. Which is still way more bandwidth than you have.

And I'll keep asking you: Is there a way to expose all of that?

Not all. But xpra info has a lot.

| Window Name | Quality | Speed | FPS | MPixels/MB | BW (calculated or actual)

We don't have per-window bandwidth ATM, and FPS is mostly meaningless unless the window is showing a video. This data is pretty close to what is shown on the "statistics" tab of the session info window. Adding per-window details might be worth doing, but I fear that users may then get more confused than before..

---

Wed, 07 Oct 2020 08:18:17 GMT - stdedos:

I'll try to remember to post tomorrow what is the XPRA_ env set.

Client runs:

XPRA_CUSTOM_TITLE_BAR=0
XPRA_EXECUTABLE=Xpra-Python3-x86_64_4.1-r27616
XPRA_SCROLL_ENCODING=0

And server was started with:

env.XPRA_LOG_DIR=/run/user/1000/xpra
env.XPRA_PROXY_START_UUID=43ed51de567549d3a76f339458ee2614
env.XPRA_PULSE_SERVER=/run/user/1000/xpra/pulse-3/pulse/native
env.XPRA_PULSE_SINK_DEVICE_NAME=Xpra-Microphone
env.XPRA_PULSE_SOURCE_DEVICE_NAME=Xpra-Speaker
env.XPRA_SERVER_SOCKET=/run/user/1000/xpra/usr-linux-3
env.XPRA_SHADOW_REFRESH_DELAY=200

---

I don't know how to interpret between these two: (example ofc)

1. The latency is obviously different, but I don't think it represents how the interaction feels to the user (there are moments where one click felt like forever, or it felt like video had hiccups; however "average" is just that - average). Then, there are so many latencies - which obviously mean something to you, but less to me. This is why I asked for FPS. Not the local window drawing FPS, but really how many screen updates in a second as a running average (e.g. of 10 seconds). I want a singular metric that responds to "if I move my mouse and click stuff, how close is what I am doing to what has landed on the server?" (then, if you feel like breaking it down, do it - like you do on the graphs page).

2. The quality/speed metrics are reversed; however they still don't make sense to me. Quality was "around the same" vivid colors etc (from the dumb user's perspective, again), but now I was able to interact okay-ish with all 3 monitors, and have it pretty responsive too.

As I said, again, I would be all for dropping to 8bit/16bit colors (I don't know how to describe quality) to get a close to 20FPS-equivalent refresh rate all day, but XPRA has had refused to "automatically" do that

However, for a change, I do have now a 100/10 mbps line. Maybe this

However, if it's any consolation, I have the same "issue"/meta-stable condition that I can use 2/3 monitors (2x1080p) with low FPS, and significant delay.

was remaining memories from before this month.

---

Wed, 07 Oct 2020 08:18:45 GMT - stdedos: attachment set

• attachment set to Xpra_cmd_2020-10-07_10-40-36.png

---

Wed, 07 Oct 2020 08:48:37 GMT - Antoine Martin:

I don't know how to interpret between these two:

Any time your client or server latency goes above 1 second, you're lucky if the connection doesn't drop at that point.

Then, there are so many latencies - which obviously mean something to you, but less to me.

If you use your pointer to hover over the latency text, a description of what it means shows up as a tooltip.

This is why I asked for FPS but really how many screen updates in a second as a running average

Unless you're watching a video or using shadow mode, this number means absolutely nothing and it is the most misleading figure there is, so I am very reluctant to show it anywhere.

The quality/speed metrics are reversed

Reversed? Please explain. Speed is how long we spend encoding a frame, in your case, low speed would be preferable since this will use less bandwidth and may actually arrive sooner.

I would be all for dropping to 8bit/16bit colors .. to get a close to 20FPS-equivalent refresh rate all day .. but XPRA has had refused to "automatically" do that

That does not help you achieve that. Drop the quality level and the picture codecs will do a far better job at deciding how to reduce the bitrate than dropping bits before handing over the pixels. In any case, most encoders don't take 8bit or 16bit input - only png/L and png/P handle 8bit input. Xpra 4.1 has an 8bit grayscale mode though: #1713 (with hacks to feed this into more encoders - including video encoders)

If you really want to force xpra to use less than 24-bit colors, you can use --pixel-depth=8 (or 16), but only with start and start-desktop modes.

However, for a change, I do have now a 100/10 mbps line

Likely to make a massive difference.

---

Wed, 07 Oct 2020 09:04:21 GMT - stdedos:

Replying to Antoine Martin:

I don't know how to interpret between these two:

Any time your client or server latency goes above 1 second, you're lucky if the connection doesn't drop at that point.

Then, there are so many latencies - which obviously mean something to you, but less to me.

If you use your pointer to hover over the latency text, a description of what it means shows up as a tooltip.

This is why I asked for FPS but really how many screen updates in a second as a running average

Unless you're watching a video or using shadow mode, this number means absolutely nothing and it is the most misleading figure there is, so I am very reluctant to show it anywhere.

(First of all, the title of this ticket is "shadow"), and I am still missing "the one" metric that tells me what does XPRA think my User Experience feels like (I can think of "interactivity", "quality"). It doesn't have to be "one" metric, but I expect one clear definition and one clear response (equivalent to what does "ping rtt" mean for the network).

The quality/speed metrics are reversed

Reversed? Please explain.

Look at the Statistics pictures I added above. First : Quality 56 / Speed 73 (average) Second: Quality 75 / Speed 59 (average)

Speed is how long we spend encoding a frame, in your case, low speed would be preferable since this will use less bandwidth and may actually arrive sooner.

I thought it was related to the image/video latency I see on my side.

I would be all for dropping to 8bit/16bit colors .. to get a close to 20FPS-equivalent refresh rate all day .. but XPRA has had refused to "automatically" do that

That does not help you achieve that. Drop the quality level and the picture codecs will do a far better job at deciding how to reduce the bitrate than dropping bits before handing over the pixels. In any case, most encoders don't take 8bit or 16bit input - only png/L and png/P handle 8bit input. Xpra 4.1 has an 8bit grayscale mode though: #1713 (with hacks to feed this into more encoders - including video encoders)

If you really want to force xpra to use less than 24-bit colors, you can use --pixel-depth=8 (or 16), but only with start and start-desktop modes.

I really want shadow mode to work 😛. Up until the point where the image appears as a Pixelate 2x2 filter, with "colors that are almost fully de-saturated" (or whatever else equivalent the codecs decide). A different example of before/after could be this:

As I am not proficient in image/video encoding (nor English for that matter), I am not sure how to accurately describe "low quality, but you can still see shapes and kind-of read text".

However, for a change, I do have now a 100/10 mbps line

Likely to make a massive difference.

It looks like it, but still a bit reluctant to switch over. (However, Ubuntu 20.04.1 was released for upgrade-install - so update of the server is on the pipeline).

---

Wed, 07 Oct 2020 09:04:40 GMT - stdedos: attachment set

• attachment set to Screenshot-chrome_2020-10-07_11-54-19.png

---

Wed, 07 Oct 2020 09:12:26 GMT - Antoine Martin:

Look at the Statistics pictures I added above. First : Quality 56 / Speed 73 (average) Second: Quality 75 / Speed 59 (average)

Right, you can only have high quality and high speed at the same time if you have ample bandwidth to handle the extra bits. Xpra chose to sacrifice the quality more in one case.

(speed) I thought it was related to the image/video latency I see on my side.

Sort of. If you have bandwidth to spare, compressing less is quicker. But if you don't, then the packet will take longer to arrive.

I am not sure how to accurately describe "low quality, but you can still see shapes and kind-of read text".

Reducing the saturation is similar to what YUV colorspace does with chroma subsampling, and this is used by video codecs and jpeg. xpra does this automatically when you lower the quality, but without losing too much of the colours, just reducing how many bits are used to encode them.

---

Wed, 07 Oct 2020 09:23:33 GMT - stdedos:

Replying to Antoine Martin:

(speed) I thought it was related to the image/video latency I see on my side.

Sort of. If you have bandwidth to spare, compressing less is quicker. But if you don't, then the packet will take longer to arrive.

That is why I get so fixed on the "FPS metric". I clearly understand what is the difference between 15 and 25 FPS in an first-person shooter game (I could compare working with a computer to an FPS game, since you do something, expect to see a result, and you re-align what you are doing based on that feedback - especially e.g. for mouse-driven interactions).

Of course, I think at this time you understand what I am trying to convey - so let's drop this thread.

Additionally, let's drop this ticket too. Having seen a very interactive session, and having ample BW available, I guess this is unlikely to happen in normal conditions (I still have network issues, but I have identified them to be an ISP/ISP's modem to blame)

---

Wed, 07 Oct 2020 09:35:28 GMT - Antoine Martin: status changed; resolution set

• status changed from new to closed
• resolution set to worksforme

Let's follow up in #2894

---

Sat, 23 Jan 2021 06:00:59 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/2792