Xpra: Ticket #1299: low fps when vigorously mousing over the video

Split from ticket:1290#comment:2: when mousing over a video region (google-chrome, youtube in this case) the framerate seems to drop from 25 down to 8. We want to know why that is. Is it the browser, X11 or maybe our code that is struggling with the repaints?



Tue, 13 Sep 2016 14:49:23 GMT - Antoine Martin: owner changed

r13689 exposes the "damage.fps", which is the number of damage events we receive from the X11 server. So now we can see

ie:

xpra info | egrep "damage.fps|encoder.fps|subregion.fps"                                            Tue Sep 13 21:35:23 2016
window.1.damage.fps=0
window.2.damage.fps=29
window.2.encoder.fps=30
window.2.video_subregion.fps=29

The youtube video I used for testing must have been pushing 29fps. The good news is that the bottleneck is in xpra only: the framerate does drop if you move the mouse frantically over the page, we end up losing the video region and sending jpeg and png updates, which quickly use too much CPU. (this is made worse because when we have / had a video region, we increase the quality of the "other" updates, often using png... which is a CPU pig) As soon as I stop moving the mouse, we pick up the video region again reasonably quickly (if not, add to #967) and things get back to normal.

I am tempted to close this ticket as "works well enough, don't do crazy things". @afarr: thoughts?


Wed, 28 Sep 2016 00:47:55 GMT - alas: owner changed

I would love to encourage that... but I've actually heard people arguing that this behavior isn't "crazy".

I'm not sure if I'd go so far as to agree that it isn't, but if we could find some way of keeping the framerate above, say, 15 fps, no matter what weird behavior someone wants to define as "not crazy"... that might be a happier medium?

That said, I would suggest pushing this to the next release, and hoping for some inspiration to fall from the heavens in the meantime.

I'll hand this back to you to decide about the milestone/closure details, and assume you'll pass it back afterward (or not).


Wed, 28 Sep 2016 01:09:18 GMT - Antoine Martin: status, milestone changed

I'm not sure we'll be able to reach 15fps but we can try.


Sat, 03 Feb 2018 05:51:29 GMT - Antoine Martin: owner, status, milestone changed

r18176 may help with this (disabled by default in r18178) as we can now rate limit the pointer position updates with:

XPRA_MOUSE_DELAY=20 xpra attach ...

This will limit the updates to (1000/20=50 fps), higher values may be needed if the connection can't take the bandwidth spikes caused by the sudden vigorous movement. If that does help, we could implement this more generically in the server, and maybe even tie this with bandwidth management. (#999)


Thu, 15 Feb 2018 22:32:25 GMT - J. Max Mena:

Was finally given some time to take a look at this one since Alex is swamped at the moment.

Running a Fedora 26 trunk r18444 server and client built from source:

If I spastically mouse around it drops to <10FPS - I've seen it drop to 8 at the lowest.

When attaching with XPRA_MOUSE_DELAY=20 it'll drop down to ~10-12FPS. If I set XPRA_MOUSE_DELAY=40 then it holds much steadier around 13-17FPS.


Thu, 08 Mar 2018 18:02:04 GMT - Antoine Martin: owner changed

As of r18666, we will set the mouse delay using the client's display vertical refresh rate (as seen in the "Native_GUI" info tools on win32 and macos, xpra/platform/gui.py). This behaviour can be disabled with XPRA_MOUSE_DELAY_AUTO=0. This changeset also adds better debug logging with "-d mouse".

This shouldn't interfere too much, I hope. If it does, I'll turn it off again.

With a standard GPU and display (60 Hz), this should give a ~16ms delay, which looks like it would correspond to ~10fps maximum with the spastic mouse action. Not much of an improvement, but setting it higher makes it possible for us to "skip" valid frames: we should be able to handle a screen update for each vertical refresh, and each of these updates could match a new pointer position.

There may be other areas we can improve so that the pointer movements don't cause the fps to drop, in particular I am looking at #1700. Maybe, we're switching to non-video because of the other damaged areas? Or just a sub-optimal encoding. r18669 tries to do better there.

I've seen a few things that warrant further investigation (ie: the video region's screen updates are too bursty). Please attach the "-d damage,compress" log of just a 4 or 5 seconds showing the "spastic" movement with low fps, and include the "fps" info (xpra info | egrep "fps|batch.delay"). We want to know if it is the encoder fps that is low or both the damage fps and encoder / video subregion fps? Is it really visually slow to refresh? ("scroll" encoding can lower the fps count, whilst still updating the screen at a high fps) Make sure to disable bandwidth detection so it won't interfere with this. And try to turn off auto-refresh to see if that helps. It is also worth checking that the video region is still detected correctly during the spastic episode, though there probably isn't much we can do to fix it if it isn't, as the extra damage events are "legitimate".


Fri, 09 Mar 2018 23:11:44 GMT - J. Max Mena:

I've spent about 30 minutes looking at this one and I keep reliably running into the issue I mentioned in #1781 - which I am very sure is going to heavily impact any testing I do here. I can post what I have, but I honestly feel that it isn't indicative of what's really going on.


Sat, 10 Mar 2018 05:52:50 GMT - Antoine Martin:

See ticket:1781#comment:4, you can avoid refresh related problems by turning off the auto-refresh.


Tue, 03 Apr 2018 18:08:28 GMT - J. Max Mena: owner changed

EDIT: Using a trunk r18986 Fedora 26 client and server

Okay I did the spastic mousing as you asked - with -d damage,compress running. I also made sure to disable the bandwidth limitations. I'll attach the log shortly.


Tue, 03 Apr 2018 18:09:03 GMT - J. Max Mena: attachment set


Wed, 04 Apr 2018 04:45:54 GMT - Antoine Martin: owner changed

The log spans 1.5 minutes and ~10000 lines, can you please specify when the problem occurred?

The framerate looks mostly steady at ~20 to 30 fps. Except on two occasions where it drops to below 10 fps:

2018-04-03 11:05:23,040 damage(5, 141, 1104, 59, {})   wid=2, scheduling batching expiry for sequence 1587 in 132.0 ms
2018-04-03 11:05:14,439 damage(5, 671, 1104, 59, {})   wid=2, scheduling batching expiry for sequence 1167 in 156.0 ms

But that's only for a few frames. Does this match what you saw? Judging by the compression options, that session is running at high speed (>=90%) and high quality (~99%) when this happens.

There is also an instance right at the start (~15fps), but this one looks more like a conservative decision to ramp up the framerate during the first few frames to prevent overwhelming low bandwidth links, so I don't think this one is a problem.

To figure out why the batch delay jumps so high (from ~30ms to 150ms), please add "-d stats" to the debug output. (please make sure to record when the framerate drops if you attach the full log file - which is going to be big)

Minor related rgb compression tuning improvements in r18988, r18990.


Thu, 05 Apr 2018 19:37:43 GMT - J. Max Mena:

I tried it again today after upping my server and client to r18990 - I got a couple framerate hiccups yesterday which you noted in the log - but I'm not seeing it today. I'll play around with it some more and see if it comes back.


Fri, 13 Apr 2018 17:34:56 GMT - J. Max Mena: status changed; resolution set

Meant to close this earlier, but I've been fairly busy.

For at least the last week, I haven't been able to reproduce the low FPS. It seems to be holding steady around 20FPS, at least according to xpra info | grep fps.

Anyways, closing.


Sat, 23 Jan 2021 05:20:28 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1299