xpra icon
Bug tracker and wiki

Opened 3 years ago

Last modified 6 months ago

#981 assigned enhancement

sync paint updates to the video

Reported by: Antoine Martin Owned by: Antoine Martin
Priority: major Milestone: 3.1
Component: server Version: 0.15.x
Keywords: Cc:

Description

Follow up from #792.

Painting all the updates together helps, allows us to reach a higher framerate, but we should not have to slow down displaying the video region just because there are other paint events coming through.

We should send those with a higher flush value and rely on the video paint to flush everything.

Attachments (5)

sync-paint-with-video.patch (7.8 KB) - added by Antoine Martin 3 years ago.
attempt at implementing this
opengl-show-wait-time.patch (1.6 KB) - added by Antoine Martin 3 years ago.
shows how long it takes to call swap_buffers
sync-paint-with-video-v2.patch (5.5 KB) - added by Antoine Martin 2 years ago.
updated patch
sync-paint-with-video-v5.patch (6.4 KB) - added by Antoine Martin 2 years ago.
patch update to r12780
sync-paint-with-video-v6.patch (6.8 KB) - added by Antoine Martin 2 years ago.
adds the ability to toggle flush using an env var

Download all attachments as: .zip

Change History (16)

Changed 3 years ago by Antoine Martin

Attachment: sync-paint-with-video.patch added

attempt at implementing this

comment:1 Changed 3 years ago by Antoine Martin

Status: newassigned

The patch above attempts to implement this feature, but doesn't seem to be making things better... maybe worse if anything!?

comment:2 Changed 3 years ago by Antoine Martin

Maybe we should set the minimum video batch delay to match the vrefresh rate in this case?
Most 4k monitors are 60Hz (some are just 30Hz), which means we should be batching for at least 16ms (rounded down to take processing time into account?) whereas the default "min-delay" is currently set at 5ms.

Changed 3 years ago by Antoine Martin

Attachment: opengl-show-wait-time.patch added

shows how long it takes to call swap_buffers

comment:3 Changed 3 years ago by Antoine Martin

r10879 added an app which can easily generate 100fps: browser/xpra/trunk/src/tests/xpra/test_apps/fps.py.
r10880 also modified the existing browser/xpra/trunk/src/tests/xpra/test_apps/test_videoregions.py test app to hit 100fps.

The good news is that the server code quickly finds the best refresh rate (60Hz on my system).
The difficulty is to find a good test case, and verify that we are making things better. The patch above allows us to see how long we have to wait for the vblank.

With the "fps" example, the time is usually in the 12 to 15ms range, with only the occasional value below 10ms, very rarely below 5ms. We almost never miss a vblank.
That's because it is quick to draw and so we spend most of the time waiting for the vblank.
With the "videoregions" example however, it's a lot more varied (it does behave a bit more like a real application...) - but I am not convinced that this is necessarily a much better baseline.

comment:4 Changed 3 years ago by Antoine Martin

According to this: NVidia OpenGL env variables, __GL_SYNC_TO_VBLANK can be used to enable/disable swapbuffers syncing to the vblank.

See also #386

Last edited 3 years ago by Antoine Martin (previous) (diff)

comment:5 Changed 3 years ago by Antoine Martin

Milestone: 0.160.17

comment:6 Changed 3 years ago by Antoine Martin

Milestone: 0.170.18

Changed 2 years ago by Antoine Martin

updated patch

comment:7 Changed 2 years ago by Antoine Martin

Following the work on #1218, here's an updated patch and a better understanding of the problems with it: r12778 shows how many rectangles we paint when we swap the opengl buffers, and whenever we paint video with other screen updates there is a long delay preceding it because it all takes too long to process.
Fixing this is not easy. Here are some ideas:

  • we could process the video region without delay, then process the rest as a delayed flush - same problem as b-frames (#800): we have no way of knowing if another video frame will actually come within a reasonable amount of time, so we need a timer to send the flush message.
  • delay the non-video regions less: this way, fewer of them will accumulate and we are less likely to miss the vblank. Problem here is that we risk a recurrence of #1218.
  • somehow prioritize video frames. Tricky since the queue has no concept of priority, and the result is likely to lead to a corrupted window since the updates are meant to be painted in order..
  • use faster encodings (ie: lz4) when we have a video region.
  • take the fps into account: both the video region fps, and the client's monitor refresh rate.
  • group all region paints into one packet to save CPU time
Last edited 2 years ago by Antoine Martin (previous) (diff)

Changed 2 years ago by Antoine Martin

patch update to r12780

Changed 2 years ago by Antoine Martin

adds the ability to toggle flush using an env var

comment:8 Changed 2 years ago by Antoine Martin

r12778 + r12779 make it easier to see the timing of screen updates.

With the latest patch above, I find that screen updates are A LOT smoother without the sync flush code.. which is the opposite of what would be expected.

Note: r12780 sends the non-video updates in the same call to send_delayed_regions if they're close to being due, which helps a bit.

Maybe this needs to be tested with a higher fps video, or a low fps monitor? (or both)

comment:9 Changed 2 years ago by Antoine Martin

Milestone: 0.181.0

Milestone renamed

comment:10 Changed 2 years ago by Antoine Martin

Milestone: 1.03.0

Re-scheduling, too risky at this point.

comment:11 Changed 6 months ago by Antoine Martin

Milestone: 3.03.1
Note: See TracTickets for help on using tickets.