Xpra: Ticket #202: hardware accelerated video decoding

Following #147, we should go one step further and offload x264 video decoding to the graphics card using libva

Some pointers:

xvba (ati). If really needed we could also talk to those directly, but that is best avoided.

Thu, 01 Nov 2012 03:12:21 GMT - Antoine Martin: status, description changed

Thu, 01 Nov 2012 09:30:44 GMT - ahuillet:

I don't think we should be using VAAPI directly for video decoding acceleration. libav is a pain, but it has a VAAPI backend, and it will probably be better to use it. Rationale: we'll need video decoding acceleration on Windows (possibly even more than on Linux). We'll want to do it through libva as well.

Thu, 01 Nov 2012 09:34:18 GMT - Antoine Martin: description changed

decoding example

Mon, 17 Jun 2013 12:22:29 GMT - Antoine Martin:

For DXVA see #346

Mon, 13 Jul 2015 05:28:27 GMT - rektide: cc set

Sun, 23 Jul 2017 06:06:40 GMT - Antoine Martin:

From the comments section on VA-API Video Acceleration On The Linux Desktop Is Nearly Ready For Chrome:

Wed, 28 Mar 2018 05:16:15 GMT - Antoine Martin: milestone changed

Wed, 20 Mar 2019 05:06:15 GMT - Antoine Martin: milestone changed

Milestone renamed

Wed, 17 Jun 2020 04:54:42 GMT - Antoine Martin: milestone changed

Now that encoding via vaapi works (#451), can we use the same solution here? (no)

The data we feed into the decoder is relatively small (so no problem uploading it to the GPU), but the decoded data is big (full size pixel frames) and so it's a waste if we have to download it to CPU memory only to re-upload to the GPU for rendering on screen. It probably negates most of the savings from decoding on the GPU - the server usually selects fast presets which aren't very taxing for a CPU software decoder. Then there's also the extra decoder complexity (except on macos with h264_vda which is a drop in replacement for h264?) and usual hardware flakiness (lack of hardware for testing, differences in drivers / OS, etc).

Staying on the GPU, as we do encoding side with nvfbc #1317, will require direct use of GPU buffers (dmabuf / pixmaps on Linux, whatever else on other platforms). This is harder, and will need to be done for each supported platform separately. PITA.

This is looking unlikely for 4.1

Sat, 23 Jan 2021 04:48:18 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/202