multi window nvfbc

[Antoine Martin]

Split from ticket:1801#comment:5.

It should be possible to use a single nvfbc capture instance for multiple monitors and just adjust the buffer offsets.

[Antoine Martin]

Basic support added in r19137.

Still TODO:

• capture only once for all monitors
• handle suspend + resume without closing capture
• fix segfault when we make the buffer copy narrower (smaller dst_pitch) - waste of GPU RAM, but not a huge problem since we don't copy more than we need
• keep track of cuda buffers we allocate and prevent memory overcommit somehow - re-use buffers?
• when shutting down, free the buffer so we don't try to use it after the cuda context is destroyed
• support multi-window mode in the "sys" backend
• nvenc is not used in some cases? (dual 4k too large for nvenc if dimension is miscalculated: individual windows can be encoded at 4k)

[Antoine Martin]

Updates:

• r19138: capture only once for all monitors
• r19145: use correct image wrapper dimensions: fixes crash when copying also fixes nvenc not being used, keep track of image wrappers to avoid using cuda context after we clean it up
• r19146: narrower buffer copy
• r19147: "sys" backend support

[Antoine Martin]

Detected a memory leak during testing for #1823.
Only occurs with nvfbc CUDA buffers and nvenc.
Downloading to system memory before re-uploading to a CUDA buffer with XPRA_NVENC_GPU_MEMCOPY=0 (introduced in r19151) does not help.
Using a non-nvfbc source for video does not leak either.

r19153 fixes this but I'm not 100% sure why that is. The image wrapper method was being called before this change, just maybe not from the right thread or not with the correct CUDA context.

[Antoine Martin]

• r19160: correct fix for the GPU memory leak
• r19157: don't call damage if nvfbc tells us the pixels have not changed
• r19159: better cuda cleanup

[Antoine Martin]

The shadow pointer shown on the client may be at the wrong location in 2.3, this is fixed in 2.4: r19603.