Xpra: Ticket #1550: NVENC7 active but not used?

I did everything like described on the website to make NVENC7 work. (NVENC8 seems to be not supported yet) Everything looks like NVENC is up and running, but it seems hardware acceleration is not done. (CPU is at 100% on server, framerate on client is at about 10fps).

Maybe i need license keys? But would i get a notice on xpra start about this?

Versions Client + Server:

xpra v2.1 (svn)
Linux game 4.11.4-1-ARCH #1 SMP PREEMPT Fri Jun 9 07:46:48 CEST 2017 x86_64 GNU/Linux
Geforce GTX 970

Cmdline Server:

xpra start :100 --auth=none --daemon=no --video-encoders=nvenc

Cmdline Client:

xpra attach ssh:game:100

Diags Server:

video.encoding.video-encoder.ffmpeg=disabled
video.encoding.video-encoder.nvenc=active
video.encoding.video-encoder.vpx=disabled
video.encoding.video-encoder.x264=disabled
video.encoding.video-encoder.x265=disabled
2017-06-15 17:50:23,872 NVidia driver version 381.22
2017-06-15 17:50:23,872 NVENC license keys:
2017-06-15 17:50:23,880 * version common: 0 key(s)
2017-06-15 17:50:23,880 * version 7: 0 key(s)
Jun 15 17:54:51 game xpra[14208]: X.Org X Server 1.19.3
Jun 15 17:54:51 game xpra[14208]: Release Date: 2017-03-15
Jun 15 17:54:51 game xpra[14208]: X Protocol Version 11, Revision 0
Jun 15 17:54:51 game xpra[14208]: Build Operating System: Linux 4.9.11-1-ARCH x86_64
Jun 15 17:54:51 game xpra[14208]: Current Operating System: Linux game 4.11.4-1-ARCH #1 SMP PREEMPT Fri Jun 9 07:46:48 CEST 2017 x86_64
Jun 15 17:54:51 game xpra[14208]: Kernel command line: initrd=\kernel\arch\initramfs-linux.img root=LABEL=arch rw intel_iommu=on loglevel=3 modprobe.blacklist=nouveau
Jun 15 17:54:51 game xpra[14208]: Build Date: 07 April 2017  05:42:48PM
Jun 15 17:54:51 game xpra[14208]:
Jun 15 17:54:51 game xpra[14208]: Current version of pixman: 0.34.0
Jun 15 17:54:51 game xpra[14208]:         Before reporting problems, check http://wiki.x.org
Jun 15 17:54:51 game xpra[14208]:         to make sure that you have the latest version.
Jun 15 17:54:51 game xpra[14208]: Markers: (--) probed, (**) from config file, (==) default setting,
Jun 15 17:54:51 game xpra[14208]:         (++) from command line, (!!) notice, (II) informational,
Jun 15 17:54:51 game xpra[14208]:         (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
Jun 15 17:54:51 game xpra[14208]: (++) Log file: "/var/run/user/1000/xpra/Xorg.:100.log", Time: Thu Jun 15 17:54:51 2017
Jun 15 17:54:51 game xpra[14208]: (++) Using config file: "/etc/xpra/xorg.conf"
Jun 15 17:54:51 game xpra[14208]: (==) Using system config directory "/usr/share/X11/xorg.conf.d"
Jun 15 17:54:51 game xpra[14208]: Warning: some of the sockets are in an unknown state:
Jun 15 17:54:51 game xpra[14208]:  /run/user/1000/xpra/game-100
Jun 15 17:54:51 game xpra[14208]:  /var/run/xpra/game-100
Jun 15 17:54:51 game xpra[14208]:  please wait as we allow the socket probing to timeout
Jun 15 17:54:57 game xpra[14208]: created unix domain socket: /run/user/1000/xpra/game-100
Jun 15 17:54:57 game xpra[14208]: created unix domain socket: /var/run/xpra/game-100
Jun 15 17:54:58 game xpra[14208]: Warning: webcam forwarding is disabled
Jun 15 17:54:58 game xpra[14208]:  the virtual video directory '/sys/devices/virtual/video4linux' was not found
Jun 15 17:54:58 game xpra[14208]:  make sure that the 'v4l2loopback' kernel module is installed and loaded
Jun 15 17:54:58 game xpra[14208]: found 0 virtual video devices for webcam forwarding
Jun 15 17:54:58 game xpra[14208]: pulseaudio server started with pid 14284
Jun 15 17:54:58 game xpra[14208]: GStreamer version 1.12.0 for Python 2.7.13 64-bit
Jun 15 17:54:58 game xpra[14208]: D-Bus notification forwarding is available
Jun 15 17:54:58 game xpra[14208]: xpra X11 version 2.1 64-bit
Jun 15 17:54:58 game xpra[14208]:  uid=1000 (docmax), gid=100 (users)
Jun 15 17:54:58 game xpra[14208]:  running with pid 14208 on Linux
Jun 15 17:54:58 game xpra[14208]:  connected to X11 display :100 with 24 bit colors
Jun 15 17:54:58 game xpra[14208]: xpra is ready.
Jun 15 17:54:59 game xpra[14208]: printer forwarding enabled using postscript and pdf
Jun 15 17:54:59 game xpra[14208]: 11.8GB of system memory


Thu, 15 Jun 2017 17:04:46 GMT - DocMAX: status changed; resolution set

looking in -d loader looks like the module pycuda is missing, just because in arch it doesnt work with cuda-7.5. will try further..


Thu, 15 Jun 2017 21:06:30 GMT - DocMAX:

NVEC is now initialized successfully. But still my client is way too slow! Whats the problem here?


Thu, 15 Jun 2017 23:21:59 GMT - DocMAX:

Checked the log again... do i really need a license key? Or is it a bug? How do i know?

2017-06-16 01:21:32,987 pycuda_info
2017-06-16 01:21:32,987 CUDA initialization (this may take a few seconds)
2017-06-16 01:21:33,093 CUDA 8.0.0 / PyCUDA 2017.1, found 1 device:
2017-06-16 01:21:33,093   + GeForce GTX 970 @ 0000:01:00.0 (memory: 93% free, compute: 5.2)
2017-06-16 01:21:33,122 * version                         : 2017.1
2017-06-16 01:21:33,122   - text                          : 2017.1
2017-06-16 01:21:33,122 cuda_info
2017-06-16 01:21:33,122 * driver
2017-06-16 01:21:33,122   - driver_version                : 8000
2017-06-16 01:21:33,122   - version                       : 8.0.0
2017-06-16 01:21:33,122 preferences:
2017-06-16 01:21:33,122 * blacklist                       : GTX 10
2017-06-16 01:17:36,174 init_cuda failed
Traceback (most recent call last):
  File "xpra/codecs/nvenc7/encoder.pyx", line 1492, in xpra.codecs.nvenc7.encoder.Encoder.init_context (xpra/codecs/nvenc7/encoder.c:12593)
  File "xpra/codecs/nvenc7/encoder.pyx", line 1657, in xpra.codecs.nvenc7.encoder.Encoder.init_nvenc (xpra/codecs/nvenc7/encoder.c:17379)
  File "xpra/codecs/nvenc7/encoder.pyx", line 1672, in xpra.codecs.nvenc7.encoder.Encoder.init_encoder (xpra/codecs/nvenc7/encoder.c:17676)
  File "xpra/codecs/nvenc7/encoder.pyx", line 1335, in xpra.codecs.nvenc7.encoder.raiseNVENC (xpra/codecs/nvenc7/encoder.c:9505)
NVENCException: initializing encoder - returned 8: This indicates that one or more of the parameter passed to the API call is invalid.
2017-06-16 01:17:36,174 encoder nvenc(BGRA/BGRX/H264 - low-latency-hq - 1920x1080) failed: initializing encoder - returned 8: This indicates that one or more of the parameter passed to the API call is invalid.
2017-06-16 01:17:36,174 error during NVENC encoder test: initializing encoder - returned 8: This indicates that one or more of the parameter passed to the API call is invalid.
2017-06-16 01:17:36,174  a license key may be required

Fri, 16 Jun 2017 07:54:17 GMT - Antoine Martin: component, description changed

Looks to me like this bug: ticket:1260#comment:16, we blacklisted GTX 10x0 cards - looks like this now affects other cards. Using an older / newer driver version may help.

(or maybe you do need a license key - unlikely, I think you should always get 2 contexts on consumer cards)


Fri, 16 Jun 2017 07:55:49 GMT - Antoine Martin:

Oh, and btw, xpra v2.1 (svn) is not a full version number, always include the full version with the exact svn revision.


Fri, 16 Jun 2017 08:00:16 GMT - DocMAX:

well, its the result of xpra --version. i dont know where else i can see the build number.


Fri, 16 Jun 2017 09:16:39 GMT - Antoine Martin:

well, its the result of xpra --version. i dont know where else i can see the build number.

When building from an svn checkout, the svnrevision should be included automatically. (captured from the output of svnversion where the code is built) When using packages, the svn version should be included already and in any case it is included in the package filename. See wiki/ReportingBugs


Sun, 18 Jun 2017 13:50:32 GMT - Antoine Martin: status changed; resolution deleted

NVENC SDK8 supported added in #1552 - this makes no difference as this new version adds almost nothing.

Will test using my GTX 970 when I get back.


Thu, 22 Jun 2017 03:09:50 GMT - DocMAX:

any updates? i'm stuck with

error during NVENC encoder test: initializing encoder - returned 8: This indicates that one or more of the parameter passed to the API call is invalid.

Thu, 22 Jun 2017 03:14:40 GMT - DocMAX:

oh, and i'm at r16116


Sun, 16 Jul 2017 11:56:19 GMT - Aynur Shakirov:

xpra 16362: the problem still exist, tested with Nvidia Quadro M2000 (driver version is 381.22) and nvenc v8. H265 codec is successfully initialize, but h264 initialization is fail.

^[[36m2017-07-16 14:44:45,201 get_preset(H264) speed=100, quality=50, lossless=False, pixel_format=BGRX, options={160: [('hq', '34DBA71D-A77B-4B8F-9C3E-B6
^[[36m2017-07-16 14:44:45,201 using preset 'low-latency-hq' for speed=100, quality=50, lossless=0, pixel_format=BGRX^[[0m
^[[36m2017-07-16 14:44:45,201 init_params(H264) using preset=low-latency-hq^[[0m
^[[36m2017-07-16 14:44:45,201 9 input format types:^[[0m
^[[36m2017-07-16 14:44:45,201 * 0x1^[[0m
^[[36m2017-07-16 14:44:45,201  + 0x1 : NV12_PL^[[0m
^[[36m2017-07-16 14:44:45,201 * 0x10^[[0m
^[[36m2017-07-16 14:44:45,201  + 0x10 : YV12_PL^[[0m
^[[36m2017-07-16 14:44:45,201 * 0x100^[[0m
^[[36m2017-07-16 14:44:45,201  + 0x100 : IYUV_PL^[[0m
^[[36m2017-07-16 14:44:45,201 * 0x1000^[[0m
^[[36m2017-07-16 14:44:45,202  + 0x1000 : YUV444_PL^[[0m
^[[36m2017-07-16 14:44:45,202 * 0x1000000^[[0m
^[[36m2017-07-16 14:44:45,202  + 0x1000000 : ARGB^[[0m
^[[36m2017-07-16 14:44:45,202 * 0x10000000^[[0m
^[[36m2017-07-16 14:44:45,202  + 0x10000000 : ABGR^[[0m
^[[36m2017-07-16 14:44:45,202 * 0x4000000^[[0m
^[[36m2017-07-16 14:44:45,202  + 0x4000000 : AYUV^[[0m
^[[36m2017-07-16 14:44:45,202 * 0x2000000^[[0m
^[[36m2017-07-16 14:44:45,202  + 0x2000000 : ARGB10^[[0m
^[[36m2017-07-16 14:44:45,202 * 0x20000000^[[0m
^[[36m2017-07-16 14:44:45,202  + 0x20000000 : ABGR10^[[0m
^[[36m2017-07-16 14:44:45,208 init_cuda failed
Traceback (most recent call last):
  File "xpra/codecs/nvenc/encoder.pyx", line 1510, in xpra.codecs.nvenc.encoder.Encoder.init_context (xpra/codecs/nvenc/encoder.c:12595)
  File "xpra/codecs/nvenc/encoder.pyx", line 1675, in xpra.codecs.nvenc.encoder.Encoder.init_nvenc (xpra/codecs/nvenc/encoder.c:17376)
  File "xpra/codecs/nvenc/encoder.pyx", line 1690, in xpra.codecs.nvenc.encoder.Encoder.init_encoder (xpra/codecs/nvenc/encoder.c:17673)
  File "xpra/codecs/nvenc/encoder.pyx", line 1353, in xpra.codecs.nvenc.encoder.raiseNVENC (xpra/codecs/nvenc/encoder.c:9507)
NVENCException: initializing encoder - returned 8: This indicates that one or more of the parameter passed to the API call is invalid.^[[0m
^[[36m2017-07-16 14:44:45,209 encoder nvenc(BGRA/BGRX/H264 - low-latency-hq - 1920x1080) failed: initializing encoder - returned 8: This indicates that on
^[[33m2017-07-16 14:44:45,209 error during NVENC encoder test: initializing encoder - returned 8: This indicates that one or more of the parameter passed
^[[36m2017-07-16 14:44:45,209  a license key may be required^[[0m

Mon, 17 Jul 2017 13:24:44 GMT - Antoine Martin:

I still get complete system lockups with my GTX 1070. It's been broken for months, time to get on it. Other NVENC tickets we should close for 2.1: #1519, #1347, #1317.


Looking at the cuda example that comes with nvenc 8, this is how they do it now in pseudo-code:

The only major differences that I can see:

The low latency example sets these options:

    encodeConfig.endFrameIdx = INT_MAX;
    encodeConfig.bitrate = 5000000;
    encodeConfig.rcMode = NV_ENC_PARAMS_RC_2_PASS_QUALITY;
    encodeConfig.gopLength = NVENC_INFINITE_GOPLENGTH;
    encodeConfig.deviceType = 0;
    encodeConfig.codec = NV_ENC_H264;
    encodeConfig.fps = 30;
    encodeConfig.qp = 28;
    encodeConfig.i_quant_factor = DEFAULT_I_QFACTOR;
    encodeConfig.b_quant_factor = DEFAULT_B_QFACTOR;
    encodeConfig.i_quant_offset = DEFAULT_I_QOFFSET;
    encodeConfig.b_quant_offset = DEFAULT_B_QOFFSET;
    encodeConfig.presetGUID = NV_ENC_PRESET_LOW_LATENCY_HQ_GUID;
    encodeConfig.pictureStruct = NV_ENC_PIC_STRUCT_FRAME;
    encodeConfig.numB = 0;
    m_stCreateEncodeParams.encodeGUID = inputCodecGUID;
    m_stCreateEncodeParams.presetGUID = pEncCfg->presetGUID;
    m_stCreateEncodeParams.encodeWidth = pEncCfg->width;
    m_stCreateEncodeParams.encodeHeight = pEncCfg->height;
    m_stCreateEncodeParams.darWidth = pEncCfg->width;
    m_stCreateEncodeParams.darHeight = pEncCfg->height;
    m_stCreateEncodeParams.frameRateNum = pEncCfg->fps;
    m_stCreateEncodeParams.frameRateDen = 1;
    m_stCreateEncodeParams.enableEncodeAsync = 0;
    m_stCreateEncodeParams.enablePTD = 1;
    m_stCreateEncodeParams.reportSliceOffsets = 0;
    m_stCreateEncodeParams.enableSubFrameWrite = 0;
    m_stCreateEncodeParams.encodeConfig = &m_stEncodeConfig;
    m_stCreateEncodeParams.maxEncodeWidth = m_uMaxWidth;
    m_stCreateEncodeParams.maxEncodeHeight = m_uMaxHeight;
    m_stEncodeConfig.gopLength = pEncCfg->gopLength;
    m_stEncodeConfig.frameIntervalP = pEncCfg->numB + 1;
        m_stEncodeConfig.frameFieldMode = NV_ENC_PARAMS_FRAME_FIELD_MODE_FRAME;

For YUV444:

            m_stEncodeConfig.encodeCodecConfig.hevcConfig.chromaFormatIDC = 3;
#OR:
            m_stEncodeConfig.encodeCodecConfig.h264Config.chromaFormatIDC = 3;

For 10 bit input:

            m_stEncodeConfig.encodeCodecConfig.h264Config.chromaFormatIDC = 3;

etc..

Modifiying the cuda example to print all method calls I see:

$ ./NvEncoderCudaInterop -i /opt/Shared/Xpra-Build-Libs/nvenc_4.0.0_sdk/Samples/YUV/1080p/PixelBlur-1920x1080.yuv  -o test -size 1920 1080 -numB 0
Encoding input           : "/opt/Shared/Xpra-Build-Libs/nvenc_4.0.0_sdk/Samples/YUV/1080p/PixelBlur-1920x1080.yuv"
         output          : "test"
         codec           : "H264"
         size            : 1920x1080
         bitrate         : 5000000 bits/sec
         vbvMaxBitrate   : 0 bits/sec
         vbvSize         : 0 bits
         fps             : 30 frames/sec
         rcMode          : CONSTQP
         goplength       : INFINITE GOP
         B frames        : 0
         QP              : 28
         preset          : DEFAULT
BufferCount : 1
AsyncMode   : 0
AllocateIOBuffers
loadframe
ConvertYUVToNV12
NvEncMapInputResource
NvEncEncodeFrame
loadframe
no encode buffer, calling ProcessOutput
ProcessOutput: nvEncLockBitstream
ProcessOutput: nvEncUnlockBitstream
NvEncUnmapInputResource
ConvertYUVToNV12
NvEncMapInputResource
NvEncEncodeFrame
loadframe
no encode buffer, calling ProcessOutput
ProcessOutput: nvEncLockBitstream
ProcessOutput: nvEncUnlockBitstream
NvEncUnmapInputResource
ConvertYUVToNV12
NvEncMapInputResource
NvEncEncodeFrame
loadframe
etc...
loadframe
ProcessOutput: nvEncLockBitstream
ProcessOutput: nvEncUnlockBitstream
Encoded 116 frames in 293.04ms
Avergage Encode Time :   2.53ms

Mon, 17 Jul 2017 17:39:51 GMT - Antoine Martin:

Well, this has taken many hours and something like ~20 to 30 full system lockups followed by reboots and lots of swearing. In the end, the bug is clearly an underflow in the nvidia API, just like we saw when HEVC support was added (which also cost me hours of wasted time back then): ticket:1046#comment:6.

Fixes in:

Most of this should be backported. We still get this error for some codec / settings combinations:

xpra.codecs.nvenc.encoder.NVENCException: initializing encoder - returned 8:
This indicates that one or more of the parameter passed to the API call is invalid.

But at least now I stand a chance of being to fix it.


Tue, 18 Jul 2017 14:44:59 GMT - Antoine Martin: owner, status changed

Now for the updates and proper fixes:

@DocMAX: please close if this works for you.


Wed, 19 Jul 2017 06:27:01 GMT - Antoine Martin:

More fixes (backporting this mess is not going to be easy - might just go for the easy option: just disable most of nvenc and recommend the newer version):


Wed, 19 Jul 2017 14:58:17 GMT - Antoine Martin: status changed; resolution set

Fixed and tested, see ticket:1519#comment:3


Wed, 19 Jul 2017 16:11:32 GMT - Antoine Martin:

Tedious backporting to older branches done and tested in r16416. We don't support HEVC in older branches (easier), use 2.1 or later if you need this.


Sat, 23 Jan 2021 05:27:23 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1550