Getting Occasional OpenGL Tracebacks

[J. Max Mena]

My server and client are both Fedora 26 machines running trunk r18446 built from source.

My client machine has an Nvidia GPU (a GTX 745 to be specific), and occasionally I will get this traceback (usually while running Firefox - as far as I can tell it's not specific to any sites, I've gotten it on Wikipedia before). Normally I frown upon writing "Weird errors in the logs" tickets without thoroughly understanding what's going on first, but OpenGL isn't my forte and usually when I see this traceback in the log the paints go a little weird. The last couple times I saw it, Firefox stopped painting altogether until I re-attached.

Here's the traceback:

2018-02-16 11:21:28,675  flush=2, image=ImageWrapper(YUV420P:(0, 0, 1244, 1, 24):PACKED), coords=(0, 325, 1244, 1), size=1244x1
2018-02-16 12:49:45,405 Error painting planar update
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/client/gl/gl_window_backing_base.py", line 988, in gl_paint_planar
    self.update_planar_textures(x, y, enc_width, enc_height, img, pixel_format, scaling=(enc_width!=width or enc_height!=height))
  File "/usr/lib64/python2.7/site-packages/xpra/client/gl/gl_window_backing_base.py", line 1054, in update_planar_textures
    glTexSubImage2D(target, 0, 0, 0, width//div_w, height//div_h, GL_LUMINANCE, GL_UNSIGNED_BYTE, pixel_data)
  File "latebind.pyx", line 32, in OpenGL_accelerate.latebind.LateBind.__call__ (src/latebind.c:989)
  File "wrapper.pyx", line 318, in OpenGL_accelerate.wrapper.Wrapper.__call__ (src/wrapper.c:6561)
GLError: GLError(
	err = 1281,
	description = 'invalid value',
	baseOperation = glTexSubImage2D,
	pyArgs = (
		GL_TEXTURE_RECTANGLE_ARB,
		0,
		0,
		0,
		622,
		0,
		GL_LUMINANCE,
		GL_UNSIGNED_BYTE,
		<memory at 0x7f914813c938>,
	),
	cArgs = (
		GL_TEXTURE_RECTANGLE_ARB,
		0,
		0,
		0,
		622,
		0,
		GL_LUMINANCE,
		GL_UNSIGNED_BYTE,
		<memory at 0x7f914813c938>,
	),
	cArguments = (
		GL_TEXTURE_RECTANGLE_ARB,
		0,
		0,
		0,
		622,
		0,
		GL_LUMINANCE,
		GL_UNSIGNED_BYTE,
		<memory at 0x7f914813c938>,
	)
)
2018-02-16 12:49:45,406  flush=0, image=ImageWrapper(YUV420P:(0, 0, 1244, 1, 24):PACKED), coords=(0, 547, 1244, 1), size=1244x1

[Antoine Martin]

I believe this is caused by #1438 + #1764: newer clients upload YUV data directly (without first converting to RGB) so the subsampled planes can end up zero sized, and using zero sized buffer makes this an illegal pixel upload command.

So:

• r18447 should prevent webp or jpeg from being used for those small sizes. These are the two non-video encodings which may use subsampling - video encodings already have size restrictions. (+minor unlikely to hit fixup for jpeg2000 in r18450)
• r18448 similar thing client side: avoids using YUV painting for webp and jpeg if the size is problematic (should only happen when connecting to older or buggy servers)
• r18449 improves the handling of pixel upload so we will get more helpful error messages if errors still occur
• the problem with errors in opengl rendering is that they can leave the opengl context in an inconsistent state: #1770

Those changes make the encoding selection even slower (extra validity checks), which increases the need for #1700.

[J. Max Mena]

Upped server and client to r18503:

OpenGL errors are gone - closing (I assume we'll follow up in #1770)