Xpra: Ticket #812: re-implement clipboard without nested main

As it is, it is just too problematic: too many bugs to list, including some unresolved ones: #669, #452 (it is also causing hangs with #598). See also #2172, #2138

We should be able to rip it out and just use plain X11 calls (see ICCCM section 2: Peer-to-Peer Communication by Means of Selections), hopefully GTK won't get too confused by this...

We can keep the existing code, at least client side, for the non-X11 platforms.

Link: x-cut-and-paste



Tue, 24 Feb 2015 03:40:34 GMT - Antoine Martin: status changed

Another reason for doing this, just hit this today (not sure how):

RuntimeError: maximum recursion depth exceeded
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/clipboard/clipboard_base.py", line 167, in _get_clipboard_from_remote_handler
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/gtk_common/quit.py", line 67, in gtk_main_quit_on_fatal_exception
    print(traceback.print_exception(etype, val, tb))
  File "/usr/lib64/python2.7/traceback.py", line 125, in print_exception
    print_tb(tb, limit, file)
  File "/usr/lib64/python2.7/traceback.py", line 69, in print_tb
    line = linecache.getline(filename, lineno, f.f_globals)
  File "/usr/lib64/python2.7/linecache.py", line 14, in getline
    lines = getlines(filename, module_globals)
  File "/usr/lib64/python2.7/linecache.py", line 40, in getlines
    return updatecache(filename, module_globals)
RuntimeError: maximum recursion depth exceeded
Original exception was:
Traceback (most recent call last):
  File "/usr/lib64/python2.7/site-packages/xpra/clipboard/clipboard_base.py", line 167, in _get_clipboard_from_remote_handler
    log("get clipboard from remote handler id=%s", request_id)
RuntimeError: maximum recursion depth exceeded

Mon, 18 May 2015 16:36:16 GMT - Antoine Martin: description changed


Wed, 17 Jun 2015 23:03:14 GMT - onlyjob: cc set


Sun, 28 Jun 2015 09:40:07 GMT - Antoine Martin:

API links:


Fri, 23 Oct 2015 06:50:45 GMT - Antoine Martin: milestone changed

Re-scheduling due to lack of time.

Another item worth considering during the re-write would be to make all clipboards "greedy" like the win32 and osx ones, or at least having the option of doing that.

#1018 is a duplicate of this ticket.


Wed, 15 Feb 2017 12:42:09 GMT - Antoine Martin: milestone changed

Likely to be quite hard.


Thu, 20 Jul 2017 16:43:47 GMT - Antoine Martin:

See also: #1589 GTK3 clipboard support


Sun, 04 Nov 2018 13:39:49 GMT - Antoine Martin:

This should help fix bugs like #2025 and #2172


Tue, 12 Mar 2019 04:26:29 GMT - Antoine Martin: priority changed

More problems (and yet more reported on IRC): #2138, see also #1844.


Tue, 19 Mar 2019 04:17:40 GMT - Antoine Martin: description changed


Fri, 22 Mar 2019 14:59:05 GMT - Antoine Martin: summary changed

Preparatory steps added in r22212.

We can't completely do without GTK because the hooks for X11 events are in the gtk main loop filter, and the events are dispatched as gobject signals. But at least it should be easier to move away from GTK at some point.

And we also need to use the xfixes API, because it's better. See also #1494


Sun, 24 Mar 2019 13:11:43 GMT - Antoine Martin:

Added:

The toy clipboard class can now send a receive clipboard data.


Wed, 27 Mar 2019 14:10:17 GMT - Antoine Martin: description changed

Partial merge:

TODO:


Thu, 28 Mar 2019 09:28:12 GMT - Antoine Martin:

Updates:

It works surprisingly well and does not ever lock the main thread!

Still TODO:

etc


Thu, 11 Apr 2019 12:26:20 GMT - Antoine Martin:

Updates:

Still TODO:


Fri, 12 Apr 2019 15:36:54 GMT - totaamwin32:

win32 updates:


Fri, 12 Apr 2019 15:37:20 GMT - totaamwin32: attachment set

win32 native clipboard work in progress


Sun, 14 Apr 2019 16:19:13 GMT - totaamwin32:

updates:


Thu, 18 Apr 2019 12:00:09 GMT - Antoine Martin:

Updates:


Sat, 20 Apr 2019 10:09:38 GMT - Antoine Martin:

Updates:

We do have to deal with the macos clipboard to be able to cleanup the codebase and remove the legacy gdk clipboard classes.


Sun, 21 Apr 2019 18:01:09 GMT - Antoine Martin:

Caused a regression: #2280.


Mon, 22 Apr 2019 04:27:40 GMT - Antoine Martin:

#2280 fixed in r22501, we don't release the selection on exit to avoid this GTK3 crash. Ideally, we should find a way to release the selection without crashing GTK3 but this will do for now.


Mon, 22 Apr 2019 15:06:24 GMT - Antoine Martin:

Updates:

Still TODO:


Tue, 30 Apr 2019 12:23:44 GMT - Antoine Martin: owner, status changed; reporter deleted

Updates:

This will do for this ticket, will follow up in #273.

@encodedEntropy: this really needs testing, ideally before #1844 so if anything is broken then we will know if it's here or there.


Fri, 31 May 2019 12:43:52 GMT - Antoine Martin: owner, status changed

Something's not right. When copying an image from eog on the server (testing for #1494):

do_owner_changed()
_send_clipboard_token_handler(X11ClipboardProxy(CLIPBOARD), ())
requesting local XConvertSelection from 'Image Viewer' for 'TARGETS' into 'CLIPBOARD-TARGETS'
client @25.162 got token, selection=CLIPBOARD, targets=None, target data=None, claim=True, can-receive=True
_send_clipboard_token_handler(X11ClipboardProxy(CLIPBOARD), (('TIMESTAMP', 'TARGETS', 'MULTIPLE', 'image/png', 'image/bmp', 'image/x-bmp', 'image/x-MS-bmp', 'image/x-icon', 'image/x-ico', 'image/x

Why do we request the targets and send a clipboard-token to the client without them? Do one or the other!

Then immediately, gedit and Terminal are being greedy and requesting the targets:

client @25.164 clipboard request for CLIPBOARD from window 0x3400125: 'gedit'
client @25.166 clipboard request for CLIPBOARD from window 0x2400002: 'Terminal'

We get the targets again:

proxy_got_contents(6, CLIPBOARD, TARGETS, ATOM, 32, <class 'bytes'>:192) data=0xa001...

And send them to the client:

client @25.167 got token, selection=CLIPBOARD, targets=(b'TIMESTAMP', b'TARGETS', b'MULTIPLE', b'image/png', b'image/bmp', b'image/x-bmp', b'image/x-MS-bmp', b'image/x-icon', b'image/x-ico', b'image/x-win-bitmap', b'image/vnd.microsoft.icon', b'application/ico', b'image/ico', b'image/icon', b'text/ico', b'image/jpeg', b'image/tiff', b'UTF8_STRING', b'COMPOUND_TEXT', b'TEXT', b'STRING', b'text/plain;charset=utf-8', b'text/plain', b'text/uri-list'), target data=None, claim=True, can-receive=True

We should filter the image/FORMAT list to only contain formats that we can process through pillow.

We tell both gedit and Terminal about the targets:

client @25.168 setting response b'\x99\x01\x00\x00.. ..\x00\x00' to property GDK_SELECTION of window 'gedit' as ATOM
client @25.168 setting response b'\x99\x01\x00\x00.. ..\x00\x00' to property GDK_SELECTION of window 'Terminal' as ATOM

Probably because we claim the clipboard again when handling this token, they request the targets again, and we respond with the cached value:

client @25.171 clipboard request for CLIPBOARD from window 0x3400125: 'gedit'
client @25.171 using existing TARGETS value as response
client @25.172 clipboard request for CLIPBOARD from window 0x2400002: 'Terminal'
client @25.172 using existing TARGETS value as response

When trying to paste it into the gimp, it requests the data in image/png format:

client @19.259 clipboard request for CLIPBOARD from window 0x42029cd: 'GNU Image Manipulation Program'
client @19.259 using existing TARGETS value as response
client @19.264 clipboard request for CLIPBOARD from window 0x42029cd: 'GNU Image Manipulation Program'
client @19.264 send_clipboard_request_handler(X11ClipboardProxy(CLIPBOARD), 'CLIPBOARD', 'image/png')

The server tries to honour it:

requesting local XConvertSelection from 'Image Viewer' for 'image/png' into 'CLIPBOARD-image/png'
proxy_got_contents(1, CLIPBOARD, image/png, INCR, 32, <class 'bytes'>:8) data=0x470b040000000000..

This fails:

xpra.x11.bindings.window_bindings.BadPropertyType: incomplete data: 196608 bytes after

Because we don't handle INCR properties.


Sat, 01 Jun 2019 05:43:55 GMT - Antoine Martin:

r22824 adds support for incremental transfers IN. (+minor fixup in r22825)

Still TODO:


Sat, 01 Jun 2019 06:43:07 GMT - Antoine Martin:

added PoC overlay code in r22826 from both images and timestamp, only for png images for now.


Sat, 01 Jun 2019 11:50:05 GMT - Antoine Martin:

Supporting this in the html5 client is going to be difficult, #1844 may help. See:


Mon, 03 Jun 2019 04:01:06 GMT - Antoine Martin:

Moving html5 to #2312.


Wed, 05 Jun 2019 11:22:15 GMT - Antoine Martin:

The double requests issue from comment:26 is fixed in r22856.


Wed, 05 Jun 2019 11:26:35 GMT - Antoine Martin:

Unrestricted html5 clipboard access (at least for chrome): #2320.


Thu, 06 Jun 2019 06:02:46 GMT - Antoine Martin:

See also brotli compression for text: #2289


Tue, 11 Jun 2019 11:31:06 GMT - Antoine Martin: status changed; resolution set

Works well enough for this release. Closing.

Issues should be filed as new bugs, pointing back to this ticket.


Wed, 12 Jun 2019 10:59:46 GMT - Antoine Martin:

Minor improvements:

See also updates in #1844.


Thu, 20 Jun 2019 08:46:10 GMT - Antoine Martin:

Another regression spotted: ticket:2338#comment:3.


Wed, 17 Jul 2019 12:08:37 GMT - Antoine Martin:

Updates:


Fri, 19 Jul 2019 14:55:42 GMT - Antoine Martin:

The unit tests found a bug: r23215. They are now also enabled for python3: r23219.


Sun, 28 Jul 2019 06:11:41 GMT - Antoine Martin:

Clipboard won't work well at all with wayland clients, even if we re-added the GTK code (see ticket:2243#comment:10).


Tue, 30 Jul 2019 13:53:35 GMT - Antoine Martin:

Clipboard did not work with xsettings=no (#2342), that's because the new clipboard needs to listen for root window events. Fixed in r23369.


Thu, 08 Aug 2019 06:03:34 GMT - Antoine Martin:

The nested-main-loop code is finally removed in r23470.


Mon, 21 Oct 2019 06:48:59 GMT - Antoine Martin:

Potential regressions: #2450, #2452


Sat, 23 Jan 2021 05:06:45 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/812