Xpra: Ticket #173: extended input device support

think: tablet pcs can be used with the program running on the higher end desktop machine.

Pressure sensitivity, fine-grained scrolling, etc.



Thu, 27 Sep 2012 15:23:56 GMT - Antoine Martin: status, milestone changed


Mon, 13 Jul 2015 05:30:50 GMT - rektide: cc set


Wed, 30 Mar 2016 04:01:59 GMT - Antoine Martin: milestone changed

See investigation in ticket:1131#comment:5

Looks like we want uinput.


Wed, 30 Mar 2016 04:03:34 GMT - Antoine Martin: description, summary changed


Mon, 30 May 2016 14:56:50 GMT - Antoine Martin:

Best links from ticket:1131#comment:5 and more:


Fri, 03 Jun 2016 10:25:03 GMT - Antoine Martin:

More information:


Fri, 03 Jun 2016 10:51:49 GMT - Antoine Martin:

And some information from a trackpad which works for smooth scrolling. From libinput-list-devices:

Device:           DLL0704:01 06CB:76AE Touchpad
Kernel:           /dev/input/event13
Group:            6
Seat:             seat0, default
Size:             101.33x56.67mm
Capabilities:     pointer
Tap-to-click:     disabled
Tap-and-drag:     enabled
Tap drag lock:    disabled
Left-handed:      disabled
Nat.scrolling:    disabled
Middle emulation: n/a
Calibration:      n/a
Scroll methods:   *two-finger edge
Click methods:    *button-areas clickfinger
Disable-w-typing: enabled
Accel profiles:   none
$ xinput list | grep -i touchpad
⎜   ↳ DLL0704:01 06CB:76AE Touchpad           	id=11	[slave  pointer  (2)]
$ xinput list-props 11
Device 'DLL0704:01 06CB:76AE Touchpad':
	Device Enabled (137):	1
	Coordinate Transformation Matrix (139):	1.000000, 0.000000, 0.000000, 0.000000, 1.000000, 0.000000, 0.000000, 0.000000, 1.000000
	libinput Tapping Enabled (275):	1
	libinput Tapping Enabled Default (276):	0
	libinput Tapping Drag Lock Enabled (277):	0
	libinput Tapping Drag Lock Enabled Default (278):	0
	libinput Accel Speed (279):	0.000000
	libinput Accel Speed Default (280):	0.000000
	libinput Natural Scrolling Enabled (281):	0
	libinput Natural Scrolling Enabled Default (282):	0
	libinput Send Events Modes Available (259):	1, 1
	libinput Send Events Mode Enabled (260):	0, 0
	libinput Send Events Mode Enabled Default (261):	0, 0
	libinput Left Handed Enabled (283):	0
	libinput Left Handed Enabled Default (284):	0
	libinput Scroll Methods Available (285):	1, 1, 0
	libinput Scroll Method Enabled (286):	1, 0, 0
	libinput Scroll Method Enabled Default (287):	1, 0, 0
	libinput Click Methods Available (288):	1, 1
	libinput Click Method Enabled (289):	1, 0
	libinput Click Method Enabled Default (290):	1, 0
	libinput Disable While Typing Enabled (291):	1
	libinput Disable While Typing Enabled Default (292):	1
	Device Node (262):	"/dev/input/event13"
	Device Product ID (263):	1739, 30382
	libinput Drag Lock Buttons (293):	<no items>
	libinput Horizonal Scroll Enabled (264):	1
$ xinput query-state 11
2 classes :
ButtonClass
	button[1]=up
	button[2]=up
	button[3]=up
	button[4]=up
	button[5]=up
	button[6]=up
	button[7]=up
ValuatorClass Mode=Relative Proximity=In
	valuator[0]=467
	valuator[1]=1056
	valuator[2]=-1401
	valuator[3]=2254
$ xinput get-button-map 11
1 2 3 4 5 6 7
$ xinput get-feedbacks 11
1 feedback class
PtrFeedbackClass id=0
	accelNum is 2
	accelDenom is 1
	threshold is 4

Two finger scrolling as seen with libinput-debug-events:

event13	POINTER_AXIS      +2.73s	vert 6.07* horiz 0.49*
event13	POINTER_AXIS      +2.74s	vert 6.56* horiz 0.49*
event13	POINTER_AXIS      +2.75s	vert 6.89* horiz 0.82*

xev only sees pairs of button events:

ButtonPress event, serial 33, synthetic NO, window 0x2800001,
    root 0xd6, subw 0x0, time 14291002, (106,47), root:(144,149),
    state 0x10, button 5, same_screen YES
ButtonRelease event, serial 33, synthetic NO, window 0x2800001,
    root 0xd6, subw 0x0, time 14291002, (106,47), root:(144,149),
    state 0x1010, button 5, same_screen YES
ButtonPress event, serial 33, synthetic NO, window 0x2800001,
    root 0xd6, subw 0x0, time 14291044, (106,47), root:(144,149),
    state 0x10, button 5, same_screen YES
ButtonRelease event, serial 33, synthetic NO, window 0x2800001,
    root 0xd6, subw 0x0, time 14291044, (106,47), root:(144,149),
    state 0x1010, button 5, same_screen YES

Wed, 15 Jun 2016 05:31:15 GMT - Antoine Martin:

FWIW: wmi exposes input device information:

import wmi
w = wmi.WMI()
[(x.Name, x.Description, x.HardwareType, x.PointingType) for x in list(w.InstancesOf("Win32_PointingDevice"))]

For more information on device fields, see WMI Tasks: Computer Hardware


Tue, 12 Jul 2016 16:52:22 GMT - Antoine Martin: milestone changed

Milestone renamed


Mon, 10 Oct 2016 05:46:57 GMT - Antoine Martin:

xorg ML response from input maintainer Peter Hutterer: https://bugs.freedesktop.org/show_bug.cgi?id=51533)

The other option is to emulate a touchpad and forward the touches as-is. Then you let userspace take care of converting 2 finger to scroll, etc. That works for android devices with screen, but obviously not for others.


Mon, 10 Oct 2016 05:47:26 GMT - Antoine Martin: milestone changed

Out of time.


Sun, 19 Feb 2017 06:28:13 GMT - Antoine Martin: milestone changed


Fri, 24 Feb 2017 04:17:20 GMT - Antoine Martin: attachment set

uinput example


Fri, 24 Feb 2017 04:17:39 GMT - Antoine Martin:

New thread about the same issue: Virtual Touch Events for XTEST. Includes this code sample: attachment/ticket/173/main_v2.c.


Fri, 10 Mar 2017 06:59:46 GMT - Antoine Martin:

This will also help handling javascript wheel events better, see ticket:1424#comment:3


Fri, 10 Mar 2017 07:00:09 GMT - Antoine Martin: status changed

See events client side: monitoring events (keyboard, mouse) in X using:

xinput test-xi2 --root

Wed, 03 May 2017 06:27:36 GMT - Antoine Martin: attachment set

handle XI2 events


Wed, 03 May 2017 16:25:17 GMT - Antoine Martin: attachment set

updated patch for r15792


Wed, 03 May 2017 16:29:06 GMT - Antoine Martin:

More links:

Maybe we want to ensure that the keyboard layout is set per device and does not change: New udev property: XKB_FIXED_LAYOUT for keyboards that must not change layouts (but how do we set this without udev?)

Preparatory work in r15790 + r15792. The patch attached allows us to receive XINPUT2 events: raw touch, keyboard and pointer events. Still TODO:


Thu, 04 May 2017 06:50:28 GMT - Antoine Martin:

More useful pointers:


Thu, 04 May 2017 10:58:01 GMT - Antoine Martin: attachment set

updated patch with device discovery


Thu, 04 May 2017 11:02:20 GMT - Antoine Martin:

This bug may prevent us from using synthetic events: XSendEvent() fails on XInput2 events

Other problems:

To solve both, we could record all XI2 events until we get a "regular" event and then use the extra data. The problems with this approach if that the raw pointer / scroll events are fine grained and may not fire a "regular" event until much later... and some may be discarded altogether.


Thu, 04 May 2017 17:01:02 GMT - Antoine Martin: attachment set

updated patch for r15794


Fri, 05 May 2017 16:30:04 GMT - Antoine Martin: attachment set

updated patch


Fri, 05 May 2017 16:38:09 GMT - Antoine Martin:

New issues:


Sat, 06 May 2017 16:59:30 GMT - Antoine Martin: attachment set

better approach: register event handlers directly with the XI2 helper class


Sat, 06 May 2017 18:22:18 GMT - Antoine Martin:

With the updated patch, we get the correct events for pointer motion and clicks (keyboard events remain to be done).

Now, the hard part: setting up the server side devices.

If we use uinput, we should ship a file in /etc/X11/xorg.conf.d/ to skip our virtual devices by default, to avoid creating loops when the client is running locally - we can match the device in an inputclass section using MatchVendor and then set Option "Ignore" "true"

The problem with uinput is that as raw as the xinput2 events are, they're not from the same level as uinput.. so some interpretation / un-translation is still going to be needed to re-inject them in such a way that libinput will generate the exact events we want.

But maybe we don't want to use uinput + libinput at all? We could also write our own input driver and feed it the events directly? This way we don't need elevated permissions to talk to /dev/uinput and we can just create our own private input socket. How would that work with dynamic devices though?


Sun, 07 May 2017 10:42:51 GMT - Antoine Martin: attachment set

updated patch: export all device properties


Sun, 07 May 2017 16:33:46 GMT - Antoine Martin: attachment set

updated patch: server know knows details of the device used for some events


Sun, 07 May 2017 16:46:39 GMT - Antoine Martin:

With the latest patch, the server has all the information it needs about the input devices (device changes need to be propagated - easy) and the input events include the device id (most of them - easy to add).

As for simulating the devices on the server, these examples may help:

The problem is that those examples work on "evdev" devices.. and we get the device information from the X11 server using the "xinput2" API... which is mapped from the kernel's evdev devices via the "libinput" or the "evdev" Xorg driver layer. It should be possible to retrieve the corresponding evdev device that libinput / evdev uses, but the events we'll receive from the X11 server will be "xinput2" events...


Mon, 08 May 2017 07:03:59 GMT - Antoine Martin: attachment set

tell Xorg to always ignore the virtual input devices we create


Mon, 08 May 2017 07:04:38 GMT - Antoine Martin: attachment set

tell Xdummy to add our virtual input devices


Mon, 08 May 2017 07:05:35 GMT - Antoine Martin: attachment set

change the udev permissions on input devices


Mon, 08 May 2017 10:03:39 GMT - Antoine Martin: attachment set

use uinput to inject events


Mon, 08 May 2017 10:06:49 GMT - Antoine Martin:

Installing:


udev will create input devices that a non-root Xorg can access. (we should probably use group access instead of 666 permisions, and restrict the changes to our virtual input devices, not all input devices... see udev rules: controlling permissions and ownership) The default Xorg server will ignore our virtual input devices:

/usr/libexec/gdm-x-session[9751]: (II) config/udev: Adding input device Fake xpra pointer (/dev/input/event17)
/usr/libexec/gdm-x-session[9751]: (**) Fake xpra pointer: Ignoring device from InputClass "Xpra devices should not be used automatically"

but Xdummy will add and enable them:

[ 96341.568] (II) config/udev: Adding input device Fake xpra pointer (/dev/input/event17)
[ 96341.568] (**) Fake xpra pointer: Applying InputClass "evdev pointer catchall"
[ 96341.568] (**) Fake xpra pointer: Applying InputClass "libinput pointer catchall"
[ 96341.568] (**) Fake xpra pointer: Applying InputClass "All devices"
[ 96341.568] (**) Fake xpra pointer: Applying InputClass "Xpra devices should not be used automatically"
[ 96341.568] (II) Using input driver 'libinput' for 'Fake xpra pointer'
[ 96341.568] (**) Fake xpra pointer: always reports core events
[ 96341.568] (**) Option "Device" "/dev/input/event17"
[ 96341.568] (**) Option "_source" "server/udev"
[ 96341.569] (II) event17 - (II) Fake xpra pointer: (II) is tagged by udev as: Mouse
[ 96341.569] (II) event17 - (II) Fake xpra pointer: (II) device is a pointer
[ 96341.569] (II) event17 - (II) Fake xpra pointer: (II) device removed
[ 96341.584] (**) Option "config_info" "udev:/sys/devices/virtual/input/input68/event17"
[ 96341.584] (II) XINPUT: Adding extended input device "Fake xpra pointer" (type: MOUSE, id 6)
[ 96341.584] (**) Option "AccelerationScheme" "none"
[ 96341.584] (**) Fake xpra pointer: (accel) selected scheme none/0
[ 96341.584] (**) Fake xpra pointer: (accel) acceleration factor: 2.000
[ 96341.584] (**) Fake xpra pointer: (accel) acceleration threshold: 4

With this in place, I can simulate devices via uinput and only Xdummy sees the events. This can be verified easily with:

xpra info  | grep -i cursor.position

For injecting events, maybe python-uinput? (at least the capabilities are better documented)

With the latest patch, I can inject relative motion... but not absolute. (mouse pointers don't do absolute) Looks like this needs more options to be able to configure the virtual devices properly (see min==max "kernel bug"):

[107288.819] (II) config/udev: Adding input device Logitech M705 - xpra pointer (/dev/input/event17)
[107288.819] (**) Logitech M705 - xpra pointer: Applying InputClass "evdev pointer catchall"
[107288.819] (**) Logitech M705 - xpra pointer: Applying InputClass "libinput pointer catchall"
[107288.819] (**) Logitech M705 - xpra pointer: Applying InputClass "All devices"
[107288.819] (**) Logitech M705 - xpra pointer: Applying InputClass "Xpra devices should not be used automatically"
[107288.819] (II) Using input driver 'libinput' for 'Logitech M705 - xpra pointer'
[107288.819] (**) Logitech M705 - xpra pointer: always reports core events
[107288.819] (**) Option "Device" "/dev/input/event17"
[107288.819] (**) Option "_source" "server/udev"
[107288.820] (II) event17 - (II) Logitech M705 - xpra pointer: (II) is tagged by udev as: Mouse
[107288.820] (EE) event17 - (EE) Logitech M705 - xpra pointer: (EE) kernel bug: device has min == max on ABS_X
[107288.820] (II) event17 - (II) Logitech M705 - xpra pointer: (II) was rejected
[107288.844] (II) event17 - not using input device '/dev/input/event17'.
[107288.844] (EE) libinput: Logitech M705 - xpra pointer: Failed to create a device for /dev/input/event17
[107288.844] (EE) PreInit returned 2 for "Logitech M705 - xpra pointer"
[107288.844] (II) UnloadModule: "libinput"
[107288.845] (II) config/udev: Adding input device DLL0704:01 06CB:76AE Touchpad - xpra pointer (/dev/input/event18)
[107288.845] (**) DLL0704:01 06CB:76AE Touchpad - xpra pointer: Applying InputClass "evdev pointer catchall"
[107288.845] (**) DLL0704:01 06CB:76AE Touchpad - xpra pointer: Applying InputClass "libinput pointer catchall"
[107288.845] (**) DLL0704:01 06CB:76AE Touchpad - xpra pointer: Applying InputClass "All devices"
[107288.845] (**) DLL0704:01 06CB:76AE Touchpad - xpra pointer: Applying InputClass "Xpra devices should not be used automatically"
[107288.845] (II) Using input driver 'libinput' for 'DLL0704:01 06CB:76AE Touchpad - xpra pointer'
[107288.845] (**) DLL0704:01 06CB:76AE Touchpad - xpra pointer: always reports core events
[107288.845] (**) Option "Device" "/dev/input/event18"
[107288.845] (**) Option "_source" "server/udev"
[107288.847] (II) event18 - (II) DLL0704:01 06CB:76AE Touchpad - xpra pointer: (II) is tagged by udev as: Mouse
[107288.847] (EE) event18 - (EE) DLL0704:01 06CB:76AE Touchpad - xpra pointer: (EE) kernel bug: device has min == max on ABS_X
[107288.847] (II) event18 - (II) DLL0704:01 06CB:76AE Touchpad - xpra pointer: (II) was rejected
[107288.872] (II) event18 - not using input device '/dev/input/event18'.
[107288.872] (EE) libinput: DLL0704:01 06CB:76AE Touchpad - xpra pointer: Failed to create a device for /dev/input/event18
[107288.872] (EE) PreInit returned 2 for "DLL0704:01 06CB:76AE Touchpad - xpra pointer"
[107288.872] (II) UnloadModule: "libinput"

Mon, 08 May 2017 10:45:50 GMT - Antoine Martin: attachment set

switch to using python-evdev


Thu, 11 May 2017 04:20:50 GMT - Antoine Martin: milestone changed

Mostly merged in r15802 (big), this doesn't do anything useful yet and is not enabled unless the client specifies --input-devices=xi.

With "xi" enabled, the server gets a list of devices, which is not used for anything yet, and the pointer packets include the device responsible for the event (as well as more details on the event), ie:

process_mouse_common from device=Logitech G400s Optical Gaming Mouse

This will have to do for this release, at least the xinput code is merged and won't bitrot too much. Enabling uinput devices for touch events only should work, but the better approach is to write a new xorg input driver so we can inject the events exactly as we receive them.


Tue, 23 May 2017 09:14:06 GMT - Antoine Martin:

This may help with MS Windows to get the device ID and layout for each keyboard device:

$ python -c "from comtypes.client import CreateObject
o=CreateObject('WbemScripting.SWbemLocator')
s=o.ConnectServer('.', 'root\\cimv2')
res=s.ExecQuery('SELECT * FROM Win32_Keyboard')
print([(x.Properties_['DeviceID'].Value, x.Properties_['Layout'].Value) for x in res])
[(u'ACPI\\PNP0303\\4&1D401FB5&0', u'00000809')]

Sun, 30 Jul 2017 18:05:23 GMT - Antoine Martin: attachment set

implement pointer as a uinput device


Mon, 31 Jul 2017 17:47:37 GMT - Antoine Martin: milestone changed

Moved the uinput server-side support to #1611.

We can keep this ticket as a tracker.


Mon, 28 Aug 2017 04:07:30 GMT - Antoine Martin:

r16721 uses xinput2 devices on posix clients by default.


Mon, 22 Jan 2018 09:12:09 GMT - urzds: cc changed


Sun, 22 Apr 2018 17:14:00 GMT - Antoine Martin:

See #1615 for touch device support


Mon, 08 Jul 2019 16:39:30 GMT - Antoine Martin: milestone changed


Sat, 23 Jan 2021 04:47:31 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/173