xpra stop all does not stop all

[stdedos]

$ xpra list
Found the following xpra sessions:
/run/user/1000/xpra:
	LIVE session at :3
	LIVE session at :4
	LIVE session at :5
	LIVE session at :6
$ xpra stop all
Trying to stop 3 displays:
 :4, :5, :6
xpra initialization error:
 failed to connect to '/run/user/1000/xpra/sntentos-precision-t3620-5':
 [Errno 11] Resource temporarily unavailable
xpra initialization error:
 cannot find live server for display :4
$ xpra list
Found the following xpra sessions:
/run/user/1000/xpra:
	DEAD session at :3 (cleaned up)
	DEAD session at :4 (cleaned up)
	DEAD session at :5 (cleaned up)
	LIVE session at :6
$ connection timed out
timeout: server did not disconnect us
^C
$ xpra list
Found the following xpra sessions:
/run/user/1000/xpra:
	LIVE session at :6

[stdedos]

Then again, sometime later:

$ xpra stop all
No xpra sessions found
$ xpra list
Found the following xpra sessions:
/run/user/1000/xpra:
	DEAD session at :6 (cleaned up)

Seems to clean them up

[Antoine Martin]

xpra stop all just runs xpra stop NN on each display it finds.

So the only potential bugs that I see here are:

• after calling stop, the socket is left behind and xpra list cleans it up
• stopping :6 failed

Both work fine here and I don't see any steps to reproduce.
How do I start those sessions? What state are they in?
Can you call xpra info on the ones that are in the "wrong" state?

[stdedos]

They are sessions from #1997 ticket, so commands were:

$ xpra start ssh://sntentos@localhost --start-child=gnome-terminal --terminate-children=yes --exit-with-children=yes
$ xpra --ssh='ssh -x' start ssh://sntentos@localhost --start-child=gnome-terminal

However, brute-forcing the server with xpra start ... miraculously fixed the issue, and I cannot replicate the state of the sessions. I didn't think of collecting xpra-info preemptively (since I expected xpra stop all to collect them)

If there is anything useful, it's on the attachment. I included everything in there; you might want to check only :3-:6

---

Maybe this could fail a bit faster? Or assume ':\d+'?

$ xpra info 7
xpra initialization error:
 unknown format for display name: 7

[Antoine Martin]

Maybe this could fail a bit faster? Or assume ':\d+'?

See #2031

Closing, feel free to re-open if you can provide steps to reproduce.

[stdedos]

I do not have much more information to add on how to replicate it.
It seems to be related to whatever is causing the "new replication" of #2011

I am attaching (almost) the same diagnostics from #2011, and also the commands ran during this.

Note that clipboard sync was on

[stdedos]

More discrepancies (no xpra info, however :/)

$ xpra shadow -d encoding,bandwitdh :0
using systemd-run to wrap 'shadow' server command
'systemd-run' '--description' 'xpra-shadow' '--scope' '--user' '/usr/bin/xpra' 'shadow' '-d' 'encoding,bandwitdh' ':0' '--systemd-run=no'
Running scope as unit run-r3e763248a434434b9b82510637a3cdd1.scope.
Entering daemon mode; any further errors will be reported to:
  /run/user/1000/xpra/:0.log
(killed server from the client)
$ xpra list
Found the following xpra sessions:
/run/user/1000/xpra:
	DEAD session at :0 (cleaned up)
	LIVE session at :2
/run/xpra:
	LIVE session at :0
	LIVE session at :2
$ xpra list
Found the following xpra sessions:
/run/user/1000/xpra:
	LIVE session at :2
/run/xpra:
	LIVE session at :0
	LIVE session at :2
$ xpra stop :0
xpra initialization error:
 failed to connect to '/run/xpra/sntentos-precision-t3620-0':
 [Errno 11] Resource temporarily unavailable
$ xpra list
Found the following xpra sessions:
/run/user/1000/xpra:
	LIVE session at :2
/run/xpra:
	DEAD session at :0 (cleaned up)
	LIVE session at :2

[Antoine Martin]

More discrepancies (no xpra info, however :/)

No server log?

[stdedos]

Replying to Antoine Martin:

More discrepancies (no xpra info, however :/)

No server log?

No :/ I started killing it before of thinking to collect xpra info.

I assumed it would give the same as trac_1998_new-instance.zip/xpra-info-1.log:

$ xpra info :1
xpra initialization error:
 failed to connect to '/run/xpra/sntentos-precision-t3620-1':
 [Errno 11] Resource temporarily unavailable

$ xpra stop :1
xpra initialization error:
 cannot find live server for display :1

$ xpra info :1
xpra initialization error:
 cannot find live server for display :1

after having it killed from the client; hence I didn't try collecting.

---

You can see that the problem in:

$ xpra list
Found the following xpra sessions:
/run/user/1000/xpra:
	DEAD session at :0 (cleaned up)
	LIVE session at :2
/run/xpra:
	LIVE session at :0
	LIVE session at :2

is not in the:

Found the following xpra sessions:
/run/user/1000/xpra:
	DEAD session at :0 (cleaned up)
	LIVE session at :2

but in the:

/run/xpra:
	LIVE session at :0
	LIVE session at :2

[stdedos]

I collected the xpra info as part of #2030.

Then "after a while", I killed the server. You can see the commands I ran, and their result. Server stayed idle for some time, but for all other purposes, I didn't do anything in-between.

[Antoine Martin]

In attachment/ticket/1998/trac_1998_new-instance.zip there is no sign of a server shutdown anywhere in the xpra-server.log.
In attachment/ticket/1998/trac_1998.zip there are some shutdown messages in xpra-server-0_2030-2.log:

2018-11-16 01:40:46,320 Shutting down in response to client request
2018-11-16 01:40:46,320 Disconnecting client Protocol(unix-domain socket:/run/user/1000/xpra/sntentos-precision-t3620-0):
2018-11-16 01:40:46,320  server shutdown
(..)
2018-11-16 01:40:46,348 xpra client 1 disconnected.

But nothing about closing sockets, I would expect to see something like this - which is what I get, even on Ubuntu 16.04:

closing tcp socket 0.0.0.0:10000
removing socket /run/user/1000/xpra/desktop-1
removing socket /run/xpra/desktop-1

Can you reproduce this on a server started with -d util?
You should see something like this:

2018-11-16 20:48:25,504 cleanups=[
    <function cleanup_tcp_socket at 0x7fe3b671f500>,
    <function close_gtk_display at 0x7fe3b6c7fb18>,
    <function cleanup_socket at 0x7fe3b7a8ede8>,
    <function cleanup_socket at 0x7fe3b7a8ed70>
    ]

Maybe --systemd-run=no helps?
Or maybe you need to start the session via ssh to trigger this bug?

[stdedos]

I randomly noticed xpra executables staying around:

$ ps -eo pid,lstart,cmd | grep -P '(^\s+P|xpra)'
  PID                  STARTED CMD
12529 Mon Nov 19 11:56:43 2018 /usr/bin/python2 /usr/bin/xpra proxy :14500 --daemon=no --bind-tcp=0.0.0.0:14500 --tcp-auth=sys --ssl-cert=/etc/xpra/ssl-cert.pem --ssl=on --bind=/run/xpra/system --auth=peercred --socket-dirs=/run/xpra --socket-permissions=666 --log-dir=/var/log --pidfile=/run/xpra.pid --debug=
15310 Mon Nov 19 11:59:11 2018 grep --color=auto -P (^\s+PID|xpra)
16253 Fri Nov 16 01:38:45 2018 /usr/bin/python2 /usr/bin/xpra -d encoding,bandwidth,stats,keyboard shadow :0 --systemd-run=no
19273 Thu Nov 15 01:18:57 2018 /usr/bin/python /usr/bin/xpra shadow -d encoding,bandwitdh :0 --systemd-run=no
19606 Thu Nov 15 01:19:15 2018 sh -c xpra initenv;if [ -x $XDG_RUNTIME_DIR/xpra/run-xpra ]; then $XDG_RUNTIME_DIR/xpra/run-xpra _proxy ":0";elif which "xpra" > /dev/null 2>&1; then xpra _proxy ":0";elif [ -x /usr/local/bin/xpra ]; then /usr/local/bin/xpra _proxy ":0";elif [ -x ~/.xpra/run-xpra ]; then ~/.xpra/run-xpra _proxy ":0";else echo "no run-xpra command found"; exit 1; fi
19634 Thu Nov 15 01:19:16 2018 /usr/bin/python /usr/bin/xpra _proxy :0
23490 Fri Nov 16 01:20:29 2018 /usr/bin/python2 /usr/bin/xpra -d encoding,bandwidth,stats,keyboard shadow :0 --systemd-run=no
29441 Thu Nov 15 01:29:55 2018 sh -c xpra initenv;if [ -x $XDG_RUNTIME_DIR/xpra/run-xpra ]; then $XDG_RUNTIME_DIR/xpra/run-xpra _proxy ":0";elif which "xpra" > /dev/null 2>&1; then xpra _proxy ":0";elif [ -x /usr/local/bin/xpra ]; then /usr/local/bin/xpra _proxy ":0";elif [ -x ~/.xpra/run-xpra ]; then ~/.xpra/run-xpra _proxy ":0";else echo "no run-xpra command found"; exit 1; fi
29459 Thu Nov 15 01:29:55 2018 /usr/bin/python /usr/bin/xpra _proxy :0
29674 Tue Nov 13 12:25:02 2018 /usr/bin/python /usr/bin/xpra attach :999 --systemd-run=no
30353 Wed Nov 14 23:53:38 2018 sh -c xpra initenv;if [ -x $XDG_RUNTIME_DIR/xpra/run-xpra ]; then $XDG_RUNTIME_DIR/xpra/run-xpra _proxy_start "--start=gnome-terminal" "--speaker=off" "--opengl=no" "--webcam=no";elif which "xpra" > /dev/null 2>&1; then xpra _proxy_start "--start=gnome-terminal" "--speaker=off" "--opengl=no" "--webcam=no";elif [ -x /usr/local/bin/xpra ]; then /usr/local/bin/xpra _proxy_start "--start=gnome-terminal" "--speaker=off" "--opengl=no" "--webcam=no";elif [ -x ~/.xpra/run-xpra ]; then ~/.xpra/run-xpra _proxy_start "--start=gnome-terminal" "--speaker=off" "--opengl=no" "--webcam=no";else echo "no run-xpra command found"; exit 1; fi
30370 Wed Nov 14 23:53:39 2018 /usr/bin/python /usr/bin/xpra _proxy_start --start=gnome-terminal --speaker=off --opengl=no --webcam=no
30406 Wed Nov 14 23:53:40 2018 /usr/bin/python /usr/bin/xpra start --start=gnome-terminal --speaker=off --opengl=no --env=XPRA_PROXY_START_UUID=8e5834378f9b4c04b6409fdf453829c1 --daemon=yes --systemd-run=no --displayfd=4
30464 Wed Nov 14 23:53:40 2018 pulseaudio --start -n --daemonize=false --system=false --exit-idle-time=-1 --load=module-suspend-on-idle --load=module-null-sink sink_name="Xpra-Speaker" sink_properties=device.description="Xpra\ Speaker" --load=module-null-sink sink_name="Xpra-Microphone" sink_properties=device.description="Xpra\ Microphone" --load=module-native-protocol-unix socket=/run/user/1000/xpra/pulse-1/pulse/native --load=module-dbus-protocol --load=module-x11-publish --log-level=2 --log-target=stderr
30675 Thu Nov 15 01:31:07 2018 sh -c xpra initenv;if [ -x $XDG_RUNTIME_DIR/xpra/run-xpra ]; then $XDG_RUNTIME_DIR/xpra/run-xpra _proxy ":0";elif which "xpra" > /dev/null 2>&1; then xpra _proxy ":0";elif [ -x /usr/local/bin/xpra ]; then /usr/local/bin/xpra _proxy ":0";elif [ -x ~/.xpra/run-xpra ]; then ~/.xpra/run-xpra _proxy ":0";else echo "no run-xpra command found"; exit 1; fi
30705 Thu Nov 15 01:31:08 2018 /usr/bin/python /usr/bin/xpra _proxy :0
$ xpra list
No xpra sessions found

Do I / can I get anything from them, or should I just kill them?

[Antoine Martin]

We want to see the server log file(s) which should contain either the clean closing statements as per comment:10, or an error during cleanup, or maybe your sessions just got cleaned too quickly as per comment:8 when they were still live.

[Antoine Martin]

Not heard back, closing.

[stdedos]

Doesn't seem to have replicated at all

[migration script]

this ticket has been moved to: ​https://github.com/Xpra-org/xpra/issues/1998