Xpra: Ticket #1141: Sound loop is created when both microphone and speaker forwarding is enabled.

Reference

https://www.xpra.org/trac/ticket/1110

I tried to test the latest changes related to microphone forwarding and speaker forwarding from the latest source of v0.16.x.

I took the 4 files

client/ui_client_base.py server/source.py sound/gstreamer.py sound/wrapper.py

And, copied it to appropriate folder in my xpra folder at following location. My current xpra version is 0.16.2

/usr/lib/python2.7/dist-packages/xpra

With the current changes, automatically the default input device get's selected in the recording tab of pavucontrol.

But, the sound loop issue is still there.

Steps to reproduce.

  1. Open an xterm from the remote machine.
  2. Enable both microphone and speaker forwarding

Sound loop starts.

Let me know, if any further information is required.



Mon, 07 Mar 2016 16:29:42 GMT - Kundan: version changed


Sat, 12 Mar 2016 18:02:34 GMT - Kundan:

Hi Antoine,

It seems, I got what is causing the sound loop.

Here, is my explanation. It seems the culprit is the dedicated pulseaudio server which is started along with the xpra session. In our case it is started by the winswitch in virt/server_util_base.py file.

The command to start the dedicated pulseaudio server is as follows:-

/usr/bin/pulseaudio --start -vvvv --disable-shm=true --daemonize=false --use-pid-file=false --system=false --exit-idle-time=-1 -n --load=module-suspend-on-idle --load=module-null-sink --load=module-native-protocol-unix socket=/tmp/pulse-60/native

The above command creates two devices one input and one output device for the xpra and the application to interact. The input device is named as null.monitor and the output device is null.

These two devices works perfectly when, any one of the stream either playback stream or the recording stream is working.

Let's take the example of playback stream.

In playback stream, The application, writes to null output device. And, xpra, does a recording from null.monitor device. Whatever, xpra records, it sends back to the client system for further processing.

In Recording stream,

xpra plays on the null output device on the server, whatever recorded sound data it has brought from the client. The application, then reads from the null.monitor device. And everything works perfectly fine.

But, when both microphone forwarding and speaker forwarding is enabled, it seems these two devices null and null.monitor, doesn't seems to be enough to handle the streams, for both xpra and application independently. We need at least 4 devices to do this work.

Now, let's see, what is happening when both microphone forwarding and speaker forwarding is enabled with the two devices, null and null.monitor.

Due to recording stream, xpra would play on null device (on server), whatever recorded audio data it has brought from the client machine. And, due to speaker forwarding xpra would record from the null.monitor and send it to the client machine.

Surprisingly, a sound loop has formed. Now, the player xpra (for microphone forwarding on server) is doing the job of the application for the recorder xpra (for speaker forwarding on server).

In order to solve this issue, I have modified the pulseaudio server command as follows. The additional --load=module-null-sink creates two more devices.

/usr/bin/pulseaudio --start -vvvv --disable-shm=true --daemonize=false --use-pid-file=false --system=false --exit-idle-time=-1 -n --load=module-suspend-on-idle --load=module-null-sink --load=module-null-sink --load=module-native-protocol-unix socket=/tmp/pulse-60/native

After this, I have done a small change in the gstreamer_util.py. This has solved my issue of sound loop.

Now, for application, I have null.2 as the output device and null.monitor as the input device.

I am attaching the diff. But, I can't think of all possible scenarios. Please make appropriate changes, so that others may not face this issue.


Sat, 12 Mar 2016 18:07:02 GMT - Kundan: attachment set

gstreamer_util diff for avoiding the sound loop.


Sun, 13 Mar 2016 03:09:28 GMT - Antoine Martin: owner changed

Some questions:


Sun, 13 Mar 2016 19:24:39 GMT - Kundan:

Answer to first question, In my opinion, I don't think we need to change the name null.2 . We can still search the pulseaudio, and search for the second output device instead of the first one. That should bring null.2 as the name and the recording device will become null.2.monitor after appending monitor. Alternatively, we can search for the default output device, if we have changed the default output device through pactl.

Answer to the second question

Yes, we want client applications to output sound on the device we record from. And, record from the device we play at.

Therefore, we won't be able to use the default device's for xpra consumption. As, the default devices would be for application's consumption.

For example, we can change the default output device to null.2 and default input device to be null.monitor through pactl.

But, the issue, would be that, during recording stream, where we play on server, we would get this hint and start playing on null.2, which we don't want. We want it to play on null. So that application could record from null.monitor.

Therefore, we always need to avoid the default devices used by the application.


Mon, 14 Mar 2016 12:43:05 GMT - Antoine Martin:

Let me re-phrase that: I want to change the name of the device to something both more meaningful and less likely to change. Can you please look into this?

You are saying xpra should not use the default devices, but your patch only changes one case, not two. I think we do want to use the default device for capturing sound (speaker forwarding), just not for injecting sound (microphone forwarding). Is that right? How will the clients application know they need to capture from the badly named "null.2"? Aren't they going to record from "null" by default, which will not have any sound? I'm really not clear on this, and the device names aren't helping.


Mon, 14 Mar 2016 16:58:18 GMT - Kundan:

Hi Antoine,

I am getting your point, that the name null.2 seems to be vague. But it is created automatically by loading the module-null-sink. Therefore, if we load, module-null-sink two times in the pulseaudio command. We are guaranteed to get, following 4 devices. Two input devices and two Output devices.

Output device:- null null.2 Input device:- null.monitor null.2.monitor

And, the above 4 devices won't change. I understand that name should be more meaningful, But, this is the way in which module-null-sink was written. I am also not saying, it can't be done. But, it seems, It would need some more head scratching, some more fighting, and don't know even after that we would achieve the result we are looking for or not. But, if we keep the task of giving a meaningful name aside. It doesn't mean we can't solve our issue.

Even earlier, we were using at least the following two, one input and one output device. As, we use to load module-null-sink one time in pulseaudio command.

Output device null Input device null.monitor

Now, about your second question.

Yes, you are right, we need to use the default device for capturing sound (speaker forwarding) but not for microphone forwarding.

Now, your second part of your second question.

But, before that I should talk about the streamline process, in which we should fix this issue. So,

step 1: Select which device we should use for application to output sound, or play, Similarly, which device we should use for application for inputting sound, or record.

We have following two sets to choose:-

  1. Output device = null, Input device = null.2.monitor
  2. Output device = null.2 Input device = null.monitor

I would prefer the second one.

Now, we would make the selected set, the default devices for the applications to interact through pactl.

pactl -s /tmp/pulse-60/native set-default-sink null.2
pactl -s /tmp/pulse-60/native set-default-source null.monitor

Now, our whole xpra code should be twisted to make this possible and happy. Because, now our client application will automatically choose null.2 as the output device and null.monitor as the recording device.

Now, I should speak about about what changes, we need from the xpra perspective.

  1. get_default_sink in pulseaudio_util.py which get's imported from one of pulseaudio_pactl_util.py or pulseaudio_palib_util.py. May be you could throw more light on it, that which implementation get's called.

My expectation was that, after changing through pactl command, it would return null.2 as the default sink. But, it doesn't. This is the first place, which we need to fix.

  1. start_receiving_sound, in wrapper.py. We need to pass null as the second argument to sink_subprocess_wrapper. This should come from the second argument of start_receiving_sound, and should be populated from server/source.py. I haven't tested it, so don't know it would work or not. But, we have to make it work.

Also, if we need to avoid hard coded value, we need to scan all the output devices and look for the first device, which will be null, which is sure.

The command to do this may look something like this.

pactl -s /tmp/pulse-60/native list sinks | grep  Name | awk '{print $2}' | head -1

This ends, the fix which we need to make from the xpra side.

Now, I would explain, why my fix works for me.

I haven't changed the default devices through pactl. Thus, the default devices remain. Output device null Input device null.monitor

During recording, at the server(Speaker forwarding), I changed the recording device to null.2.monitor, through my change.

And, During playing at the server (microphone forwarding), automatically null device is used as the default device.

Since, I know, what is happening behind the scenes. When I made my application, I recorded from the null.monitor which is the default one (so no issues) And, Played on null.2 (Which a normal application won't do, as it can't look behind the scenes)

And, everything worked fine for me. But, definitely. It is not right for a dumb user. We need to fix it, as I mentioned above. Also, I had made this change and attached it here, so that I could explain you what is happening.

Now, at the last,

Since application, need not know whether it is playing to null or null.2 . Recording from null.monitor or null.2.monitor. After setting the default devices, it would be automatically taken care. I don't think, we need to really worry about these names, null, null.2. But, if you still think, we need to have a valid name. We can work on that. (But, it would be an enhancement, Not a defect). By following above at least we can unblock this issue.


Mon, 14 Mar 2016 17:48:10 GMT - Kundan: owner changed


Tue, 15 Mar 2016 02:38:11 GMT - Kundan:

One more thing to tell.

The logic to select the default sink needs to be used in get_default_sink is also fairly simple.

It could be like following:-

pactl -s /tmp/pulse-60/native info | grep "Default Sink" | awk '{print $3}'

But, in our implementation of get_default_sink in pulseaudio_pactl_util.py, it seems we have used the stat subcommand to pactl, which doesn't report anything like Default Sink. An output from my machine of both pactl info and pactl stat.

pactl info output

pactl -s /tmp/pulse-60/native info
Server String: /tmp/pulse-60/native
Library Protocol Version: 30
Server Protocol Version: 30
Is Local: yes
Client Index: 8
Tile Size: 65472
User Name: prashant
Host Name: prashant-VirtualBox
Server Name: pulseaudio
Server Version: 6.0
Default Sample Specification: s16le 2ch 44100Hz
Default Channel Map: front-left,front-right
Default Sink: null
Default Source: null.monitor
Cookie: 2f6b:170d

pactl stat output

pactl -s /tmp/pulse-60/native stat
Currently in use: 1 blocks containing 63.9 KiB bytes total.
Allocated during whole lifetime: 1 blocks containing 63.9 KiB bytes total.
Sample cache size: 0 B

Tue, 15 Mar 2016 06:46:16 GMT - Antoine Martin: owner changed

I am getting your point, that the name null.2 seems to be vague. But it is created automatically by loading the module-null-sink.


pulseaudio --start -n --daemonize=false --system=false --exit-idle-time=-1 \
    --load=module-suspend-on-idle \
    "--load=module-null-sink sink_name=test1" \
    --load=module-native-protocol-unix \
    --load=module-dbus-protocol \
    --log-level=2 --log-target=stderr

This creates a sink named "test1". Now that you know how to do it, please look into naming those devices so that we can make it clearer which one is used for what, then I'll try to write the code to look them up. Preferably without spawning a shell, pactl, grep and awk every time.


Wed, 16 Mar 2016 16:56:45 GMT - Kundan:

Hi Antoine,

I tried to do this.

/usr/bin/pulseaudio --start -vvvv --disable-shm=true --daemonize=false --use-pid-file=false --system=false --exit-idle-time=-1 -n --load=module-suspend-on-idle --load=module-null-sink sink_name="xpra_output_private" --load=module-null-sink sink_name="xpra_output_device" --load=module-native-protocol-unix socket=/tmp/pulse-60/native

I wanted to change the description, and set the default devices as well. I tried but could not do it, from the command line of pulseaudio. It seems it is posible to do this from default.pa or pacmd.

But, I failed to find a way to specify a customized default.pa to the pulseaudio command, or specify a customized server name to the pacmd command.

I am writing what i tried to specify, may be that would help.

sink_name=xpra_private_output sink_properties=device.description="Xpra Private Output Device"
source_name=xpra_private_output.monitor source_properties=device.description="Xpra Input Device"
sink_name=xpra_output_device sink_properties=device.description=\"Xpra Output Device\"
source_name=xpra_output_device.monitor source_properties=device.description="Xpra Private Input Device"

I was thinking to put following name to the monitor devices, But seems the module doesn't accept source_name field. It simply adds, monitor in the last of provided sink_name.

xpra_private_output.monitor = xpra_input_device xpra_output_device.monitor = xpra_input_private

The default devices which I am thinking to set is as follows:-

set-default-sink xpra_output_device set-default-source xpra_private_output.monitor


Thu, 17 Mar 2016 01:05:27 GMT - Kundan: owner changed


Fri, 18 Mar 2016 15:04:00 GMT - Antoine Martin: owner changed

Sorry about the delay.

The "sink_name" and "sink_properties" can be set but you need to quote and escape them, I have edited comment:9 to include the missing quotes. Here is another example:

pulseaudio --start -n --daemonize=false --system=false --exit-idle-time=-1 \
    --load=module-suspend-on-idle \
    --load=module-native-protocol-unix \
    --load=module-dbus-protocol \
    --log-level=2 --log-target=stderr \
    '--load=module-null-sink sink_name="Xpra-Output" sink_properties=device.description="Xpra\ Private\ Output"'
    '--load=module-null-source source_name="Xpra-Input"'

I didn't manage to set the description on the source module for some reason, but I didn't try very hard. It seems that the sink_name accepts spaces but converts them to underscore.. (so let's not use spaces at all) The "monitor" devices are derived from the module they monitor, so it makes sense that the source_name cannot be set.


As for the default devices, doesn't it just pick the first one? In which case we can just change the ordering? If that's the case, we should be able to just generate the correct command line, then just teach xpra about the device names it should look for? (sounds almost too easy to be true)


Fri, 18 Mar 2016 16:00:51 GMT - Antoine Martin:

Please take a look at r12164, it creates named devices by default and uses that string to look them up when choosing what device to play back on / record from. I believe this gets us pretty close.

This works fine for playback, but I guess this still creates a sound loop. Please help me clarify why we would need 4 devices (as seen in comment:10) as I am still totally confused by this.

Here's what makes sense to me:


Sat, 19 Mar 2016 14:21:40 GMT - Kundan:

Hi Antoine,

We, need another output device, for us, ie. xpra to output sound, which we do during microphone forwarding, and the application should record sound from it's monitor device. This seems to be the way through which application can get the recorded sound.

Thus, we need one set of input and ouput device for speaker forwarding and other set for microphone forwarding.

But, why we need it.

We need it because, if xpra would also use the same output device which is used by application. The speaker forwarding code would not be able to differentiate between, what sound is coming from application or it is coming from the xpra itself for microphone forwarding. It would just record from the it's monitor device and send it back to the client. The same sound data which was sent to the server by microphone forwarding.

This will create the sound loop.

Therefore, what I think that loading two times module module-null-sink is enough. It would create those 4 devices. Two sink's and two sources(monitors).

But, if we would load module module-null-source, it would create only one source without any monitor device. Thus is not helpful to us.

I have tried to create the pulseaudio command with your recommendation, which seems to be working.

/usr/bin/pulseaudio --start -vvvv --disable-shm=true --daemonize=false \
    --use-pid-file=false --system=false --exit-idle-time=-1 -n \
    --load=module-suspend-on-idle \
    --load=module-null-sink sink_name="xpra_output_private" sink_properties=device.description="Xpra\ Private\ Output\ Device" \
    --load=module-null-sink sink_name="xpra_output_device" sink_properties=device.description="Xpra\ Output\ Device" \
    --load=module-native-protocol-unix socket=/tmp/pulse-61/native

After this, we need to do the following.

pactl -s /tmp/pulse-61/native set-default-sink xpra_output_device
pactl -s /tmp/pulse-61/native set-default-source xpra_output_private.monitor

We, (xpra) has to record sound from following input device for speaker forwarding.

xpra_output_device.monitor.

We, (xpra) has to play sound on following output device for microphone forwarding.

xpra_output_private This should fix our issue of sound loop.

Some more information about this sound loop is on comment:2.


Sun, 20 Mar 2016 14:43:22 GMT - Antoine Martin:

Well, that was harder than I thought! r12190 does what is required - I think. See commit message.


For the record, this is what I get when I start the client with just speaker:

And with just the microphone:


The things I don't like / would like to do better (but may have to wait for another release):

etc..


@kundanvp: does this work for you?


Mon, 21 Mar 2016 16:01:42 GMT - Kundan:

Hi Antoine,

It would be hard for me to test on the trunk, as I am currently working on v0.16.2. Is there any plan to port these changes to lower branches?


Tue, 22 Mar 2016 03:19:43 GMT - Antoine Martin:

This is how things work:

This change is big and changes the default config file. I don't think it will ever be backported.

As for testing, why don't you just test from trunk? Building from source is documented on the wiki: wiki/Building, and there are beta packages for most distros with this change included.


Wed, 23 Mar 2016 07:43:20 GMT - Kundan:

I tried to test the changes with the latest beta packages on Ubuntu 15.10. Xpra version is v0.17.0. The device selection seems to be proper.

Inorder to make it work. I had to change the pulseaudio command of winswitch. The changes are following:-

/usr/bin/pulseaudio --start -vvvv --disable-shm=true --daemonize=false --use-pid-file=false --system=false --exit-idle-time=-1 -n --load=module-suspend-on-idle --load=module-null-sink sink_name="Xpra-Speaker" sink_properties=device.description="Xpra\ Speaker" --load=module-null-sink sink_name="Xpra-Microphone" sink_properties=device.description="Xpra\ Microphone" --load=module-native-protocol-unix socket=/tmp/pulse-61/native

I had to also, change auth=file in winswitch so that, xpra could be started properly. Seems this is a requirement of xpra 0.17.0. And, these things needs to be rectified from the winswitch side.

But, the recording stream seems to be faulted. It is not able to record anything.

In the xpra logs. I found following thing on server logs which seems to be fishy.

 using pulseaudio device:
2016-03-23 12:21:37,060  'Built-in Audio Analog Stereo'
2016-03-23 12:21:38,671 the sound source process has failed to start, stopping it
2016-03-23 12:21:39,877 sound output using pulseaudio device:^[[0m
2016-03-23 12:21:39,877 sound output  'Xpra Microphone'^[[0m

For the playback stream, things seems to work properly.

2016-03-23 12:20:07,004 Connection lost
2016-03-23 12:20:19,191 using pulseaudio device:
2016-03-23 12:20:19,191  'Monitor of Xpra Speaker'
2016-03-23 12:21:03,004 New unix-domain connection received on /home/prashant/.xpra/prashant-VirtualBox-61
2016-03-23 12:21:03,005 New unix-domain connection received on /home/prashant/.xpra/prashant-VirtualBox-61
2016-03-23 12:21:03,063 Connection lost

Please look into the issue. And, let me know if any more information is required.


Wed, 23 Mar 2016 14:46:14 GMT - Antoine Martin:

I had to change the pulseaudio command of winswitch. The changes are following:-


Please always use the patch format for sending changes. I believe I have done what you requested in http://winswitch.org/trac/changeset/5344.


I had to also, change auth=file in winswitch so that, xpra could be started properly. Seems this is a requirement of xpra 0.17.0.


It is a requirement, for more details see: r11766 and r8690. Support in winswitch should be fixed in http://winswitch.org/trac/changeset/5343.


But, the recording stream seems to be faulted. It is not able to record anything. .. Please look into the issue. And, let me know if any more information is required.


I don't see anything there, please post the -d sound or -d gstreamer debug output. And please confirm if this problem is with xpra on its own or if the problem only occurs when launched from winswitch.


Wed, 23 Mar 2016 16:20:45 GMT - Kundan:

Hi Antoine,

The changes, which you have done in winswitch, is exactly, what I did in winswitch. So, I am happy for that. But, next time I will attach the diff. I was hesitant, as I thought posting the diff for winswitch here, will be appropriate or not. But, seems there is no issue.

Regarding the issue with microphone forwarding. I am attaching a screenshot of pavucontrol output of both client and server system (through xterm).

The issue is that, earlier when I was testing with v0.16.2. When, the xpra application in the client's pavucontrol's recording tab, blinked. The similar blink I noticed, in the xpra application in server's pavucontrol's playback tab.

But, here, in v0.17.0. I am not able to see the matching blinks in server's pavucontrol. However, the xpra application in client system is blinking properly, means it is recording the data properly.

Since, in the server's pavucontrol's playback tab, the xpra application was not blinking. I suspected whether recording is happening or not.

I tried to record through a small application, in the xterm. But, my suspicion seems, to be right. Recording is not happening.

It seems, somewhere, we are not able to send the appropriate sound data to the server. So, I suspect whether the pipeline between client to server is working properly or not.

I don't think, running xpra directly, or running through winswitch will affect that. But, I will collect that details. About the log output, also. I will collect and attach it.

But, I thought to make the issue clearer first. Please let me know, if there are any queries.


Wed, 23 Mar 2016 16:21:53 GMT - Kundan: attachment set

Screenshot showing status of pavucontrol at both server and client.


Wed, 23 Mar 2016 18:44:53 GMT - Kundan:

Hi Antoine,

Attaching the logs generated during attaching the xterm session.

Command used was:-

/usr/bin/python /usr/bin/xpra --title=@title@ --session-name=XTerm --passrd-file=/home/nishant/.winswitch/client/sessions/946ffc372f77467480ecd3580c9a6613/session.pass -z1 --window-icon=/home/nishant/.winswitch/client/sessions/946ffc372f77467480ecd3580c9a6613/session.icon -d sound attach tcp:127.0.0.1:12462 >attach.log 2>&1

Also, attaching the xpra.log which get's generated on the server in .winswitch/server/sessions/61.


Wed, 23 Mar 2016 18:52:08 GMT - Kundan: attachment set

log generated during attach of the xterm session.


Wed, 23 Mar 2016 18:57:35 GMT - Kundan: attachment set

xpra.log which get's generated at ~/.winswitch/server/sessions/61/


Fri, 25 Mar 2016 07:00:31 GMT - Kundan:

Hi Antoine,

I have generated new logs, on the server through, enabling debugging at run tine for the appropriate session of xpra for xterm, which was already started by winswitch. I have used following command.

xpra control :61 <password file> debug enable sound

Sorry, I was not able to start xpra without winswitch. My default pulseaudio server was not getting killed. It was respawning itself. And, xpra was telling that for using the sound support either I should start it in a clean environment without pulseaudio or dedicated user. Please let me know, How it can be done.

Attaching the sound debug logs which was generated at ~/.winswitch/server/sessions/61/ for only microphone forwarding and for a short duration.


Fri, 25 Mar 2016 07:01:55 GMT - Kundan: attachment set

xpra log which was generated at the server after enabling sound debugging.


Fri, 25 Mar 2016 14:42:31 GMT - Antoine Martin:

Sorry, I was not able to start xpra without winswitch.


Err?? Just stop winswitch, then "xpra start". What's the problem?


My default pulseaudio server was not getting killed. It was respawning itself. And, xpra was telling that for using the sound support either I should start it in a clean environment without pulseaudio or dedicated user. Please let me know, How it can be done.


The easiest way is to use a different user, one without an existing desktop session so it won't have a pulseaudio server running already. (ie: "sudo su - test")

What am I looking for in this log you attached?


Fri, 25 Mar 2016 20:37:59 GMT - Kundan:

Hi Antoine,

I tried with only xpra without winswitch. Issue is there as well. I am also not clear of what log would be needed to track this issue. Issue is that the sound data which is getting played at the server through xpra _sound_play is not what was recorded from the client. It seems xpra _sound_play at the server for microphone forwarding is playing some null sound data which is not the actual sound data which was recorded from the client.

Please help how we should proceed on this issue. As, it is breaking the microphone forwarding in v0.17.0.

Note: I am not talking about the device selection, the device selection seems to be proper.

I am attaching a short video to make my point clear. Issue is easily reproducible. Steps are following:-

  1. Start an xpra session with xterm as the command through following command.
xpra --no-daemon start ssh:192.168.20.30:67 --start=xterm
  1. Attach that session on the other machine through following command.
xpra attach ssh:192.168.20.30:67
  1. Open pavucontrol on the machine where session was attached.
  1. Open a pavucontrol in the xterm through pavucontrol &.
  1. Notice the vibrations in the bar below silence. You will notice that in the client's pavucontrol where session was attached, is constantly blinking and changing. But, the server's pavucontrol which was opened through xterm is just remaining mute, is not blinking.

Please let me know, what more information is needed to track this issue.


Fri, 25 Mar 2016 20:41:04 GMT - Kundan: attachment set

A short video to show, the difference between client and server's pavucontrol.


Thu, 31 Mar 2016 15:15:50 GMT - Antoine Martin: attachment set

microphone recording shown through pavucontrol at each end


Thu, 31 Mar 2016 15:24:52 GMT - Antoine Martin:

How I tested:

The server log shows:

using pulseaudio device:
 'Monitor of Xpra Speaker'
sound output using pulseaudio device:
sound output  'Xpra Microphone'

I recorded a test stream using arecord test.wav within the session, clapped my hands. The only microphone is connected to the client system. Then played it back and could hear my clapping.

The sound was also visible through pavucontrol: microphone recording shown through pavucontrol at each end

So whatever you did... is different. I have no idea what. Maybe the stream is not coming through for whatever reason? (though I would expect the sound process to exit with an error in that case..) Maybe post a brief -d sound,gstreamer output of both client and server, doing just microphone forwarding and nothing else. (a few seconds is enough)


Fri, 01 Apr 2016 17:42:38 GMT - Kundan:

Hi Antoine,

I repeated the exact steps, which you mentioned. But, I am still seeing this issue. This issue doesn't reproduce on my machine on v0.16.3.

I am attaching the server logs and client logs with both sound and gstreamer debugging enabled.

Command for starting the xterm session.

xpra start :100  --start-child=xterm -d sound,gstreamer --no-daemon --bind-tcp=192.168.56.101:10000 >server_out.log 2>&1

Command for attaching the session.

xpra attach tcp:192.168.56.101:10000 -d sound,gstreamer --no-daemon --microphone=on >client_out.log 2>&1

The microphone didn't started 1st time. I had to try 2,3 times to enable microphone forwarding so that xpra start showing in the client and server's pavucontrol.


Fri, 01 Apr 2016 17:44:58 GMT - Kundan: attachment set

server xpra log.


Fri, 01 Apr 2016 17:45:34 GMT - Kundan: attachment set

server xpra log.


Fri, 01 Apr 2016 18:00:39 GMT - Kundan: attachment set

client xpra log.


Sat, 02 Apr 2016 07:31:09 GMT - Antoine Martin:

Your debug logs include both microphone and speaker output, which makes it hard to know what is what. And why are there 2 server logs? What is the difference? Please don't use tar.gz for single files, if you really must compress them, then use plain gzip (or even better: xz). Have you tried different codecs?


From the first server log, I see:

22:49:31,214 sound output add_data(0 bytes, {'codec': 'opus+gdp', 'start-of-stream': True}) queue_state=starting
22:49:31,216 sound output pushed     0 bytes, new buffer level:   0ms, queue state=starting
22:49:31,250 info_update: {'volume': 100, 'pipeline': 'appsrc do-timestamp=1 name=src emit-signals=0 block=0 is-live=0 stream-type=0 format=4 \
    ! gdpdepay ! opusdec ! audioconvert ! audioresample ! volume name=volume volume=1.0 \
    ! queue name=queue min-threshold-time=0 max-size-buffers=0 max-size-bytes=0 max-size-time=450000000 leaky=2 \
    ! pulsesink client-name=Xpra async=True device=Xpra-Microphone qos=True sync=False', 'codec_description': '', \
    'bytes': 0, 'queue': {'cur': 0, 'min': 0, 'max': 450, 'pct': 0, 'overruns': 0, 'state': 'starting', 'underruns': 0}, \
    'state': 'ready', 'codec': 'opus+gdp', 'buffers': 1}
22:49:33,061 verify_started() ... codec=opus+gdp
stop_sending_sound() sound source=source_subprocess_wrapper(None)
cleanup() sending cleanup request to sound source
sound_data(opus+gdp, 0, {'end-of-stream': True}, ()) sound sink=sink_subprocess_wrapper(4359)

So the server is receiving an "end-of-stream" from the client and stopping the sink.

When it does start properly a bit later on, I do see a constant stream of messages like this one:

sound output pushed    65 bytes, new buffer level: 280ms, queue state=pushing

Which would correspond to an empty / silence sound buffer.


From the client log, I see:

sound source stream status: <enum GST_TASK_STARTED of type GstTaskState>
...
new_stream(opus+gdp)
sound source new preroll1: <GstSample at 0x7f6198016420>
sound source emit_buffer data=<type 'str'>, len=197, metadata={'duration': 0L, 'timestamp': -1}
sound source emit_buffer data=<type 'str'>, len=197, metadata={'duration': 0L, 'timestamp': -1}
sound source emit_buffer data=<type 'str'>, len=298, metadata={'duration': 0L, 'timestamp': -1}
sound source emit_buffer data=<type 'str'>, len=403, metadata={'duration': 0L, 'timestamp': -1}
sound source emit_buffer data=<type 'str'>, len=137, metadata={'duration': 0L, 'timestamp': -1}
sound source SoundPipeline.cleanup()
sound source SoundPipeline.stop() state=active
sound source stopping

So it is starting to capture sound data, then quickly exits. I believe this is because of this:

verify_started() process=<subprocess.Popen object at 0x7f05386b2c50>, last_info={}, codec=opus+gdp
the sound output process has failed to start, stopping it

The sound process takes too long to startup and we decide it has failed and stop it. r12304 increases the timeout to 3 seconds (and maybe we can make it higher still), you can change it with XPRA_SOUND_START_TIMEOUT=10000 xpra ... (for 10 seconds). Maybe this will help. Is it a slow system?

When it does start properly later on, I see:

verify_started() process=<subprocess.Popen object at 0x7f05386b2f50>, \
    last_info={'volume': 100, 'pipeline': 'pulsesrc device=alsa_input.pci-0000_00_05.0.analog-stereo name=src \
    ! queue name=queue min-threshold-time=0 max-size-buffers=0 max-size-bytes=0 max-size-time=50000000 leaky=2 ! volume name=volume volume=1.0 \
    ! opusenc bitrate-type=2 complexity=0 ! gdppay crc-payload=0 crc-header=0 \
    ! appsink name=sink emit-signals=true max-buffers=10 drop=true sync=false async=false qos=false', \
    'codec_description': '', 'pid': 4872, 'bytes': 5132, 'actual-buffer-time': 64000, 'queue': {'cur': 0}, 'state': 'active', 'codec': 'opus+gdp', \
    'actual-latency-time': 64000, 'time': 1459531346, 'buffers': 65}, codec=opus+gdp

Followed by the constant 65-bytes sound packets:

new_sound_buffer(source_subprocess_wrapper(4872), 65, {'duration': 20000000, 'timestamp': 1200000000, 'time': 1459531346979})

Which corresponds to the silence you are seeing. Is the pulsesrc device specified the right one? device=alsa_input.pci-0000_00_05.0.analog-stereo

Can you record from it using gst-launch to verify? Something like (based on your data above):

gst-launch-1.0 pulsesrc device=alsa_input.pci-0000_00_05.0.analog-stereo name=src \
    ! queue name=queue min-threshold-time=0 max-size-buffers=0 max-size-bytes=0 max-size-time=50000000 leaky=2 ! volume name=volume volume=1.0 \
    ! opusenc bitrate-type=2 complexity=0 ! gdppay crc-payload=0 crc-header=0 \
    ! filesink location=test-opus.gdp

Which you can then playback using:

gst-launch-1.0 filesrc location=./test-opus.gdp ! gdpdepay ! opusdec ! autoaudiosink

You may want to use a different sound codec+muxer combination if you want to be able to playback using a regular media player. (ie: mp3) Maybe also try removing the elements which aren't needed when testing from the command line: volume and queue.

FYI: you can identify the gstreamer pipeline used at any time using ps -ef | grep "_sound" or xpra info | grep _sound.


Sat, 02 Apr 2016 12:56:43 GMT - Kundan:

Hi Antoine,

I was able to record from the device alsa_input.pci-0000_00_05.0.analog-stereo.

I used the exact command which you mentioned for recording, and playing on the client. And, it seems to be working fine. I am able to hear my sound properly, and also, the pavucontrol was showing the application, and sound was visible through blinks below silence.

Regarding adding the server log two times, I will remember the preference next time. Actually, I thought to abort the upload as it was taking time. And, uploaded the compressed one next time. But, it seems, both uploads, got successful.

For, capturing both microphone forwarding and speaker forwarding. It's hard for me to comment as, I followed the exact same steps,and didn't enabled speaker forwarding. But, tried to enable microphone forwarding from the xpra menu, after it got failed. The only thing which I did in the last was to see the statistics through xpra menu on the client. May be, that would have shown the alsa_output device. If that is the case. I am sorry for that.


Sat, 02 Apr 2016 13:01:01 GMT - Kundan:

One more thing,

I had tried with one more codec, But that too opus. Only opus not with gdp, may be with ogg. But, it was also showing the same issue.


Sat, 02 Apr 2016 15:14:45 GMT - Kundan:

I tried with speex codec as well with the same steps as you mentioned, But the issue remained there.


Sat, 02 Apr 2016 17:34:17 GMT - Kundan:

Hi Antoine,

I have done a small change in src.py on the client. This has solved that 65 byte issue. But, it has some negative impact on the server. Therefore, the xpra application doesn't show in the pavucontrol of the server started from the xterm.

I know this change is not the final one. But, it can lead you in the direction of further investigation. I am attaching the change in diff format.


Sat, 02 Apr 2016 17:35:21 GMT - Kundan: attachment set

src.py temporary diff.


Sun, 03 Apr 2016 05:51:58 GMT - Antoine Martin:

But, it has some negative impact on the server. Therefore, the xpra application doesn't show in the pavucontrol of the server started from the xterm.


Why not? All this does is to activate a workaround for broken gi bindings. There is an easier way too: just set extract_dup = None #getattr(buf, "extract_dup", None).

This should have no effect on the pipeline or process, only on how we capture the sound data from the pipeline. Ubuntu 15.10 may not be supported by the 0.17 branch since it is going to be EOL-ed quite soon.


Sun, 03 Apr 2016 13:13:35 GMT - Kundan:

Hi Antoine,

I tried your suggested workaround as well, i.e

extract_dup = None #getattr(buf, "extract_dup", None)

Actually, it seems, there is something fishy, which is happening on the server side after this change. The state of the queue never changes to pushing. It's always starting.

sound output pushed   794 bytes, new buffer level:   0ms, queue state=starting

I could not go beyond push_buffer in sink.py on the server. And, thus was not able to figure out what might be causing this. Or, why the queue_pushing function didn't get called, which is setting the state as pushing.

I am attaching the server and client logs. This time I have followed following two commands, to start and attach the session, and later on try to enable only the microphone forwarding from the xpra menu. This way the speaker forwarding, doesn't trigger by itself, leading to cleaner logs.

xpra start :100  --start-child=xterm -d sound,gstreamer --no-daemon --bind-tcp=192.168.56.101:10000 >server_out1.log 2>&1
xpra attach tcp:192.168.56.101:10000 -d sound,gstreamer --no-daemon --microphone=off --speaker=off >client_out1.log 2>&1

Sun, 03 Apr 2016 13:14:55 GMT - Kundan: attachment set

server new xpra log.


Sun, 03 Apr 2016 13:19:45 GMT - Kundan: attachment set

client new xpra log.


Tue, 05 Apr 2016 13:07:11 GMT - Antoine Martin: owner changed

Actually, it seems, there is something fishy, which is happening on the server side after this change.


This change has exactly the same effect as your change in attachment/ticket/1141/src.py.diff. If you are seeing different behaviour then something else was changed since your previous attempt.

If the number of bytes in the sound packets varies, you can be pretty confident that you are forwarding something other than silence.

I am tempted to close this ticket, as things are working for me on the distributions we support. Feel free to open a new ticket for Ubuntu-specific issues, just don't expect me to spend too much time on it.. Sorry.


@afarr: over to you for testing, you can probably ignore comment:16 onwards.


Tue, 05 Apr 2016 18:51:39 GMT - Kundan:

Hi Antoine,

One last thing to add from my side, Actually, the issue of "state of the queue not changing to pushing", was coming with my earlier change as well. And, as, you mentioned, the change you suggested has the same effect as my change, Therefore, it was coming with the suggested change as well. (Thus, I termed it as a temporary fix.)

As, Ubuntu 15.10 is not on your radar for v0.17.0. I understand, there is not much value in focusing time there. I am also happy with v0.16.3 on Ubuntu 15.10, as things which I require is working there.

Thanks, to you, I learnt a lot of things here about xpra. And, I will be troubling you again, If I stuck somewhere.


Sun, 24 Jul 2016 12:06:51 GMT - Antoine Martin:

Someone seemed to have problems with this new code (closed needinfo): #1221.


Wed, 27 Jul 2016 18:36:56 GMT - J. Max Mena:

Testing with a Windows r13031 trunk (1.0) client against a r13101 trunk Fedora 23 Server:

Now that I can repro it (it's taken me a couple hours reading just to realize how easy it is, I thought it would be more complicated), I can't quite figure out what was done to get around it. I'll keep reading, but in the meantime, I'll attach what I mentioned earlier.


Wed, 27 Jul 2016 18:37:59 GMT - J. Max Mena: attachment set

pavucontrol on the input devices page


Wed, 27 Jul 2016 18:38:24 GMT - J. Max Mena: attachment set

pavucontrol on the output devices page


Wed, 27 Jul 2016 18:39:23 GMT - J. Max Mena: attachment set

re uploading, forgot to have the microphone loop running while getting output.


Tue, 25 Oct 2016 01:04:08 GMT - alas: owner changed

It looks like maxmylyn has some logs, but didn't pass this back to you.

I'd add that I tried also somewhat recently (win32 1.0 r13673 I think) and also encountered the sound loop.

Looking again at comments up to and including 15, I don't see any indications of special command line parameters to pass to specify sound output/monitor for xpra vs. other server applications... and while I found a file to add null2 to, if so inclined, I didn't find where the pulseaudio initialization command was set up, so I couldn't try that "solution" to compare the results.

I'll pass this to you to look at what we've posted, and maybe you'll have some new hints to try?


Wed, 26 Oct 2016 00:52:26 GMT - alas:

Tried a little more with 1.0 r14282 fedora 23 server and 1.0 r14155 osx client ... without headphones, and I still get the feedback sound loop very quickly - and I was just running with a pair of xterm start-childs (the sound of my coughing and typing was being picked up and re-broadcast back at me).

I was looking for the output server and client side (like you'd done with comment:15), with both microphone and speaker =on, using -d sound both sides... found what seems to match your description (though the character escaping syntax is apparently different on osx v. fedora...):

Client-side:

\'--load=module-null-sink sink_name="Xpra-Speaker" sink_properties=device.description="Xpra\\ Speaker"\' \'--load=module-null-sink sink_name="Xpra-Microphone" sink_properties=device.description="Xpra\\ Microphone"\'

Server-side:

2016-10-25 17:26:21,398 init_pulseaudio() pulseaudio=True, pulseaudio_command=pulseaudio --start -n --daemonize=false --system=false --exit-idle-time=-1 --load=module-suspend-on-idle '--load=module-null-sink sink_name="Xpra-Speaker" sink_properties=device.description="Xpra\ Speaker"' '--load=module-null-sink sink_name="Xpra-Microphone" sink_properties=device.description="Xpra\ Microphone"'

Fri, 14 Jul 2017 16:02:48 GMT - Antoine Martin: milestone changed

Someone with interested in Ubuntu can take a look at this. (that's not me)

See also #1266 and #1219


Sat, 22 Jul 2017 07:13:07 GMT - Kundan:

Hi Antoine,

I invested some time here. And, have successfully got the root cause of the issue. I am attaching a short diff, which fixes this issue. Please accommodate it. If someone would like to test it, after applying the fix, that would be also good. But, I have done my unit testing.


Sat, 22 Jul 2017 07:15:47 GMT - Kundan: attachment set

Sound Loop Fix.


Sat, 22 Jul 2017 13:30:11 GMT - Antoine Martin: owner changed

And, have successfully got the root cause of the issue.

How did you identify the problem? It looks to me like this only changes the default sink and source plugins used, and I then found a glaring bug, fixed in r16461.

Without this bug fix, your change could actually cause the sound pipeline to fail because it would use an invalid plugin. Please test again with these more correct changes.. you may find that the problem is still present if your "fix" relied on a broken pipeline.

You may play with the src and sink settings without needing to modify the code, using these env vars:

XPRA_SOUND_SRC=pulsesrc XPRA_SOUND_SINK=pulsesink

I am attaching a short diff, which fixes this issue.

The patch was incomplete, it needed to add the function to pulseaudio_none_util which is used on other platforms. Applied in r16460.


Sat, 22 Jul 2017 19:42:30 GMT - Kundan:

Hi Antoine,

Actually, the issue was that, the Xpra should play on the device Xpra Microphone in context of Server. But, it was choosing Xpra Speaker, since it is the default device. Somehow, the argument which we use to pass to pulsesink plugin i.e device="Xpra Microphone" was not working due to a bug in our code. And, the bug was, that get_default_sink was failing, which was visible in the logs. I dug further, and found that the reason was that get_pactl_server was failing to be accessed.

Since, I was in the context of get_default_sink not get_default_source, Therefore, I don't think the change r16461 would impact me?

But, I found another issue. And, the issue is that xpra _sound_play process doesn't start on the server with latest source. The reason is mysterious.

I am attaching a short diff, containing change at two places. The first place is just a log modification based upon your change r16461. And, second in sink.py which is the mysterious fix. The log before return, is not only printing some meaningful details, but it is stopping xpra _sound_play from failing. I was not able to explain, how it fixed the issue. Please help me understand.


Sat, 22 Jul 2017 19:44:14 GMT - Kundan: attachment set

mysterious fix before return in sink.py.


Sat, 22 Jul 2017 20:07:06 GMT - Antoine Martin:

The fist part of the patch that fixes the logging message to match the actual code has been applied in r16462. But the second part is just an extra log statement, I very much doubt that this fixes anything - please double check and try replacing it with time.sleep(0) or something similar. I can't see why an extra statement would make any difference here. If the pipeline has failed to setup, please specify the reason - which should show up as a log message. I don't see how the process can continue without setting up a valid pipeline.

Actually, the issue was that, the Xpra should play on the device Xpra Microphone in context of Server. But, it was choosing Xpra Speaker, since it is the default device. Somehow, the argument which we use to pass to pulsesink plugin i.e device="Xpra Microphone" was not working due to a bug in our code.

What bug? How did you ascertain this?

And, the bug was, that get_default_sink was failing, which was visible in the logs.

What message exactly?

But, I found another issue. And, the issue is that xpra _sound_play process doesn't start on the server with latest source. The reason is mysterious.

What is the error message? How did you find that out?


Sun, 23 Jul 2017 08:41:06 GMT - Kundan:

Hi Antoine,

The message was "get_default_sink() no pulsesink",which is coming from the exception section of get_default_sink function. It, seems to me. In this case, the code was trying autoaudiosink instead of pulsesink and without any custom option like device="Xpra Microphone". So, autoaudiosink chose the default device, which was Xpra Speaker. And, the pipeline didn't fail.

But, the change which we have done (i.e enabling get_pactl_server), can have a negative impact on the client side, which I noticed today. It is other than the issue which I mentioned as the mysterious issue, which I would discuss later.

The impact is there; because, now, client side code will try to search Xpra Microphone in the list of available devices, which it will never find.

What we require, is that, if start_receiving_sound is called from the client context, i.e during speaker forwarding, we should use the plugin, autoaudiosink which would play on the default device on the client system. But, when we are calling start_receiving_sound from the server context, i.e Microphone forwarding, we should use the plugin, pulsesink with device=Xpra Microphone, because Xpra Microphone is not a default device.

But, we in code, don't differentiate, between these two cases. Also, the plugin selection, happens much later in sink.py. Till, that we pass sink_type as None.

I am adding a small fix to show my point. The fix is Linux only.

The mysterious issue seems to be an intermittent issue. It occurs often, but not always. Sometimes, things work properly. It looks to me, that in the failure case, the sound_subprocess itself doesn't start, because even the output of first get_default_sink in sink.py doesn't show up. This shows up when things work properly. I am talking about following message.

sound output using pulseaudio device:
sound output 'Xpra Microphone'

Also, in the above case, in ps -eaf output, we are not able to see any xpra with _sound_play option. Which, means either it started and closed, or never started.

I would like to park the mysterious isssue for now. May be taken care of through some other ticket.


Sun, 23 Jul 2017 08:43:26 GMT - Kundan: attachment set

Fixing the client side of sink pipleine as well.


Sun, 23 Jul 2017 12:50:23 GMT - Antoine Martin:

Updates and fixes:

One remaining problem: I'm not seeing level changes for my virtual microphone with pavucontrol. (but recording is active and the packets flowing, weird)

@Kundan: does that work for you?


Mon, 24 Jul 2017 18:24:25 GMT - Kundan:

Yes Antoine, I tested it. The sound loop is not showing up now.

Thanks, for all the support.


Tue, 25 Jul 2017 07:10:03 GMT - Antoine Martin: status changed; resolution set

I think we can close this now. Feel free to re-open if the sound loop re-appears, or create new tickets for other issues.


Mon, 18 Sep 2017 04:49:12 GMT - Antoine Martin: milestone changed


Sat, 23 Jan 2021 05:16:07 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1141