Xpra: Ticket #1110: Microphone forwarding doesn't work properly.

Microphone forwarding seems to work, but even in a simple recording application which records the sound and puts in a file with pulse as the recording device. The sound output data from the file is blank. I mean to say file is not blank, but what it contains is not our recorded sound. It's just mute. However, The playback works with same pulse device. I have seen this issue, in Ubuntu 14.04, xpra version v0.15.10.

Please let me know, if I am doing it in wrong way.

Wed, 03 Feb 2016 18:37:31 GMT - Antoine Martin: owner changed

Please see wiki/ReportingBugs for guidelines, and wiki/Sound. There is not enough information in this ticket for me to investigate.

Thu, 04 Feb 2016 18:04:43 GMT - Kundan: attachment set

Thu, 04 Feb 2016 18:05:04 GMT - Kundan: attachment set

Thu, 04 Feb 2016 18:05:47 GMT - Kundan: attachment set

Thu, 04 Feb 2016 18:15:23 GMT - Antoine Martin:

Too many details missing: What xpra command lines did you use? what was the output of those commands, what device does it use? Mixer settings, etc.. (pavucontrol is your friend)

Thu, 04 Feb 2016 18:24:55 GMT - Kundan: owner changed

Hi Antoine, Sorry for being vague in my queries.

I have attached three folders for testing the sound thing. I tested it on Ubuntu 14.04.3, with winswitch version 0.12.21, xpra version v0.15.10.

The folders, audioInputTest, and audioOutputTest contains C++ program compilable over Qt5.5.1. The third folder contains the compiled copy.

The steps for reproduction are as follows:-

  1. I complied the audioInputTest and audioOutputTest and created the executable audioInputTest and audioOutputTest.
  1. I opened a remote Terminal through winswitch version 0.12.21 which used xpra version 0.15.10 to show it.
  1. I ran the executable audioInputTest to record the sound and put into a file test.raw.
  1. I ran the executable audioOutputTest to play the sound from the test.raw file.

We saw no sound coming out. However, when we run the same steps on the same host. We see sound coming out.

I further used some combinations and found playback stream is working properly. But, the record stream is not working properly.

Please help, and let me know if any more information is required.

Thu, 04 Feb 2016 18:32:27 GMT - Kundan:

One more thing.

I had already changed, the /etc/xpra/xpra.conf to enable the microphone thing. So, that microphone forwarding could work. But, it didn't.

Fri, 05 Feb 2016 16:38:29 GMT - Kundan: attachment set

Fri, 05 Feb 2016 16:41:19 GMT - Kundan:

The pulse device was not selected in the earlier, audioInputTest. Therefore, added a new audioInputTestNew.tat.gz, where pulse device is selected. We are testing with pulse device for both record and playback stream.

Sorry for the confusion.

Sun, 14 Feb 2016 08:23:43 GMT - Kundan:

I tested with Ubuntu 15.10 xpra version v0.16.2, with the same earlier steps. But, this time I opened xterm. The winswitch version is 0.12.21.

  1. I opened the info(xterm) from the xpra menu which opens just near the winswitch icon, and enabled the microphone. But, this time also, the microphone forwarding didn't work.
  1. I then opened the pavucontrol and found that there were no xpra sessions in the recording tab.
  1. I then tried to open winswitch in debug mode. winswitch_applet --debug_mode
  1. On trying to repeat the 1st step. I found some issue getting logged, which was saying that start_sending_sound needs 7 arguments. However, only 6 arguments provided.

After, that I done some modification in ui_client_base.py in the following directory on my machine /usr/lib/python2.7/dist-packages/xpra/client.

I am attaching the updated file. Changes done on the line 1985 and 1986.

After that, on repeating the 1st step, I was able to see xpra entry in the recording tab of pavucontrol. But, the device selected was a monitor device.

On testing, it again didn't worked.

Then, I changed the device selected from monitor device to the regular device, and voila the remote recording(microphone forwarding) started working.

Thanks antoine for your hard work. It seems you made it work in v0.16.2.

But, still I am seeing one issue.

When I enable both microphone and speaker forwarding. I see a sound loop happening. It means, what I say returns back to me like echo.

It seems, both record stream and playback stream, are interfering in some way. But, what i failed to understand, that if these two streams are separate, why they are interfering. (May be it has to do with server's xpra's pulseaudio server). But, you know it better.

If possible, if we could make both the streams mutually exclusive and non-interfering. That would be better.

Thanks again.

Sun, 14 Feb 2016 08:27:29 GMT - Kundan: attachment set

The xpra client file (modified line 1985 1986)

Wed, 17 Feb 2016 15:27:48 GMT - Antoine Martin: owner changed

Please supply your changes in standard patch format against a specific branch and version, and please also explain why you made that change.

When I enable both microphone and speaker forwarding. I see a sound loop happening. It means, what I say returns back to me like echo.

pavucontrol is your friend. Please provide more details on which streams are hooked where and how. Maybe you are connecting to an existing pulseaudio server, from an existing session, rather than using a dedicated one for your xpra session. This can easily create sound problems, including loops.

Wed, 24 Feb 2016 17:13:41 GMT - Kundan:

Hi, Antoine,

I am attaching the patch in standard format, for microphone forwarding thing.

The reason for making the change.

When I ran winswitch_applet in debug mode. While starting microphone from the client computer, I was getting the error, that start_sending_sound requires 7 arguments, while the arguments provided was only 6. Then I checked the definition of start_sending_sound function in wrapper.py in /usr/lib/python2.7/dist-packages/xpra/sound. It was indeed 7.

Then, I noticed that on the server xpra _sound_record process get's created. But, on the client it doesn't. So, I tried to scan the python code there in whole xpra folder for instances of calling start_sending_sound.

I got one in /usr/lib/python2.7/dist-packages/xpra/server/source.py

There, 7 arguments was being passed, and 1st argument was getting populated just before that line.

plugins = self.sound_properties.get("plugins") <--- This line
ss = start_sending_sound(plugins, self.sound_source_plugin, codec, volume, [codec], self.pulseaudio_server, self.pulseaudio_id)

I copied this additional line in ui_client.py file as well in the client subdirectory, and passed plugins as the 1st argument.

This, made the microphone forwarding work for me, from the client computer to the server computer.

My xpra version is xpra v0.16.2.

I don't know much about the specific branch and version. But, I created the diff from the svn and the directory was Xpra/tags/v0.16.x/src/xpra/client.

Wed, 24 Feb 2016 17:15:16 GMT - Kundan: attachment set

Passing the plugins as the first argument to start_sending_sound

Wed, 24 Feb 2016 17:45:09 GMT - Antoine Martin:

You are absolutely right, very sorry for wasting your time like this. (I somehow got confused between microphone and speaker when I tested... despite the fact that your ticket title is absolutely correct)

I've applied to trunk in r12040. Then I hit another error: AttributeError: 'source_subprocess_wrapper' object has no attribute 'sequence', which is also fixed in r12041.

Both these changes have been applied to v0.16.x in r12042. Please close this ticket if that works for you.

Thanks you very much for the patch!

Wed, 24 Feb 2016 18:14:30 GMT - Kundan:

Now, I would like to talk about, the sound loop.

Steps done by me.

  1. I applied the patch mentioned above.
  2. Then, I opened an xterm from the other machine.
  3. Then, I made the speaker and microphone on from the xpra menu of xterm.
  4. Then, Changed the recording device from the monitor device to the other non-monitor one. This started the recording for me. But, also started the sound loop.
  1. Then , I disabled the speaker. And, was not able to hear the loop.

As, winswitch is controlling the pulseaudio server creation for the xpra session. I think, the pulseaudio server is the dedicated one, which is being used here. Not that of the system.

Also, I would like to share the ps -eaf output from the server system. I am attaching some screenshots to specify the steps done.

Wed, 24 Feb 2016 18:17:46 GMT - Kundan: attachment set

Playback stream of xpra, the opened xterm, and the microphone, speaker as active, active.

Wed, 24 Feb 2016 18:22:54 GMT - Kundan: attachment set

Recording stream, opened xterm, and state of microphone and speaker.

Wed, 24 Feb 2016 18:24:01 GMT - Kundan: attachment set

Changed the device from monitor to non-monitor one through pavucontrol.

Wed, 24 Feb 2016 18:31:37 GMT - Kundan: attachment set

Pulse and xpra output grep from the ps -eaf output. This has been done on the server.

Wed, 24 Feb 2016 18:33:11 GMT - Kundan:

Please let me know, if any more information is required.

Thu, 25 Feb 2016 03:52:21 GMT - Antoine Martin: owner, status changed

Congratulations... you've hit another bug!

We are selecting the wrong device on the client for microphone forwarding, it should not be a monitor device! (and also this should really be in the UI so we can let the user select which device they actually want to have forwarded)

You can temporarily workaround this problem by specifying the device to use on the command line (you will also need this trivial fix: r12044):

xpra attach tcp:.... --sound-source=pulesrc:device=yourdevicename

ie I tested with xpra attach tcp:SOMEHOST:10000 --sound-source=pulsesrc:device=alsa_input.pci-0000_00_14.2.analog-stereo -d gstreamer --microphone=on

You can find the list of device names by running: ./xpra/sound/pulseaudio/pulseaudio_util.py

Thu, 25 Feb 2016 04:29:48 GMT - Antoine Martin: owner, status changed

Should be fixed (in trunk) in r12048.

@kundanvp: does this work for you?

FYI: there are some more fixes related to microphone forwarding in r12055 + r12056

Thu, 25 Feb 2016 06:15:02 GMT - Antoine Martin:

Note: as of r12050 + r12051, you can also specify the sound source device using the syntax:

xpra attach .... --microphone="on:Built-in Audio Analog Stereo"

Sat, 27 Feb 2016 05:12:44 GMT - Kundan:

Hi Antoine,

I tried the following command,

/usr/bin/python /usr/bin/xpra --title=@title@ --session-name=XTerm \
   --password-file=/home/nishant/.winswitch/client/sessions/b170959609164cc0b587c9fb57b6f865/session.pass \
   -z1 --window-icon=/home/nishant/.winswitch/client/sessions/b170959609164cc0b587c9fb57b6f865/session.icon \
   attach tcp: \
   --sound-source=pulsesrc:device=alsa_input.pci-0000_00_05.0.analog-stereo \
   -d gstreamer --microphone=on

Actually, I opened an XTerm session with winswitch, and then learned what command it used for attaching, then I detached the session, and tried the above command manually.

My expectation was, it would open the microphone with the device alsa_input.pci-0000_00_05.0.analog-stereo,

But, it seems, it faulted and given the following error.

2016-02-27 10:27:40,410 sound source pipeline error: gst-resource-error-quark: Failed to connect stream: Invalid argument (1)
2016-02-27 10:27:40,411 sound source  pulsesrc.c(1620)
2016-02-27 10:27:40,411 sound source  gst_pulsesrc_prepare ()
2016-02-27 10:27:40,411 sound source  /GstPipeline:pipeline0/GstPulseSrc:pulsesrc0

The whole sequence of messages, it has given is as follows:-

2016-02-27 10:27:35,974 Xpra gtk2 client version 0.16.2-r11892
2016-02-27 10:27:35,975  running on Linux Ubuntu 15.10 wily
2016-02-27 10:27:36,442 GStreamer version 1.6 for Python 2.7
pci id for fd 3: 80ee:beef, driver (null)
libGL error: core dri or dri2 extension not found
libGL error: failed to load driver: vboxvideo
2016-02-27 10:27:37,203 PyOpenGL warning: missing accelerate module
2016-02-27 10:27:37,204 PyOpenGL warning: missing array format handlers: numeric, vbo, vbooffset
2016-02-27 10:27:37,204 OpenGL Version: 3.0 Mesa 11.0.2
2016-02-27 10:27:37,205 OpenGL support is missing:
2016-02-27 10:27:37,205  PyOpenGL version 3.1 or later is required (found version 3.0.2)
2016-02-27 10:27:37,662  detected keyboard: rules=evdev, model=pc105, layout=us
2016-02-27 10:27:37,664  desktop size is 1301x722 with 1 screen:
2016-02-27 10:27:37,665   :0.0 (344x191 mm - DPI: 96x96) workarea: 1236x698 at 65x24
2016-02-27 10:27:37,665     VGA-0
2016-02-27 10:27:38,160 Xpra X11 server version 0.16.2-r11892
2016-02-27 10:27:38,160  running on Linux Ubuntu 15.10 wily
2016-02-27 10:27:38,341 using pulseaudio device:
2016-02-27 10:27:38,341  'Monitor of Built-in Audio Analog Stereo'
2016-02-27 10:27:38,440 Attached to tcp: (press Control-C to detach)
2016-02-27 10:27:40,410 sound source pipeline error: gst-resource-error-quark: Failed to connect stream: Invalid argument (1)
2016-02-27 10:27:40,411 sound source  pulsesrc.c(1620)
2016-02-27 10:27:40,411 sound source  gst_pulsesrc_prepare ()
2016-02-27 10:27:40,411 sound source  /GstPipeline:pipeline0/GstPulseSrc:pulsesrc0
2016-02-27 10:27:40,534 server is not responding, drawing spinners over the windows
2016-02-27 10:27:41,799 server is OK again

The pavucontrol didn't have any entry in the recording tab.

Am, I doing it in the wrong way. Or, something more need to be fixed.

Can you help me, how this workaround could be made to work in xpra version 0.16.2. I have taken the changes for both attribute error and the 7 arguments requirement.

Sat, 27 Feb 2016 05:29:39 GMT - Antoine Martin:

Assuming that you do have a device called alsa_input.pci-0000_00_05.0.analog-stereo which you can verify running gstreamer util, the syntax should be correct. Try -d sound, as -d gstreamer is only supported in 0.17 onwards.

(PS: please try to use the "code-block" from the formatting options when pasting code, so I don't have to edit your comments)

Sun, 28 Feb 2016 09:05:31 GMT - Kundan: status changed; resolution set

Hi Antoine,

I used -d sound, as well. But, could not see the microphone forwarding running through it. Nothing appears in the pavucontrol recording tab and in Info Xterm, the microphone state is error. In the terminal, the main issue which we see is the same "Failed to connect to stream: Invalid argument". Seems this is a gstreamer issue.

But, I am currently happy with the workaround that, I am able to change the monitor device to the non-monitor device with pavucontrol itself. It works, at least for one way communication from client to server for microphone forwarding.

But, the erring thing is that, when I enable both microphone and speaker and my recording device is selected as a non-monitor one in pavucontrol. I am able to hear a sound loop. This is annoying. My expectation was that, since microphone forwarding and speaker forwarding is separate streams, there should not be any interferences.

But, I hope that you have already fixed this issue in latest trunk, and I will be able to test it when 0.17 would be released.

Currently, I am testing with v0.16.2.

I am closing this issue.

Sat, 23 Jan 2021 05:15:15 GMT - migration script:

this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/1110