xpra icon
Bug tracker and wiki

Opened 4 years ago

Closed 4 years ago

#1110 closed defect (fixed)

Microphone forwarding doesn't work properly.

Reported by: Kundan Owned by: Kundan
Priority: major Milestone: 0.17
Component: sound Version: trunk
Keywords: Cc:

Description

Microphone forwarding seems to work, but even in a simple recording application which records the sound and puts in a file with pulse as the recording device. The sound output data from the file is blank. I mean to say file is not blank, but what it contains is not our recorded sound. It's just mute. However, The playback works with same pulse device. I have seen this issue, in Ubuntu 14.04, xpra version v0.15.10.

Please let me know, if I am doing it in wrong way.

Attachments (10)

audioInputTest.tar.gz (1.4 KB) - added by Kundan 4 years ago.
audioOutputTest.tar.gz (1.4 KB) - added by Kundan 4 years ago.
testAudio.tar.gz (367.5 KB) - added by Kundan 4 years ago.
audioInputTestNew.tar.gz (1.4 KB) - added by Kundan 4 years ago.
ui_client_base.py (125.5 KB) - added by Kundan 4 years ago.
The xpra client file (modified line 1985 1986)
start_sending_sound.diff (754 bytes) - added by Kundan 4 years ago.
Passing the plugins as the first argument to start_sending_sound
Screenshot from 2016-02-24 22-59-29.png (520.4 KB) - added by Kundan 4 years ago.
Playback stream of xpra, the opened xterm, and the microphone, speaker as active, active.
Screenshot from 2016-02-24 23-00-55.png (513.7 KB) - added by Kundan 4 years ago.
Recording stream, opened xterm, and state of microphone and speaker.
Screenshot from 2016-02-24 23-04-09.png (562.5 KB) - added by Kundan 4 years ago.
Changed the device from monitor to non-monitor one through pavucontrol.
Screenshot from 2016-02-24 23-33-14.png (162.8 KB) - added by Kundan 4 years ago.
Pulse and xpra output grep from the ps -eaf output. This has been done on the server.

Change History (27)

comment:1 Changed 4 years ago by Antoine Martin

Owner: changed from Antoine Martin to Kundan

Please see wiki/ReportingBugs for guidelines, and wiki/Sound.
There is not enough information in this ticket for me to investigate.

Changed 4 years ago by Kundan

Attachment: audioInputTest.tar.gz added

Changed 4 years ago by Kundan

Attachment: audioOutputTest.tar.gz added

Changed 4 years ago by Kundan

Attachment: testAudio.tar.gz added

comment:2 Changed 4 years ago by Antoine Martin

Too many details missing: What xpra command lines did you use?
what was the output of those commands, what device does it use?
Mixer settings, etc.. (pavucontrol is your friend)

comment:3 Changed 4 years ago by Kundan

Owner: changed from Kundan to Antoine Martin

Hi Antoine,
Sorry for being vague in my queries.

I have attached three folders for testing the sound thing. I tested it on Ubuntu 14.04.3, with winswitch version 0.12.21, xpra version v0.15.10.

The folders, audioInputTest, and audioOutputTest contains C++ program compilable over Qt5.5.1. The third folder contains the compiled copy.

The steps for reproduction are as follows:-

  1. I complied the audioInputTest and audioOutputTest and created the executable audioInputTest and audioOutputTest.
  1. I opened a remote Terminal through winswitch version 0.12.21 which used xpra version 0.15.10 to show it.
  1. I ran the executable audioInputTest to record the sound and put into a file test.raw.
  1. I ran the executable audioOutputTest to play the sound from the test.raw file.

We saw no sound coming out. However, when we run the same steps on the same host. We see sound coming out.

I further used some combinations and found playback stream is working properly. But, the record stream is not working properly.

Please help, and let me know if any more information is required.

comment:4 Changed 4 years ago by Kundan

One more thing.

I had already changed, the /etc/xpra/xpra.conf to enable the microphone thing. So, that microphone forwarding could work. But, it didn't.

Changed 4 years ago by Kundan

Attachment: audioInputTestNew.tar.gz added

comment:5 Changed 4 years ago by Kundan

The pulse device was not selected in the earlier, audioInputTest. Therefore, added a new audioInputTestNew.tat.gz, where pulse device is selected. We are testing with pulse device for both record and playback stream.

Sorry for the confusion.

comment:6 Changed 4 years ago by Kundan

I tested with Ubuntu 15.10 xpra version v0.16.2, with the same earlier steps.
But, this time I opened xterm. The winswitch version is 0.12.21.

  1. I opened the info(xterm) from the xpra menu which opens just near the winswitch icon, and enabled the microphone. But, this time also, the microphone forwarding didn't work.
  1. I then opened the pavucontrol and found that there were no xpra sessions in the recording tab.
  1. I then tried to open winswitch in debug mode. winswitch_applet --debug_mode
  1. On trying to repeat the 1st step. I found some issue getting logged, which was saying that start_sending_sound needs 7 arguments. However, only 6 arguments provided.

After, that I done some modification in ui_client_base.py in the following directory on my machine /usr/lib/python2.7/dist-packages/xpra/client.

I am attaching the updated file. Changes done on the line 1985 and 1986.

After that, on repeating the 1st step, I was able to see xpra entry in the recording tab of pavucontrol. But, the device selected was a monitor device.

On testing, it again didn't worked.

Then, I changed the device selected from monitor device to the regular device, and voila the remote recording(microphone forwarding) started working.

Thanks antoine for your hard work. It seems you made it work in v0.16.2.

But, still I am seeing one issue.

When I enable both microphone and speaker forwarding. I see a sound loop happening. It means, what I say returns back to me like echo.

It seems, both record stream and playback stream, are interfering in some way. But, what i failed to understand, that if these two streams are separate, why they are interfering. (May be it has to do with server's xpra's pulseaudio server). But, you know it better.

If possible, if we could make both the streams mutually exclusive and non-interfering. That would be better.

Thanks again.

Changed 4 years ago by Kundan

Attachment: ui_client_base.py added

The xpra client file (modified line 1985 1986)

comment:7 Changed 4 years ago by Antoine Martin

Owner: changed from Antoine Martin to Kundan

Please supply your changes in standard patch format against a specific branch and version, and please also explain why you made that change.


When I enable both microphone and speaker forwarding. I see a sound loop happening. It means, what I say returns back to me like echo.


pavucontrol is your friend.
Please provide more details on which streams are hooked where and how.
Maybe you are connecting to an existing pulseaudio server, from an existing session, rather than using a dedicated one for your xpra session.
This can easily create sound problems, including loops.

comment:8 Changed 4 years ago by Kundan

Hi, Antoine,

I am attaching the patch in standard format, for microphone forwarding thing.

The reason for making the change.

When I ran winswitch_applet in debug mode. While starting microphone from the client computer, I was getting the error, that start_sending_sound requires 7 arguments, while the arguments provided was only 6. Then I checked the definition of start_sending_sound function in wrapper.py in /usr/lib/python2.7/dist-packages/xpra/sound. It was indeed 7.

Then, I noticed that on the server xpra _sound_record process get's created. But, on the client it doesn't. So, I tried to scan the python code there in whole xpra folder for instances of calling start_sending_sound.

I got one in /usr/lib/python2.7/dist-packages/xpra/server/source.py

There, 7 arguments was being passed, and 1st argument was getting populated just before that line.

plugins = self.sound_properties.get("plugins") <--- This line
ss = start_sending_sound(plugins, self.sound_source_plugin, codec, volume, [codec], self.pulseaudio_server, self.pulseaudio_id)

I copied this additional line in ui_client.py file as well in the client subdirectory, and passed plugins as the 1st argument.

This, made the microphone forwarding work for me, from the client computer to the server computer.

My xpra version is xpra v0.16.2.

I don't know much about the specific branch and version. But, I created the diff from the svn and the directory was Xpra/tags/v0.16.x/src/xpra/client.

Last edited 4 years ago by Antoine Martin (previous) (diff)

Changed 4 years ago by Kundan

Attachment: start_sending_sound.diff added

Passing the plugins as the first argument to start_sending_sound

comment:9 Changed 4 years ago by Antoine Martin

You are absolutely right, very sorry for wasting your time like this.
(I somehow got confused between microphone and speaker when I tested... despite the fact that your ticket title is absolutely correct)

I've applied to trunk in r12040. Then I hit another error: AttributeError: 'source_subprocess_wrapper' object has no attribute 'sequence', which is also fixed in r12041.

Both these changes have been applied to v0.16.x in r12042. Please close this ticket if that works for you.

Thanks you very much for the patch!

comment:10 Changed 4 years ago by Kundan

Now, I would like to talk about, the sound loop.

Steps done by me.

  1. I applied the patch mentioned above.
  2. Then, I opened an xterm from the other machine.
  3. Then, I made the speaker and microphone on from the xpra menu of xterm.
  4. Then, Changed the recording device from the monitor device to the other non-monitor one. This started the recording for me. But, also started the sound loop.
  1. Then , I disabled the speaker. And, was not able to hear the loop.

As, winswitch is controlling the pulseaudio server creation for the xpra session. I think, the pulseaudio server is the dedicated one, which is being used here. Not that of the system.

Also, I would like to share the ps -eaf output from the server system.
I am attaching some screenshots to specify the steps done.

Changed 4 years ago by Kundan

Playback stream of xpra, the opened xterm, and the microphone, speaker as active, active.

Changed 4 years ago by Kundan

Recording stream, opened xterm, and state of microphone and speaker.

Changed 4 years ago by Kundan

Changed the device from monitor to non-monitor one through pavucontrol.

Changed 4 years ago by Kundan

Pulse and xpra output grep from the ps -eaf output. This has been done on the server.

comment:11 Changed 4 years ago by Kundan

Please let me know, if any more information is required.

comment:12 Changed 4 years ago by Antoine Martin

Owner: changed from Kundan to Antoine Martin
Status: newassigned

Congratulations... you've hit another bug!

We are selecting the wrong device on the client for microphone forwarding, it should not be a monitor device!
(and also this should really be in the UI so we can let the user select which device they actually want to have forwarded)

You can temporarily workaround this problem by specifying the device to use on the command line (you will also need this trivial fix: r12044):

xpra attach tcp:.... --sound-source=pulesrc:device=yourdevicename

ie I tested with xpra attach tcp:SOMEHOST:10000 --sound-source=pulsesrc:device=alsa_input.pci-0000_00_14.2.analog-stereo -d gstreamer --microphone=on

You can find the list of device names by running: ./xpra/sound/pulseaudio/pulseaudio_util.py

Last edited 4 years ago by Antoine Martin (previous) (diff)

comment:13 Changed 4 years ago by Antoine Martin

Owner: changed from Antoine Martin to Kundan
Status: assignednew

Should be fixed (in trunk) in r12048.

@kundanvp: does this work for you?

FYI: there are some more fixes related to microphone forwarding in r12055 + r12056

Last edited 4 years ago by Antoine Martin (previous) (diff)

comment:14 Changed 4 years ago by Antoine Martin

Note: as of r12050 + r12051, you can also specify the sound source device using the syntax:

xpra attach .... --microphone="on:Built-in Audio Analog Stereo"
Last edited 4 years ago by Antoine Martin (previous) (diff)

comment:15 Changed 4 years ago by Kundan

Hi Antoine,

I tried the following command,

/usr/bin/python /usr/bin/xpra --title=@title@ --session-name=XTerm \
   --password-file=/home/nishant/.winswitch/client/sessions/b170959609164cc0b587c9fb57b6f865/session.pass \
   -z1 --window-icon=/home/nishant/.winswitch/client/sessions/b170959609164cc0b587c9fb57b6f865/session.icon \
   attach tcp:127.0.0.1:12462 \
   --sound-source=pulsesrc:device=alsa_input.pci-0000_00_05.0.analog-stereo \
   -d gstreamer --microphone=on

Actually, I opened an XTerm session with winswitch, and then learned what command it used for attaching, then I detached the session, and tried the above command manually.

My expectation was, it would open the microphone with the device alsa_input.pci-0000_00_05.0.analog-stereo,

But, it seems, it faulted and given the following error.

2016-02-27 10:27:40,410 sound source pipeline error: gst-resource-error-quark: Failed to connect stream: Invalid argument (1)
2016-02-27 10:27:40,411 sound source  pulsesrc.c(1620)
2016-02-27 10:27:40,411 sound source  gst_pulsesrc_prepare ()
2016-02-27 10:27:40,411 sound source  /GstPipeline:pipeline0/GstPulseSrc:pulsesrc0

The whole sequence of messages, it has given is as follows:-

2016-02-27 10:27:35,974 Xpra gtk2 client version 0.16.2-r11892
2016-02-27 10:27:35,975  running on Linux Ubuntu 15.10 wily
2016-02-27 10:27:36,442 GStreamer version 1.6 for Python 2.7
pci id for fd 3: 80ee:beef, driver (null)
libGL error: core dri or dri2 extension not found
libGL error: failed to load driver: vboxvideo
2016-02-27 10:27:37,203 PyOpenGL warning: missing accelerate module
2016-02-27 10:27:37,204 PyOpenGL warning: missing array format handlers: numeric, vbo, vbooffset
2016-02-27 10:27:37,204 OpenGL Version: 3.0 Mesa 11.0.2
2016-02-27 10:27:37,205 OpenGL support is missing:
2016-02-27 10:27:37,205  PyOpenGL version 3.1 or later is required (found version 3.0.2)
2016-02-27 10:27:37,662  detected keyboard: rules=evdev, model=pc105, layout=us
2016-02-27 10:27:37,664  desktop size is 1301x722 with 1 screen:
2016-02-27 10:27:37,665   :0.0 (344x191 mm - DPI: 96x96) workarea: 1236x698 at 65x24
2016-02-27 10:27:37,665     VGA-0
2016-02-27 10:27:38,160 Xpra X11 server version 0.16.2-r11892
2016-02-27 10:27:38,160  running on Linux Ubuntu 15.10 wily
2016-02-27 10:27:38,341 using pulseaudio device:
2016-02-27 10:27:38,341  'Monitor of Built-in Audio Analog Stereo'
2016-02-27 10:27:38,440 Attached to tcp:127.0.0.1:12462 (press Control-C to detach)

2016-02-27 10:27:40,410 sound source pipeline error: gst-resource-error-quark: Failed to connect stream: Invalid argument (1)
2016-02-27 10:27:40,411 sound source  pulsesrc.c(1620)
2016-02-27 10:27:40,411 sound source  gst_pulsesrc_prepare ()
2016-02-27 10:27:40,411 sound source  /GstPipeline:pipeline0/GstPulseSrc:pulsesrc0
2016-02-27 10:27:40,534 server is not responding, drawing spinners over the windows
2016-02-27 10:27:41,799 server is OK again

The pavucontrol didn't have any entry in the recording tab.

Am, I doing it in the wrong way. Or, something more need to be fixed.

Can you help me, how this workaround could be made to work in xpra version 0.16.2. I have taken the changes for both attribute error and the 7 arguments requirement.

Last edited 4 years ago by Antoine Martin (previous) (diff)

comment:16 Changed 4 years ago by Antoine Martin

Assuming that you do have a device called alsa_input.pci-0000_00_05.0.analog-stereo which you can verify running gstreamer util, the syntax should be correct. Try -d sound, as -d gstreamer is only supported in 0.17 onwards.

(PS: please try to use the "code-block" from the formatting options when pasting code, so I don't have to edit your comments)

comment:17 Changed 4 years ago by Kundan

Resolution: fixed
Status: newclosed

Hi Antoine,

I used -d sound, as well. But, could not see the microphone forwarding running through it. Nothing appears in the pavucontrol recording tab and in Info Xterm,
the microphone state is error.
In the terminal, the main issue which we see is the same "Failed to connect to stream: Invalid argument". Seems this is a gstreamer issue.

But, I am currently happy with the workaround that, I am able to change the monitor device to the non-monitor device with pavucontrol itself. It works, at least for one way communication from client to server for microphone forwarding.

But, the erring thing is that, when I enable both microphone and speaker and my recording device is selected as a non-monitor one in pavucontrol. I am able to hear a sound loop. This is annoying.
My expectation was that, since microphone forwarding and speaker forwarding is separate streams, there should not be any interferences.

But, I hope that you have already fixed this issue in latest trunk, and I will be able to test it when 0.17 would be released.

Currently, I am testing with v0.16.2.

I am closing this issue.

Note: See TracTickets for help on using tickets.