Similar to noVNC, and probably re-using some of its code (websockets, etc)
Made much easier by #474
I can start a websockets proxy (on port 8080) that points back to xpra (on port 10000):
./websockify.py --web ./Xpra/trunk/html5/ 8080 localhost:10000
And if I use the new tcp-proxy code:
xpra start :10 --bind-tcp=0.0.0.0:10000 --tcp-proxy=127.0.0.1:8080
When we make websocket request to the xpra port (10000), the http traffic goes through xpra to the proxy then back to xpra as tcp! Eventually, the websockets proxy code should be moved into xpra, but this is good enough for experimenting and testing.
experimental html5 client is now capable of showing an xterm window!
modified bencoder that can talk to an xpra server and handle byte buffers efficiently (original code is MIT licensed)
What I've used:
The license reads: Free to use and distribute at will, so long as you are nice to people, etc
. May need clarification... as free licenses are unrestricted, and asking for people to "be nice" may be incompatible with other free licenses!
I will attach some code once it is cleaned up.
What still needs to be done:
map-event
packet which is unnecessary using HTML5 (we can place things precisely wherever we like)
And maybe further down the line:
Initial code merged in r5028: sort of works in chrome and firefox (partial window updates don't work, partial keyboard support, no mouse support, ..)
Many fixes (see changesets) in: r5029 + r5030 + r5031 + r5032 + r5033
Actually makes this almost usable!
Many more changes in r5034 to r5066, it looks a lot better:
BackSpace
)
http://localhost:8080/index.html?host=localhost&port=8080&connect=true
(and validate all values to prevent XSS)
updated screenshot of html5 client running in full-window mode
r5076 + r5077 + r5078 + r5079 + r5080 add a cython version of the bencoder, which makes it substantially faster (important since the html5 version uses the bencoder rather than rencode)
Running the updated bencode test shows the difference in performance:
test_compare_cython() results: {'python': 1903, 'cython': 1255} (in milliseconds)
Many more improvements in r5081 to r5089, including support for fullscreen and maximized windows, window top bar icons, etc..
Sadly, I've come to realize that the way I have used the canvas is all wrong: each window should be its own DIV
instead, and we can manage:
etc
But I have to leave this for now... it is what it is.
Needs more time than I can afford to spend on it. Re-scheduling.
Using a CSS decorations on a canvas-per-window is a must though.
work by Joshua Higgins to make this use a canvas within a div for each window (much much better!)
superseding my previous patch, cleaned up, many improvements
Attached updated patch
slightly modified patch
The patch above makes the following changes on your patch:
util.js
and websock.js
function pass()
(not referenced anywhere?)
index.html
since it is mostly unchanged and removed extra closing div
do_send_new_screen_size
(I seem to remember it avoided errors when closing the connection?)
I did this as I was reviewing the code. I then tried to run it but it must have got stuck... will try again later.
Note: as part of #614, r6934 adds support for YAML as packet encoding.
@joshiggins: Any HTML5 updates?
Replying to totaam:
@joshiggins: Any HTML5 updates?
Adding missing window functions for icon, maximise and actually destroying windows when connection is closed (currently leaves the last drawn windows floating around). Started moving websockets code into a worker. Maybe it would be good to start SVN revisions to make it easier tracking bugs in these patches/?
There is a bug that affects even the unpatched html5 client which causes an invalid packet header error to be thrown with a large number of screen updates, which looks like the buffer in protocol.js to be at fault but nailing it has been proving difficult!
Adding missing...
Good stuff!
Maybe it would be good to start SVN revisions
svn repo info sent separately
which causes an invalid packet header error to be thrown with a large number of screen updates
Just a guess, but it might be that those are rgb
updates compressed with zlib, try turning that off and see if it helps. (see #614)
r7227 merges latest html5-v2.patch from comment 14
Some updates in r7228 + r7230 + r7255 + r7271
However, using a Websocket within a Webworker is not supported in Firefox and it doesn't look like it's coming anytime soon (see https://bugzilla.mozilla.org/show_bug.cgi?id=504553) so the web worker method will only be used if WebKit? is detected.
It's usable at this stage with not so bad performance. In addition to some bits in comment 3 also needs:
Seems to work well in Chrome, Safari and Firefox. Reports from Internet Explorer would be welcome.
I'd very much like to try out the recent changes to the html5 client but I'm having an issue with the 0.14.x release (I was able to run the client with 0.13.x). The message I'm seeing in the log file is "no matching packet encoder found!". Looking at the code for 0.14.x I'm not sure the following logic is correct:
svn annotate http://xpra.org/svn/Xpra/tags/v0.14.x/src/xpra/net/protocol.py | less -N
352 6964 antoine def enable_encoder_from_caps(self, caps): 353 6983 antoine opts = packet_encoding.get_enabled_encoders(order=packet_encoding.PERFORMANCE_ORDER) 354 6982 antoine for e in opts: 355 6982 antoine if caps.boolget("rencode"): 356 6982 antoine self.enable_encoder(e) 357 6982 antoine return True 358 6982 antoine log.error("no matching packet encoder found!") 359 6982 antoine return False
Especially when comparing it to a prior version of the same method: svn annotate -r 6981 http://xpra.org/svn/Xpra/trunk/src/xpra/net/protocol.py
6964 antoine def enable_encoder_from_caps(self, caps): 6965 antoine if packet_encoding.use_rencode and caps.boolget("rencode"): 5076 antoine self.enable_rencode() 6965 antoine elif packet_encoding.use_yaml and caps.boolget("yaml"): 6964 antoine self.enable_yaml() 6965 antoine elif packet_encoding.use_bencode and caps.boolget("bencode", True): 6964 antoine self.enable_bencode() 6964 antoine else: 6969 antoine log.error("no matching packet encoder found!") 6969 antoine return False 6969 antoine return True
However both methods are pretty different from how it was coded in 0.13.x. I'm not quite sure what to try next if this isn't the cause of my connection issue. Is the 0.14.x logic correct? Is there something else I should look at? Thanks - Sam
You're right, that looks completely wrong! It must have got broke after the testing in #614. Does r7365 + r7374 fix things for you?
That looks much better. I'll try it out later today. Thanks - Sam
Tried it out and it works much better then before , great work! I was wondering if we could get a couple of switches to say make it so app starts full screen with no min/close window perhaps if we are only gonna use it for one app that will use the entire canvas space of the browser area.
Yeah, just tried it and it works great! I've mirrored the html code here for easy access: http://xpra.org/html5/
I managed to hit this bug though:
ValueError: invalid literal for long() with base 10: '563.991455078125' 2014-09-17 10:20:33,210 failed to parse bencode packet: 6c31363a63...
Which decodes to:
$ python -c 'import binascii;print(binascii.unhexlify("6c31363a63..."))' l16:configure-windowi1ei563.991455078125ei487.991455078125ei499ei316ed21:encodings.rgb_formatsl4:RGBX4:RGBAeee
So the configure window uses dimensions that are not whole numbers, and that crashes the server bdecoder because that's invalid.
Also saw these in the websockify server log (not sure if they are related):
code 400, message Bad request syntax ('\x88\x8f\\\xb9\xac\xc6_Q\xf8\xa7.\xde\xc9\xb2|\xda\xc0\xa9/\xdc\xc8') code 400, message Bad HTTP/0.9 request type ('\x88\x8f\x0e\x06\xd6I') code 400, message Bad HTTP/0.9 request type ('\x88\x8f#\xaf\x85\x15')
I think r7677 should fix sending window dimensions that are not whole numbers
@aradtech I like the idea of having a switch to full screen an app with no borders, should be trivial to implement but how would you handle for example, popup windows? Draw them with no chrome but in the position they requested?
I've made it easier to setup the html client in #689.
It would be nice if the host and port fields could be populated with the host and port from the http request. Even better would be if it could auto-connect by default, and present a password box if the server requires authentication.
r7994 adds auto connect to host and port from http request, authentication coming soon.
I don't know how to help debug these issues: I am getting no windows about 30% of the time (just force reload the page), and sometimes I get a window but its contents do not get painted until I move it, the rest of the time it works as expected..
joshiggins , could you please contact me via email at aradtech@…, we are interested in having you do some more work on the html5 client.
r8710 and r8712 makes the client side protocol handling more stable.
It's more stable now. Next up, window drawing will be rationalised to iron out inconsistent behaviour (like sometimes not immediately painting contents), so that we have a solid foundation to start adding more encodings.
Replying to totaam:
Hi, I couldn't get the html5 client to work properly. Can you please check if I did something wrong, or if it might be an error?
In order to reproduce the behavior, I created a Dockerfile (http://paste.ubuntu.com/10518212/). You can use Docker on Linux or Boot2Docker on Windows or OS X to create an Xpra container.
The image can be build with docker build -t xpratest .
and a new container be started with docker run --rm -p 8001:8001 xpratest
.
When I connect to the web, I would expect both, xeyes and xterm to be rendered properly. Instead, the pupils of xeyes are not moved but painted over, when a movement occurs (the mouse is moved), and the xterm window stays empty. The result looks like https://www.dropbox.com/s/7g8xojwce3z1x3t/Screenshot%202015-03-03%2019.38.29.png?dl=0.
I used the current SVN revision 8743 as well as 8696, which has once already worked before. If I am using an Ubuntu image instead of Debian Jessie, the result is the same.
Do you have any idea what the problem might be?
I couldn't build the supplied Dockerfile correctly. I had to put the packages on a single line and run the Xpra server as root, for some reason the created user wanted to use / as it's home directory.
However, you are missing the python-imaging package.
I use a container for development and I can share mine if you are interested.
Replying to joshiggins:
I couldn't build the supplied Dockerfile correctly. I had to put the packages on a single line and run the Xpra server as root, for some reason the created user wanted to use / as it's home directory.
Maybe you were using an older Docker version? I think I remember some changes on 1.5.
However, you are missing the python-imaging package.
Great, that solved my problem, thank you!
I use a container for development and I can share mine if you are interested.
That would be great, maybe I made some other mistakes.
However, you are missing the python-imaging package.
Great, that solved my problem, thank you!
In theory, the server should be able to work without python-imaging.. Not well, but it should be able to work using plain rgb encoding. (assuming that the html client handles that)
In theory, the server should be able to work without python-imaging.. Not well, but it should be able to work using plain rgb encoding. (assuming that the html client handles that)
Is there any difference in handling between rgb24 and rgb? the HTML5 client asks for rgb24...
r8746 + r8747 completes refactoring (and makes it the default) so that the client is more javascript-y. The implementation detail is largely unchanged but the following is notable
XpraWindow.paint()
so that we avoid painting twice
XpraClient
object. Right now the behaviour is to connect automatically to the host and port from the HTTP request.
r8767 fixes a bug in the protocol handling that has evaded me for months, that usually manifests itself in windows not being drawn immediately.
r8781 adds jpeg and png support to the HTML5 client.
Right now we use the native Image() object which needs the data base64 encoded - seems like a faff but even so it turned out considerably faster than png.js.
First of all, thank you for your great job!
When using your latest changeset (r8781), there is a strange issue with some applications. If I run a QT application using wine, all menu elements (file menu, dropdowns, ...) open below the current window, not in front of it as expected. This only occurs when using the html5 client.
Do you have any idea what the problem might be? What information should I provide you in order to locate the issue?
Another issue is about the keyboard. When using a German keyboard layout, I can type all letters and numbers. There are some special characters like the exclamation mark I can type as well, but there is e.g. no key mapped to the period, so I'm unable to type it. Is it a known issue? Is there a way to configure the keyboard usage to include all German keys, maybe even including German "Umlauts"?
r9046 adds a style class for the window type MENU (we only had DROPDOWN_MENU and POPUP_MENU). Hopefully there shouldn't be any more menu elements appearing behind windows.
I've created a ticket for the keyboard problem #837.
H.264 decoding support is stabilising in r9045. It will start to be selected if mutually available since r9049.
Use the new connection interface introduced in r9050 to change the preferred encoding by navigating to http://server:port/connect.html
The default behaviour without specifying /connect.html is still to connect automatically. If the connection is not successful, it will redirect to the connection interface.
Replying to joshiggins:
r9046 adds a style class for the window type MENU (we only had DROPDOWN_MENU and POPUP_MENU). Hopefully there shouldn't be any more menu elements appearing behind windows.
Thank you for your update! Unfortunately the problem still persists for my useless. I'd provide a Dockerfile, but this didn't work the last time. All I did was install a current wine version and ran few Qt Applications. None of them (like VLC in the pictures above) worked properly: They displayed all menu items, dropdowns and tooltips behind the window. I had to move the window away in order to see them, and be able to select an entry.
Do you have any other idea what I might try, or what debug / log information I should provide?
I've created a ticket for the keyboard problem #837.
Great, thank you!
@extasic I can't seem to reproduce this with the latest revision.
The wine applications use the DIALOG type hint for popups but these are still correctly placed above the WINDOW type.....
If you can tell me which window class is applied to your popup windows it would be useful. This is easy in chrome just by right clicking and selecting inspect element. It will show the html markup which should specify a class="window-WINDOW_TYPE" for the popup window.
Failing that I can take another stab at the Dockerfile.
Not had time to look into it, but with the latest trunk I get:
disconnect: invalid packet format, not an xpra client?
It was fine just hours/days ago.
Replying to joshiggins:
@extasic I can't seem to reproduce this with the latest revision.
The wine applications use the DIALOG type hint for popups but these are still correctly placed above the WINDOW type.....
If you can tell me which window class is applied to your popup windows it would be useful. This is easy in chrome just by right clicking and selecting inspect element. It will show the html markup which should specify a class="window-WINDOW_TYPE" for the popup window.
This is how the resulting HTML looks in Chrome. The highlighted Canvas represents a file menu (just like in the VLC picture I posted above). The div above the dialog one represents the main window. So there might be an error in the window type, right? But shouldn't there be something like a z-index anyway? Any workaround for this?
Just to mention it again: This error occurs for me only when running any Qt application in wine. If I run a non-Qt application in wine, or a Qt application directly in Linux, I couldn't reproduce the error. If I use the native xpra client, the error doesn't occur, even when running a Qt application under wine.
My guess is that those applications use the wrong window type, it probably should not be NORMAL (sadly, lots of applications do that - ie: chrome does). We have some code in the xpra client to try to workaround such broken behaviour. @joshiggins: maybe the window is "transient-for" or "override-redirect" and you can use that?
Added a workaround in r9138. I was able to reproduce with VLC under Wine, it creates NORMAL override-redirect windows for the menus.
You can ignore comment:45, I was hitting the wrong test server - I need sleep!
I've uploaded a short video clip demonstrating the current state of H.264 decoding as a result of changes referred to in comment:42
https://www.youtube.com/watch?v=3yOoGWGyvgM
It's actually pretty good and remains usable.
I can confirm Josh's changes work on F21.
And would like to say thank you to Josh for your awesome work on the Xpra Html5 client and our requests.
Replying to joshiggins:
Added a workaround in r9138. I was able to reproduce with VLC under Wine, it creates NORMAL override-redirect windows for the menus.
Works for me too, thank you!
r9148 + r9149 adds support for LZ4 packet compressor
It's looking good for the release!
I have tagged v0.15.x, so let's move new work to a new ticket: #850
It will be easier to know what was delivered in this release and what remains to be done.
If you are hitting this ticket wanting to report a bug, please open a new ticket instead.
(update milestone)
There is now a wiki page for the HTML5 client: wiki/Clients/HTML5.
this ticket has been moved to: https://github.com/Xpra-org/xpra/issues/473