xpra icon
Bug tracker and wiki

Opened 5 months ago

Closed 4 months ago

Last modified 4 months ago

#1913 closed defect (fixed)

high memory usage

Reported by: Antoine Martin Owned by: J. Max Mena
Priority: major Milestone: 2.4
Component: core Version: 2.3.x
Keywords: Cc:

Description

Despite #1861 and #1838, the client and server still use too much memory, even when disabling most of the video and csc modules - or even nuking them from the filesystem.

Change History (3)

comment:1 Changed 4 months ago by Antoine Martin

Status: newassigned

According to how to get "real" process memory and environ in Python, we aren't really using as much memory as previously thought.
The amount of memory which would be freed if the process was terminated right now is only around 200MB for a server, much less for a client.
Caveats: this does not include shared libraries we share with other processes, GPU memory, etc.

With the debug code added in r19985, we use psutil (docs: https://psutil.readthedocs.io/en/latest/ psutil]) to dump memory info at time intervals:

XPRA_MEM_USAGE_LOGGER=100 xpra start-desktop --start=openbox --daemon=no
2018-08-01 16:13:11,847 init_html_proxy(..) html=None
2018-08-01 16:13:11,857 memory usage: pfullmem(rss=60104704, vms=552919040, shared=24821760, text=4096, lib=0, data=42868736, dirty=0, uss=46182400, pss=48150528, swap=0)
2018-08-01 16:13:11,944 serving html content from: /usr/share/xpra/www
2018-08-01 16:13:11,944 get_auth_modules(local-auth, [], {..})
2018-08-01 16:13:11,944 get_auth_modules(tcp, [], {..})
2018-08-01 16:13:11,944 get_auth_modules(ws, [], {..})
2018-08-01 16:13:11,944 get_auth_modules(wss, [], {..})
2018-08-01 16:13:11,944 get_auth_modules(ssl, [], {..})
2018-08-01 16:13:11,944 get_auth_modules(ssh, [], {..})
2018-08-01 16:13:11,945 get_auth_modules(rfb, [], {..})
2018-08-01 16:13:11,945 get_auth_modules(vsock, [], {..})
2018-08-01 16:13:11,945 get_auth_modules(udp, [], {..})
2018-08-01 16:13:11,945 init_auth(..) auth={'udp': None, 'rfb': None, 'vsock': None, 'tcp': None, 'ssl': None, 'wss': None, 'ws': None, 'ssh': None, 'unix-domain': None}
2018-08-01 16:13:11,945 file transfer: init_attributes('auto', 100, 'no', 'auto', 'auto', '/usr/bin/xdg-open', False)
2018-08-01 16:13:11,945 file transfer attributes={'open-files-ask': False, 'open-url': True, 'file-ask-timeout': 3600, 'open-files': True, 'file-size-limit': 100, 'file-transfer': True, 'file-chunks': 65536, 'file-transfer-ask': False, 'printing-ask': False, 'open-url-ask': False, 'printing': False}
2018-08-01 16:13:11,945 ChildReaper(None)
2018-08-01 16:13:11,958 memory usage: pfullmem(rss=68050944, vms=979922944, shared=29216768, text=4096, lib=0, data=173236224, dirty=0, uss=40009728, pss=50673664, swap=0)

The "data" memory usage goes up by ~128MB around the time we init websockify and the auth modules / file transfer.

comment:2 Changed 4 months ago by Antoine Martin

Owner: changed from Antoine Martin to J. Max Mena
Status: assignednew

Helped by r19986, I found that the memory usage is coming from importing websockify. More specifically: websockify imports numpy which is responsible for most of that 128MB memory usage. Follow up ticket: #1926.

r19987 changes some of the xpra codec loading code so we don't import numpy unless we really have to (ie: pycuda requires it).
With this change and turning off websockify with --html=no, the memory usage is now much lower:

2018-08-01 18:05:02,405 memory usage: pfullmem(rss=89583616, vms=755613696, shared=36974592, text=4096, lib=0, data=78819328, dirty=0, uss=74027008, pss=75572224, swap=0)

The next big chunks of memory come from the video and csc codecs (~40MB), which is to be expected. And also dbus (20MB!).

@maxmylyn: FYI.

Last edited 4 months ago by Antoine Martin (previous) (diff)

comment:3 Changed 4 months ago by J. Max Mena

Resolution: fixed
Status: newclosed

Noted and closing.

(Dbus uses far more memory than I thought it would have - 30mb is quite a lot for something so simple)

Note: See TracTickets for help on using tickets.