xpra icon
Bug tracker and wiki

This bug tracker and wiki are being discontinued
please use https://github.com/Xpra-org/xpra instead.

Changes between Version 6 and Version 7 of Testing

03/15/13 06:04:53 (8 years ago)
Antoine Martin

add info on automated perf tests


  • Testing

    v6 v7  
    2323* CentOS 5.x clients with both old servers (CentOS 5.x) and new ones (Fedora 18+ or Debian sid)
    2424* Debian Squeeze or Ubuntu Lucid packages.
     26== Automated performance and regression testing ==
     27The [http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/test_measure_perf.py xpra.test_measure_perf] script can be used to run a variety of client applications within an xpra session (or optionally vnc) and get statistics on how well the encoding performed. The data is printed out at the end in CSV format which you can then import into any tool to compare results - you can find some examples generated using [http://www.sofastatistics.com/home.php sofastats] [http://xpra.org/stats/ here].
     28It can also be useful to redirect the test's output to a log file to verify that none of the tests failed with any exceptions/errors (looking for exception messages in the log afterwards).
     29At the moment it does not have a command line interface, and all the options have to be edited directly in the source file.
     30One has to be very careful when interpreting the results, here are some examples of misleading statistics:
     31* higher CPU usage is not necessarily a bad thing if the framerate has increased
     32* lower latency is good, but not if the highest latency is up
     33* when averaging samples, all the tests must have completed (if one test run was cut short, it may be skewed)
     34* averaging between throttled and non-throttled test samples is not particularly useful
     35* averaging between many different test commands is not particularly useful, especially:
     36* gtkperf tests really skew the results and should generally be excluded from sample data (or not run at all)
     37* etc..