xpra icon
Bug tracker and wiki

Changes between Version 6 and Version 7 of Testing


Ignore:
Timestamp:
03/15/13 06:04:53 (7 years ago)
Author:
Antoine Martin
Comment:

add info on automated perf tests

Legend:

Unmodified
Added
Removed
Modified
  • Testing

    v6 v7  
    2323* CentOS 5.x clients with both old servers (CentOS 5.x) and new ones (Fedora 18+ or Debian sid)
    2424* Debian Squeeze or Ubuntu Lucid packages.
     25
     26== Automated performance and regression testing ==
     27The [http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/test_measure_perf.py xpra.test_measure_perf] script can be used to run a variety of client applications within an xpra session (or optionally vnc) and get statistics on how well the encoding performed. The data is printed out at the end in CSV format which you can then import into any tool to compare results - you can find some examples generated using [http://www.sofastatistics.com/home.php sofastats] [http://xpra.org/stats/ here].
     28It can also be useful to redirect the test's output to a log file to verify that none of the tests failed with any exceptions/errors (looking for exception messages in the log afterwards).
     29At the moment it does not have a command line interface, and all the options have to be edited directly in the source file.
     30One has to be very careful when interpreting the results, here are some examples of misleading statistics:
     31* higher CPU usage is not necessarily a bad thing if the framerate has increased
     32* lower latency is good, but not if the highest latency is up
     33* when averaging samples, all the tests must have completed (if one test run was cut short, it may be skewed)
     34* averaging between throttled and non-throttled test samples is not particularly useful
     35* averaging between many different test commands is not particularly useful, especially:
     36* gtkperf tests really skew the results and should generally be excluded from sample data (or not run at all)
     37* etc..