xpra icon
Bug tracker and wiki

Changes between Version 28 and Version 29 of Testing


Ignore:
Timestamp:
11/28/15 10:07:10 (5 years ago)
Author:
Antoine Martin
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • Testing

    v28 v29  
    3636{{{#!div class="box"
    3737== Automated Performance and Regression Testing ==
    38 The [http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/test_measure_perf.py xpra.test_measure_perf] script can be used to run a variety of client applications within an xpra session (or optionally vnc) and get statistics on how well the encoding performed. The data is printed out at the end in CSV format which you can then import into any tool to compare results - you can find some examples generated using [http://www.sofastatistics.com/home.php sofastats] [http://xpra.org/stats/ here] and [/ticket/147#comment:11 here]. There is a facility for generating charts directly from the CSV data, using a script that we now provide, described below.
     38The [/browser/xpra/trunk/src/tests/xpra/test_measure_perf.py xpra.test_measure_perf] script can be used to run a variety of client applications within an xpra session (or optionally vnc) and get statistics on how well the encoding performed. The data is printed out at the end in CSV format which you can then import into any tool to compare results - you can find some examples generated using [http://www.sofastatistics.com/home.php sofastats] [http://xpra.org/stats/ here] and [/ticket/147#comment:11 here]. There is a facility for generating charts directly from the CSV data, using a script that we now provide, described below.
    3939
    4040It can also be useful to redirect the test's output to a log file to verify that none of the tests failed with any exceptions/errors (looking for exception messages in the log afterwards).
    4141
    42 At the moment it does not have a command line interface, and all the options have to be edited directly. However, we have improved that process by splitting out the configuration data into a separate file: [http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/perf_config_default.py xpra.perf_config_default.py].
     42At the moment it does not have a command line interface, and all the options have to be edited directly. However, we have improved that process by splitting out the configuration data into a separate file: [/browser/xpra/trunk/src/tests/xpra/perf_config_default.py xpra.perf_config_default.py].
    4343
    4444Note: to take advantage of iptables packet accounting (mostly for comparing with VNC which does not provide this metric), follow the error message and setup iptables rules to match the port being used in the tests, ie: by default:
     
    5757{{{#!div class="box"
    5858== Generating Charts from Performance Test Results ==
    59 To create multiple output files which can be used to generate charts, using [http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/test_measure_perf_charts.py xpra.test_measure_perf_charts]:
    60 * Build a config class by taking a copy from [http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/perf_config_default.py xpra.perf_config_default.py], then making changes as necessary.
     59To create multiple output files which can be used to generate charts, using [/browser/xpra/trunk/src/tests/xpra/test_measure_perf_charts.py xpra.test_measure_perf_charts]:
     60* Build a config class by taking a copy from [/browser/xpra/trunk/src/tests/xpra/perf_config_default.py xpra.perf_config_default.py], then making changes as necessary.
    6161* Determine the values of the following variables: prefix: (a string to identify the data set), ID: (a string to identify the variable that the data set is testing, for example '14', for testing version 14), repetitions: the number of times you want to run the tests.
    6262* The data file names you will produce will then be in the format: prefix_id_rep.csv.
     
    7373The optional arguments "1 14" and "2 14" pass the rep and ID values in to the script, which will store them in the "Custom Params" column of the data.
    7474
    75 After running your script, open your copy of [http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/test_measure_perf_charts.py xpra.test_measure_perf_charts] and change the variables at the top of the file so the script will point to the series of data files that you have generated.
     75After running your script, open your copy of [/browser/xpra/trunk/src/tests/xpra/test_measure_perf_charts.py xpra.test_measure_perf_charts] and change the variables at the top of the file so the script will point to the series of data files that you have generated.
    7676
    7777These variables are described in the script. Once this has been done, run the script on the command line, and an HTML file will be generated containing your charts.