Changes between Version 13 and Version 14 of Testing

Timestamp:

12/11/13 10:16:54 (8 years ago)

Author:

Antoine Martin

Comment:

--

Testing

@@ +1 @@
+{{{#!div class="box"
 = Testing =
 
 The current milestone can be found on the [/roadmap roadmap] page.
+}}}
 
+{{{#!div class="box"
 == Generic Regression Testing Issues ==
 * Python versions: we support Python 2.4 to 2.7 (ie: CentOS5.x for old versions of python) - code that is not backwards compatible can often make it into the repository and should be fixed before major releases (ie syntax: r2906, r2901, r2885, r2839, r2748, r2746 + r2747, r2710, r2703, r2704, r2705, r2616, r2615, r2608, r1280, r991, r990), simply compile-testing it is often enough to spot those, other issues may affect packaging (ie: #116) or also sometimes more complex (r2683), which means testing beta package builds, other bugs can be more difficult to identify and even more difficult to fix (ie: #251, #215). We also want to check that the Python 3.x version can be built via the [/browser/trunk/src/python3-build python3-build] script (compile tested), though actually using/testing it is not required at present since it isn't officially supported.
@@ -14 +17 @@
 * installations vs upgrades: sometimes this makes a difference if the OS decides to keep the old default configuration in place..
 * memory leaks (not spotted by automated tests as these do not run for long enough to trigger the problem)
+}}}
 
+
+{{{#!div class="box"
 == Specific Testing Combinations ==
 The release notes should help in figuring out what has changed and therefore what is likely to require more thorough testing.
@@ -24 +30 @@
 * CentOS 5.x clients with both old servers (CentOS 5.x) and new ones (Fedora 18+ or Debian sid)
 * Debian Squeeze or Ubuntu Lucid packages.
+}}}
 
+
+{{{#!div class="box"
 == Automated performance and regression testing ==
 The [http://xpra.org/trac/browser/xpra/trunk/src/tests/xpra/test_measure_perf.py xpra.test_measure_perf] script can be used to run a variety of client applications within an xpra session (or optionally vnc) and get statistics on how well the encoding performed. The data is printed out at the end in CSV format which you can then import into any tool to compare results - you can find some examples generated using [http://www.sofastatistics.com/home.php sofastats] [http://xpra.org/stats/ here] and [/ticket/147#comment:11 here].
@@ -40 +49 @@
 * etc..
 Please also see:
+}}}
 
+
+{{{#!div class="box"
 == Misleading Statistics ==
 One has to be very careful when interpreting performance results, here are some examples of misleading statistics:
@@ -51 +63 @@
 * the automated tests may run with the "vpx" and "x264" encodings many more times because of the many extra options that these encodings support (quality, opengl, etc..), giving more sample data, and more varied, this cannot be compared directly with other encodings without some initial filtering.
 * etc..
+}}}