| @@ -1,41 +1,41 @@ | | | @@ -1,41 +1,41 @@ |
1 | .\" $NetBSD: tests.kyua.7,v 1.3 2014/03/18 18:20:40 riastradh Exp $ | | 1 | .\" $NetBSD: tests.kyua.7,v 1.4 2014/06/02 19:17:08 gdt Exp $ |
2 | .\" | | 2 | .\" |
3 | .\" Copyright (c) 2010 The NetBSD Foundation, Inc. | | 3 | .\" Copyright (c) 2010 The NetBSD Foundation, Inc. |
4 | .\" All rights reserved. | | 4 | .\" All rights reserved. |
5 | .\" | | 5 | .\" |
6 | .\" Redistribution and use in source and binary forms, with or without | | 6 | .\" Redistribution and use in source and binary forms, with or without |
7 | .\" modification, are permitted provided that the following conditions | | 7 | .\" modification, are permitted provided that the following conditions |
8 | .\" are met: | | 8 | .\" are met: |
9 | .\" 1. Redistributions of source code must retain the above copyright | | 9 | .\" 1. Redistributions of source code must retain the above copyright |
10 | .\" notice, this list of conditions and the following disclaimer. | | 10 | .\" notice, this list of conditions and the following disclaimer. |
11 | .\" 2. Redistributions in binary form must reproduce the above copyright | | 11 | .\" 2. Redistributions in binary form must reproduce the above copyright |
12 | .\" notice, this list of conditions and the following disclaimer in the | | 12 | .\" notice, this list of conditions and the following disclaimer in the |
13 | .\" documentation and/or other materials provided with the distribution. | | 13 | .\" documentation and/or other materials provided with the distribution. |
14 | .\" | | 14 | .\" |
15 | .\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND | | 15 | .\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND |
16 | .\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, | | 16 | .\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, |
17 | .\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF | | 17 | .\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF |
18 | .\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. | | 18 | .\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. |
19 | .\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY | | 19 | .\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY |
20 | .\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL | | 20 | .\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL |
21 | .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE | | 21 | .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE |
22 | .\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS | | 22 | .\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS |
23 | .\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER | | 23 | .\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER |
24 | .\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR | | 24 | .\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR |
25 | .\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN | | 25 | .\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN |
26 | .\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. | | 26 | .\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. |
27 | .\" | | 27 | .\" |
28 | .Dd February 24, 2013 | | 28 | .Dd June 2, 2014 |
29 | .Dt TESTS 7 | | 29 | .Dt TESTS 7 |
30 | .Os | | 30 | .Os |
31 | .Sh NAME | | 31 | .Sh NAME |
32 | .Nm tests | | 32 | .Nm tests |
33 | .Nd introduction to the NetBSD test suite | | 33 | .Nd introduction to the NetBSD test suite |
34 | .Sh DESCRIPTION | | 34 | .Sh DESCRIPTION |
35 | The | | 35 | The |
36 | .Nx | | 36 | .Nx |
37 | test suite provides a collection of automated tests for two major purposes. | | 37 | test suite provides a collection of automated tests for two major purposes. |
38 | On the one hand, the test suite aids | | 38 | On the one hand, the test suite aids |
39 | .Em developers | | 39 | .Em developers |
40 | in catching bugs and regressions in the code when they performing modifications | | 40 | in catching bugs and regressions in the code when they performing modifications |
41 | to the source tree. | | 41 | to the source tree. |
| @@ -128,26 +128,41 @@ tests and show which ones failed: | | | @@ -128,26 +128,41 @@ tests and show which ones failed: |
128 | $ kyua report | | 128 | $ kyua report |
129 | .Ed | | 129 | .Ed |
130 | .Pp | | 130 | .Pp |
131 | This other example would generate an HTML report ready to be published on a | | 131 | This other example would generate an HTML report ready to be published on a |
132 | web server, possibly the built-in | | 132 | web server, possibly the built-in |
133 | .Xr httpd 8 : | | 133 | .Xr httpd 8 : |
134 | .Bd -literal -offset indent | | 134 | .Bd -literal -offset indent |
135 | $ kyua report-html --output ~/public_html/tests | | 135 | $ kyua report-html --output ~/public_html/tests |
136 | .Ed | | 136 | .Ed |
137 | .Pp | | 137 | .Pp |
138 | For further details on the command-line interface of Kyua, please refer | | 138 | For further details on the command-line interface of Kyua, please refer |
139 | to its manual page | | 139 | to its manual page |
140 | .Xr kyua 1 . | | 140 | .Xr kyua 1 . |
| | | 141 | .Ss Test environment considerations |
| | | 142 | Tests can be invoked as an unprivileged user, in which case tests that |
| | | 143 | require privileges will be skipped. |
| | | 144 | If run as root, an unprivileged user will be used for tests that |
| | | 145 | require privileges. |
| | | 146 | For maximal coverage, the standard approach is to invoke tests as root. |
| | | 147 | .Pp |
| | | 148 | Ideally, tests are self-contained and do not either depend on or |
| | | 149 | perturb the host environment, aside from skipping tests when optional |
| | | 150 | facilities are not available. |
| | | 151 | In reality, tests load and unload modules, and do other things that |
| | | 152 | might cause problems. |
| | | 153 | While it is not entirely safe to run tests on a multi-user system, |
| | | 154 | permanent problems or crashes from doing so are viewed as bugs and |
| | | 155 | should be reported. |
141 | .Ss Configuring the tests | | 156 | .Ss Configuring the tests |
142 | Some test cases in the | | 157 | Some test cases in the |
143 | .Nx | | 158 | .Nx |
144 | test suite require the administrator to manually set up some configuration | | 159 | test suite require the administrator to manually set up some configuration |
145 | properties before they can run. | | 160 | properties before they can run. |
146 | Unless these properties are defined, the tests that require them will be marked | | 161 | Unless these properties are defined, the tests that require them will be marked |
147 | as skipped and thus they will not be really executed. | | 162 | as skipped and thus they will not be really executed. |
148 | .Pp | | 163 | .Pp |
149 | Test suites are configured by defining the values to their configuration | | 164 | Test suites are configured by defining the values to their configuration |
150 | variables in | | 165 | variables in |
151 | .Pa /etc/kyua/kyua.conf . | | 166 | .Pa /etc/kyua/kyua.conf . |
152 | The format of this file is detailed in | | 167 | The format of this file is detailed in |
153 | .Xr kyua.conf 5 . | | 168 | .Xr kyua.conf 5 . |