1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
|
== Problem: Testing == #none
Rainbow desperately needs testing because introducing Rainbow into the
software stack causes a number of fundamental changes in the mechanism by
which activities are started. What evidence can we afford to collect on the
actual effect of the changes we're trying to introduce?
Situation:
There are two scenarios that we're trying to manage, namely, `stable'
and `devel'.
`Packaged' Scenario:
The problem here is to verify that a given version of the `olpc-security'
package is complete and correct with respect to the appropriate
`olpc-security-tests' package on a given `stable' configuration.
In particular, we wish to verify that all the dependencies of the
olpc-security package are available in appropriate versions and that the
installation packages behave identically to the unpackaged development
code, as sampled by the tests in the `olpc-security-tests' package.
`Unpackaged' Scenario:
The problem is to make it fast and easy to run the tests in the
olpc-security-tests checkout with respect to the development configuration,
which consists, to a first approximation, of a possibly modified base
image, a kernel image, an initramfs, and a working-copy for each of the
dependencies of olpc-security, one of which will be a working-copy of an
olpc-security-tests package.
The testing itself that we want to do isn't particularly hard: to a first
approximation, there are three properties we wish to test:
item conf. type desc.
1) (S) [I] Check that our packages cleanly install,
2) (S) [R] Check that all installed activities run for 30 seconds
without crashing,
3) (D,S) [F] Perform both positive and negative checks on our P_*
features, and later, their combinations.
Plan:
Herbert has shown Michael how to use QEMU to run sugar images, so we can
actually address these scenarios in a reasonable way by automating the process
of assembling the pieces.
Followup:
Help?
|
