diff options
Diffstat (limited to 'docs/testing')
-rw-r--r-- | docs/testing | 53 |
1 files changed, 53 insertions, 0 deletions
diff --git a/docs/testing b/docs/testing new file mode 100644 index 0000000..dcfa5e2 --- /dev/null +++ b/docs/testing @@ -0,0 +1,53 @@ +== Problem: Testing == #none + + Rainbow desperately needs testing because introducing Rainbow into the + software stack causes a number of fundamental changes in the mechanism by + which activities are started. What evidence can we afford to collect on the + actual effect of the changes we're trying to introduce? + +Situation: + + There are two scenarios that we're trying to manage, namely, `stable' + and `devel'. + + `Packaged' Scenario: + + The problem here is to verify that a given version of the `olpc-security' + package is complete and correct with respect to the appropriate + `olpc-security-tests' package on a given `stable' configuration. + + In particular, we wish to verify that all the dependencies of the + olpc-security package are available in appropriate versions and that the + installation packages behave identically to the unpackaged development + code, as sampled by the tests in the `olpc-security-tests' package. + + `Unpackaged' Scenario: + + The problem is to make it fast and easy to run the tests in the + olpc-security-tests checkout with respect to the development configuration, + which consists, to a first approximation, of a possibly modified base + image, a kernel image, an initramfs, and a working-copy for each of the + dependencies of olpc-security, one of which will be a working-copy of an + olpc-security-tests package. + + The testing itself that we want to do isn't particularly hard: to a first + approximation, there are three properties we wish to test: + + item conf. type desc. + 1) (S) [I] Check that our packages cleanly install, + + 2) (S) [R] Check that all installed activities run for 30 seconds + without crashing, + + 3) (D,S) [F] Perform both positive and negative checks on our P_* + features, and later, their combinations. + +Plan: + + Herbert has shown Michael how to use QEMU to run sugar images, so we can + actually address these scenarios in a reasonable way by automating the process + of assembling the pieces. + +Followup: + + Help? |