Subject: RE: A suggestion for the release notes?
From: WJCarpenter (bill-abisource@carpenter.ORG)
Date: Thu Nov 30 2000 - 13:52:55 CST
ben> What's involved with regression tests? I also have a plethora of
ben> platforms and OSs at my displosal. Tell me what you want!
I had in mind something like the following. Given a collection of
sample documents, someone cranks through them to make sure they load,
display, and maybe print OK. The more platforms and configuration
permuations, the better.
We have sample documents in the Abi source tree. Some are in the
"test" subdir and either illustrate a particular feature or
demonstrate a particular bug. Both are useful for regression testing.
We also have a collection of documents in the "docs" subdir that
describe various things about Abi. No particular reason to avoid
these when regression testing.
I think the above would be pretty easy for someone (who happened to
have the time) to crank through once in a while. More sophisticated
things could be done, but might take more time to implement. For
example, test documents could be constructed to trigger some kind of
expected console output in the debug builds. The extra effort comes
from the fact that you have to sift through the UT_DEBUGMSG output,
and it changes a lot due to its being an invaluable tool for ongoing
development. Still, this could lead to some labor-saving test
automation (ie, some of the tests could be batched rather than needing
human eyeballs to look at the screen to see a problem/feature).
There used to be someone (I forget who, but I think it was a
SourceGear person) who periodically wandered through Bugzilla to see
what things had been fixed, verify that fixes really were fixed, etc.
A testing volunteer could resume that role, and it would certainly be
a help to the project. As a side effect, bugs that really are bugs
could have test documents created to demonstrate them, and those test
documents could be added to the pile described above. (A typical test
document should be self-describing, as you will find in many of the
existing examples. It either has text saying something like "Notice
that the next line is displayed in the wrong font", or it has a recipe
for a set of actions that lead to some problem.)
I think all of the above are done sometimes by developers in the
normal course of events, whether the test documents get added to CVS
or remain private to the developer. The point of adding them to CVS
with a person who has regression testing as an area of interest is
that it will help catch problems accidentally created by someone
working on something other than what the test document was intending
to illustrate.
-- bill@carpenter.ORG (WJCarpenter) PGP 0x91865119 38 95 1B 69 C9 C6 3D 25 73 46 32 04 69 D6 ED F3
This archive was generated by hypermail 2b25 : Thu Nov 30 2000 - 13:48:34 CST