From: Jesper Skov (jskov@redhat.com)
Date: Sun Apr 28 2002 - 02:31:51 EDT
On Sat, 2002-04-27 at 23:15, Larry Kollar wrote:
> > When I do QA+RM for a customer for Red Hat/eCos it costs me at a minimum
> > a full days work to follow a QA sheet by hand and go through the
> > motions, do the README, etc.. We have pretty much a turn-key release
> > system, and before a release we have run 12k+ tests in our test farm.
> > It's a chore to do - but I do it because it's necessary and because I'm
> > paid to do it.
>
> A QA "sheet"? As in singular?
Yeah. Well, the "sheet" is a web page. A long one :) It would print to
multiple sheets, no doubt. But what we're testing is a bootstrap monitor
- not a big thing to test. The automated testing does most of our
coverage (luckily!)
> - Unless someone can provide us with a ready-made form
> for creating test cases, that needs to be done. I'll
> "take that action" if nothing pre-made is forthcoming.
> I'd *like* to see a web form that allows over-the-Net
> entry of test case results; or users could email them
> to abi-dev or perhaps a mailing list created for the
> purpose. (The web form could also do the mailing.)
Web form would be best; allowing a pool of people to help:
Number Test title Assigned State
...
101. Lists 0% done, 0% PASS, 0% FAIL
101.1 List creation 1 unassigned N/A
101.2 List creation 2 unassigned N/A
...
Total 76% done, 98% PASS, 2% FAIL
Clicking test title should pop up a window with a description of the
test that needs to be made, and if the test has been assigned, that page
should also have PASS/FAIL/SKIP buttons.
The assignment part would be nice to be without - but my thought is that
someone deciding to test an entire section (such as "Lists") could lock
those subtests so someone else didn't step on his toes during the
process. Maybe we could start without it, and just point people to
join #abiword-qa to coordinate before doing any QA.
Btw, I have started to play a little with the AbiCommand plugin to make
an automatic testing infrastructure. There are limits to how much can be
tested automatically though - the user interface, which is a big part of
a WP, must be tested by hand, at least.
Cheers,
Jesper
This archive was generated by hypermail 2.1.4 : Sun Apr 28 2002 - 02:33:36 EDT