From: Jesper Skov (jskov@redhat.com)
Date: Fri Apr 26 2002 - 03:18:59 EDT
On Thu, 2002-04-25 at 23:14, David Chart wrote:
> On Thu, 2002-04-25 at 18:28, Mike Nordell wrote:
> > Paul Rohr wrote:
> > > Discovering bad bugs at step 3 is a clear indicator that the release was
> > > *not*, in fact, ready for prime time.
> >
> > Indeed. Would it be feasible to create a "test" release before major
> > releases? It would obviously be a greater burden for all, and it would
> > probably create "test2" and so on releases, but it would probably iron out
> > the "obvious" bugs.
> >
> > Since I really don't know how much effort goes into a full release-cycle
> > it's possible this idea is in reality not possible.
> >
>
> I think the release process itself has, in the past, been far too quick.
>
> For the 0.99, 0.9, and 0.7 series, I don't think this was a serious
> problem. They were *supposed* to be development releases. Serious bugs
> were to be expected. For the 1.0 series, however, I think we need to
> slow down a bit.
>
> So, I suggest the following:
[snip very lovely description]
I only see one problem with this: we do not have anyone willing to take
on the job as QA+Release Engineering who is able to spend time on
putting a release candidate through its paces and ask developers to fix
problems found. If this is should have any chance of happening in a
timely fashion, we need someone dedicated to the task.
Do I need point out that spending scarce developer resources on that
would be a bad choice?
We'd need that person (non-programmer, but able to do a CVS checkout and
build AbiWord) to document the process so it can be handed over to
someone else should it be necessary.
Maybe we need to advertise for such a person?
And no, I really don't think using one of the few active developers for
this. Not only because it'd make a cut in the limited amount of
development time we have "assigned" (volunteered) to AbiWord, but
because, IMO, developers are pisspoor at doing a QA+RE job.
When I do QA+RM for a customer for Red Hat/eCos it costs me at a minimum
a full days work to follow a QA sheet by hand and go through the
motions, do the README, etc.. We have pretty much a turn-key release
system, and before a release we have run 12k+ tests in our test farm.
It's a chore to do - but I do it because it's necessary and because I'm
paid to do it.
Now compare that to the rush jobs we do with AbiWord. Er, rather, there
*is no comparison*. We just tag the tree (with varying success, I might
add, this should be scripted!), get people to build for all the
platforms, and hope *others* will report problems they find in time for
us to do something about it. Mostly though, when there's not a brown
paperbag issue to be addressed and we skip a release mid-stream, the
provided binaries are the release. The End.
It's not very good. Actually, it's pisspoor. We're lucky we have had so
many good releases - dumb luck!
But then, we're volunteers, and doing QA/RE *is* boring, requiring lots
of discipline. And it requires a dedicated warm body to do it.
Am I volunteering? No, just posting my observations. In short: we need
to get *way* better at handling a release now that we've gone 1.x. And
we need *someone* with time, energy and discipline (and preferrably no
programming skill) to take care of it.
Actually, I must admit that I some weeks back seriously considered doing
stable, tested binary releases of AbiWord for $1 per download. The fee
would be there to recover some of the cost of doing the QA+RE work.
Unfortunately, some personal matters have made it so that I probably
won't have the necessary time, and the Danish tax authorities are being
enough of a nuisance to take out what hope I had of making some money on
it to cover an ADSL upgrade. But it's a good idea - maybe *you* would
want to pick it up?
Jesper
This archive was generated by hypermail 2.1.4 : Fri Apr 26 2002 - 03:20:31 EDT