Re: 2.4 timetable

On Tue, 2005-07-05 at 08:55 +0100, Tomas Frydrych wrote:
>
> sum1_lists@yahoo.com wrote:
>
> > There are at least 15 open regressions in Bugzilla that were introduced
> > during 2.3.x alone; there are 81 known bugs that cause AbiWord to hang,
> > crash, or lose data; there are 897 known bugs in total, although that
> > number may be a bit high due to every bug being marked as new since the
> > recent Bugzilla upgrade.
>
> On the other hand there only something like 20 critical+ P1 & P2 bugs,

Well, it's the coder that gets to decide where any given bug lies on his
or her list of priorities, QA can only make initial suggestions to help
out with that. The coder also decides when he or she accepts a bug,
though acceptance of bugs is an oft-overlooked practice around here,
what milestone to target it at. Which, because it requires being set
explicitly (rather than the way priorities, often ignored unlike
severity, are set to P3 by default), is why I think target milestones
are important. They are, as the name suggests, absolute targets, rather
than relative qualifiers (a marking of P1 is only important to the
extent that it indicates greater importance than a bug of the _same
target_ but P2 priority). So let's look at severity, which coders tend
to pay a bit of attention to, and target milestones, which have absolute
and constant meanings.
There are 61 high severity bugs currently targetted for 2.4. That does
not count the 67 which are not targetted at all (and of course excludes
those targetted at 2.4.x or 2.6 or future). It also does not count
plugins. It also does not count the 307 non-high severity bugs which
are also *targetted for 2.4*

> number of which are only related to plugins and number of which are hard

Well, the numbers above disregard plugins (though some open plugin bugs
pertain to items that will probably be advertised as highlights of 2.4).

> to reproduce. So lets make an effective use of the bugzilla so we can

Interestingly enough, I tend to see a lot of bug-closing on account of
bugs being difficult to reproduce.

As for making effective use of the bugzilla, this is what I want to
focus on in this email...

> fix the bugs that really matter. There will always be bugs in
> application of this size, that is life; a relase will always be a
> compromise between number of factors of which bug count is, of course an
> important one.

...

>
> Actually, 2.3 was branched late, not early. The morale problem was

I believe sum1 was referring to the plan, of which you personally were a
cheerleader, relating branching to the state of bugzilla (a relatively
tiny amount of first priority immediacy-targetted bugs to guarantee that
there was merely a bare minimal level of progress on bugs rather than
the complete abandonment that actually occurred for several weeks), with
some absolute constraints in case it turned out undoable by times agreed
upon by active coders like you and martin and marc. That plan and those
dates were thrown out the window as soon as you, well, changed your
mind. Hence, early. Just a thought though, this isn't what I wanted to
focus on...

> the past experience very little work gets done on AW over the summer

And for three weeks of winter after 2.2, there wasn't a single bloody
commit that I can recall. Does this mean we should only ever release on
hubndom's birthday and on halloween? Because that'd actually be an
improvement come to think of it, and it'd be something at least slightly
more agreeable to both QA and to your solstice theory of release
engineering... Just a thought though, this isn't what I wanted to focus
on...

> months, so if we do not get 2.4 out of the way in the next couple of
> weeks, there is not going to be any significant change to the bug count

So? I think the whole crux of the argument was that it is more
important to release something that can definitively break the
reputation of instability than to release something coinciding with the
migration of middle-aged African swallows and older European ones.

> until mid autumn -- if you want 2.4 to be significantly improved against
> what 2.3 is now we are talking relase early next year on the assumption

So you're saying that if we want 2.4 to be significantly better than 2.3
is, we (I say we because it involves QA and code alike) have to keep
working as QA suggests, and if we want 2.4 to be no (or insignificantly)
better than 2.3 is, we have to do it now... Yeah, that seems to be the
gist of it (-: BTW, I'll take advantage of the current paragraph to
point out that it is no more fun (as evidenced by my shameful lack of
activity on bugzilla and scores of sticky notes to the tune of 'localize
and file such and such') to QA bugs than it is to fix them. Probably
less, and hereafter follow the theory behind that assertion. When you
fix a bug, at least, judging from my occasional experience at it, you
get a rich satisfaction of knowing that you just fixed a bug and AbiWord
is now that much more usable. When you fix a bug, QA gets only the
helpless, desperate awareness that in the course of verifying your fix,
it ran into 3 more bugs. Furthermore, that our reports will be closed
and we'll be ignored and scorned if we can't not only find and file the
bug but also make it so precisely detailed, localized, and fool-proof a
report that it won't take the coder more than 5 minutes to find and fix
it. So don't think that we constantly lobby for more bug fixing because
it's fun for us to file and localize and reproduce and retest and retest
and retest new bugs. Yeah, a lot of retesting happens because bugs go
for 3 years without being fixed. A lot of bugs also go unfiled because
they just aren't worth the trouble compared to the ones that are already
filed and still open. It's a lot more fun to play with new features for
us, too. What makes us QA is that we submit to the greater importance
of fixing a hard-to-reproduce bug in a used-by-most-people piece of code
than importance of fixing an obvious-cause bug in some obscure new
infrared-hooked-mind-reading-frame-folded-toc-partial-grammar-checked-dbus-integrated abstract generation plugin feature that only works on linux. No matter how many HOURS we spend pinning down a fool-proof testcase. But that's not what I wanted to focus on...

> So, we need to find a resonable compromise between QA point of view and
> the other factors.

Of course. And you are absolutely right about how:

On Mon, 2005-07-04 at 22:11 +0100, Tomas Frydrych wrote:
> inrows into the bug count. If you feel there are reasons not to release
> next Friday, then best if you indicate so in the bugzilla.

This is what I wanted to focus on. For years I've advocated substantive
use of bugzilla as the means by which to reconcile the oft-opposed
viewpoints of QA and coder. This is why we have severity, which is a
factor but not the sole factor in priority, priority, which takes into
account other big-picture concerns of the coder such as how many people
will actually encounter the bug, and which is relative only to bugs of
the same target milestone, and target milestone, which is the target set
by the developer (be it suggested by QA and consented to or explicitly
decided upon by the assignee) by which it is intended to have this bug
fixed.
This the theory. The difference between that methodology and what we
have now is the very reason we have a wholly-deserved reputation for
instability: the hundreds of open 2.4 bugs you're about to ignore for
2.4 that are currently or have been targetted at 2.2 or earlier. That's
not 'showed up after 2.2.0', rather it can be as much as 'reported
against 0.9, targetted at 1.0, postponed to 1.2, 2.0 passes, postponed
to 2.0.x, postponed to 2.2, postponed to 2.2.x, postponed to 2.4
(explicitly or implicitly), ignored or postponed to 2.4.x...

THE SOLUTION IN A NUTSHELL:
Assigned coders look at, read, ponder over (the whole process takes
about 30 seconds maximum per bug once you get used to it), and retarget
the bugs that they have no intention of fixing by 2.4 (Tomas, you
yourself happen to have 20 open bugs targetted for 2.4, not that I
checked or anything...)
The remaining bugs targetted for (not just at) 2.4 (that includes bugs
targetted at 2.2.x) are those the assigned coders themselves deem
necessarily fixed by 2.4, and 2.4 is not released without those fixes
until the target milestone is updated taking into account new
informatino (for example, the fix turned out to have some unexpected
caveats, that sort of thing).
QA gets its input by filing bugs, suggesting targets and priorities, and
engaging in the fluid dialogue that occurs over bugzilla whereby targets
are settled upon and priorities change to reflect the relative
bug-environment of a given milestone. Presumably, if the coders are
reading and not just mass-retargetting, it means we have some marginal
ability to get our point about AbiWord's general bugginess across to the
coders.
Coders get their input by being the final say. After all, it's whoever
fixes the bug that determines whether or not the bug gets fixed. We
(QA) recognize and respect that ability of skill and commitment of time
just as we would like for coders to recognize and respect our input and
eye for the big bug picture. If QA and coders have some irreconcilable
difference of opinion about the importance of a bug, though this should
never happen because everybody explains their reasoning in objective
detail, it is inevitably the coder that gets his way by virtue of being
the coder.

There is nothing new about this. You coders keep calling for ways to
reconcile your desire to release with our desire to give the signal
handler less use, but it's been right under your noses since 1998,
bugzilla. That's what targets are for. We tried to get this system of
automated dynamic reconciliation of differences in use with 2.4, but you
balked. Now you ask for it again. Well, here it is again. And it's
taken right out of your own messages. Retarget your bugs with
justification about why they aren't really that important. That's a
one-time thing. Then you just have to read and pay attention when you
have new bug mail so that this doesn't have to be a retrospective ordeal
with every milestone release.
As your estimations about your abilities to accomplish a given set of
fixes in a given timeframe change, the targets and priorities can adapt.
If QA suggests something and you think it's overly ambitious, say so and
suggest a more realistic timeframe, That's what all these numbers and
letters and words and symbols in bugzilla (like P1, 2.4, critical) are
for.

I'm not suggesting that you postpone 2.4 until another 800 bugs are
fixed. Neither is sum1. Nothing of the sort.
We're suggesting that you decide on a release based on reasoned criteria
with substantive significance to the program itself. All but 2.5 of
AbiWord's users see only AbiWord, not the tide-related development
activity levels of the coders. If 2.4 is based on not having severe
bugs held over for three milestones and on balancing our bug budget to
set ourselves up for greater gains in the bug department in future
milestones (by having targets that are both reasonable and paid
attention), the user community can appreciate that. If 2.4 is based on
getting another milestone out the door before the birds turn about, the
user community has nothing to anticipate from any given milestone and we
keep up our reputation of run-in-place releases.

The reasons not to release ARE indicated in bugzilla. Now the ball is
in your court. You have the opportunity to turn the age-old question of
'QA or hoo-ray' into a laughing matter of the past, and bugzilla is
there to help you do it.
Of course, by 'you', I mean the coders (myself included) with 2.4 bugs
assigned. Tomas only has 20. God help martin, but the fact is, even he
has less than an hour's worth of bugs to read, respond, and potentially
retarget from 2.4. We only ask for thought out reasoning to be the
basis of retargets, no that they always agree with ours.

As for my 1 2.4 bug, half my classes are over after this week. That
doesn't stop the work from coming, but it means that I don't allow
myself to be included in your summer-drain theory, because with the
addition of several hours and a laptop to my week, I am that much more
likely to find my way past the remaining compiler error message that
holds up the fix for it, as well as to fix other things that I've
already retargetted to 2.4.x. Not less likely.

I'd apologize for the length of this email, but you folks should be
pretty good and quick at processing words by now. Just don't hang or
abort (-:

-MG
Received on Tue Jul 5 22:39:24 2005