Increasing QA contributions

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Increasing QA contributions

Ludovic Hirlimann-3
Hi,

At this moment Quality assurance is handled  by a very small group of
person (between 5 and 15) and what is mostly being done is bug triage. A
few more people show up during TestDays to tests the new features. We
have a fairly nice number of people subscribed to the
Thunderbird-testers mailing list. This mailing list is used to announce
testing events so people that don't have time to follow on a daily basis
- can easily be told when a testing event is organized.

What does QA cover :

1) Bugzilla maintainance
This is where most of the work happens at the moment. This is the part
of work where bugs are being looked at. This means figuring out if a new
bug is really i) a bug, ii) not already fixed , iii) not already in
bugzilla iv) gathering as much information so developers can understand
the issue better.
There is a backlog of bugs that need to be taken care of. This is mostly
done on Bug days.

2) Testing
a) Manual

This is organized in two ways , usually on a bug day with testing of new
features. And prior to a release we use the mozilla litmus tools to run
bug days

b) Automatic
This is mostly left to devs.

Why do we need more contributions :

1) Triaging
When Thunderbird 3 hits the streets we are going to get a significant
bunch of new bugs generated by the release and people trying it out. We
will definitively need more people to read bug and triage them. Triaging
is easy but takes time - you need to figure out how bugzilla works and
what bugs are already present in it. Ideally we would need peopel to
join this effort now so they can train themselves and be ready when 3.0
is released.

2) Testing
a) Manual
Manual testing takes time and there is no way to test all the
software/hardware configuration available out there - so the more people
to join these events the better as more issues can be found.

b) Automatic
There are two things involved in automatic testing :
        i) Porting existing test case from litmus and making sure they are
automated. Once automated the test will be run. And the time that was
used on running the test can be spent on another test.  Doing this
automation needs manpower and people willing to spend some time on it.
        ii) Adding more unit tests and writing other tests
                We have the infrastructure - but not the man power. The more of those
test we have the better each releases becomes because when things break
they are discovered a lot faster.

How do we do recruitment for new QA contributors :

in bugzilla - when people are responding from time to time we ask them
if they want to join.
in #thunderbird - when helping someone when we feel they are enthusiast
about thunderbird we will engage them and ask them if they'd like to help.

What have we tried :
  - Forums we use the Mozillazine forums, the mozilla news groups to
announce our event where people can show up an participate - this has
been going on for a Year or so and I think we've reached the people
involved in those medium.
  - Interaction with other communities I've talked to people involved
with OpenSolaris (with results - but it's a small community), and the
major linux vendors (OpenSuse, Ubuntu and Fedora) - with less success.
The later is my fault and I need to reengage the linux distros.

Things that are planned and only work in progress :
        A writing xpcshell testscript workshop - so people that are interested
in writing test, learning the internals of thunderbird and writing test
can start in a more efficient way then just starting and learning
everything by themselves.

        Being more demanding when helping users in #thunderbird.

        Changing the Welcome page for nightlies - To let our users know that we
need more manpower in QA.
       

Any ideas or thought on what could be done to increase the community
participating to our QA effort ?

Thoughts on what we've been doing up to now ?

Ludovic
--
Ludovic Hirlimann MozillaMessaging QA lead
http://www.spreadthunderbird.com/aff/79/2
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Wayne Mery
On 4/8/2009 9:11 AM, Ludovic Hirlimann wrote:

> Hi,
>
> At this moment Quality assurance is handled by a very small group of
> person (between 5 and 15) and what is mostly being done is bug triage. A
> few more people show up during TestDays to tests the new features. We
> have a fairly nice number of people subscribed to the
> Thunderbird-testers mailing list. This mailing list is used to announce
> testing events so people that don't have time to follow on a daily basis
> - can easily be told when a testing event is organized.
>
> What does QA cover :
>
> 1) Bugzilla maintainance
> This is where most of the work happens at the moment. This is the part
> of work where bugs are being looked at. This means figuring out if a new
> bug is really i) a bug, ii) not already fixed , iii) not already in
> bugzilla iv) gathering as much information so developers can understand
> the issue better.
> There is a backlog of bugs that need to be taken care of. This is mostly
> done on Bug days.
>
> 2) Testing
> a) Manual
>
> This is organized in two ways , usually on a bug day with testing of new
> features. And prior to a release we use the mozilla litmus tools to run
> bug days
>
> b) Automatic
> This is mostly left to devs.
>
> Why do we need more contributions :
>
> 1) Triaging
> When Thunderbird 3 hits the streets we are going to get a significant
> bunch of new bugs generated by the release and people trying it out. We
> will definitively need more people to read bug and triage them. Triaging
> is easy but takes time - you need to figure out how bugzilla works and
> what bugs are already present in it. Ideally we would need peopel to
> join this effort now so they can train themselves and be ready when 3.0
> is released.
>
> 2) Testing
> a) Manual
> Manual testing takes time and there is no way to test all the
> software/hardware configuration available out there - so the more people
> to join these events the better as more issues can be found.
>
> b) Automatic
> There are two things involved in automatic testing :
> i) Porting existing test case from litmus and making sure they are
> automated. Once automated the test will be run. And the time that was
> used on running the test can be spent on another test. Doing this
> automation needs manpower and people willing to spend some time on it.
> ii) Adding more unit tests and writing other tests
> We have the infrastructure - but not the man power. The more of those
> test we have the better each releases becomes because when things break
> they are discovered a lot faster.
>
> How do we do recruitment for new QA contributors :
>
> in bugzilla - when people are responding from time to time we ask them
> if they want to join.
> in #thunderbird - when helping someone when we feel they are enthusiast
> about thunderbird we will engage them and ask them if they'd like to help.
>
> What have we tried :
> - Forums we use the Mozillazine forums, the mozilla news groups to
> announce our event where people can show up an participate - this has
> been going on for a Year or so and I think we've reached the people
> involved in those medium.
> - Interaction with other communities I've talked to people involved with
> OpenSolaris (with results - but it's a small community), and the major
> linux vendors (OpenSuse, Ubuntu and Fedora) - with less success. The
> later is my fault and I need to reengage the linux distros.
>
> Things that are planned and only work in progress :
> A writing xpcshell testscript workshop - so people that are interested
> in writing test, learning the internals of thunderbird and writing test
> can start in a more efficient way then just starting and learning
> everything by themselves.
>
> Being more demanding when helping users in #thunderbird.
>
> Changing the Welcome page for nightlies - To let our users know that we
> need more manpower in QA.
>
>
> [1] Any ideas or thought on what could be done to increase the community
> participating to our QA effort ?
>
> [2] Thoughts on what we've been doing up to now ?
>
> Ludovic

As you say, we have some good improvement and we are far better off now
than last year both in QA and in volunteer coders. But over time there
is an ebb and flow to our volunteer support, as in any voluntary
project. So we do need more people.

I am perhaps the least useful to comment, having been so close to the
issue. For example, established areas like mozillazine and
mozilla.support.thunderbird have talented and dedicated people there,
and yet response there has been sparse - why I don't know. So I too am
very interested in others' comments.

Big picture ... have we sought out OSS systems that are exemplary
models? (I know I haven't.)  Are there such systems? How do they attract
and retain volunteers?

One would think and hope a new start page in beta builds appealing to
current has got to help. They obviously care enough to be testing.  But,
how to effectively motivate them out of that comfort zone?  For example,
has such a page been effective for FF?  (I don't know - but I haven't
seen a steady trickle of new FF triagers over the last year. Or, perhaps
I'm just not seeing it.)

For the (existing) community ... We now have -testers mailing list and
#tb-qa which have been great.  But it feels like we are ready for more
venues for the community - maybe something at QMO, a newsgroup/mailing
list, or something entirely new.  (new =~ FF beta facebook
http://www.facebook.com/group.php?gid=54860359570 )


Other thoughts?  New thoughts?  Happy thoughts?  Crazy thoughts?



refs:
- https://wiki.mozilla.org/Thunderbird:Testing
- https://wiki.mozilla.org/Thunderbird:Bugdays_Planning
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Wayne Mery
see also https://wiki.mozilla.org/Mozilla_Community_Action_Day_2009
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Clint Talbert-3
In reply to this post by Wayne Mery
Ludovic, Wayne,

I think these are great posts detailing the hard work of driving forward
with QA recruitment.  It's something we struggle with across the Mozilla
Project.  One thing we have to realize is that success in solving the
volunteer recruitment  problem in any area, is success in every area in
terms of visibility, word of mouth, and progress.  So, I'm always
thinking in terms of those kinds of grandiose visions (volunteers
everywhere!) when thinking about volunteers in the short term.  That
might help to explain some of the rationale behind my thoughts.  Let me
reply more specifically below....
On 4/9/09 8:35 AM, Wayne Mery wrote:

>>
>> [1] Any ideas or thought on what could be done to increase the community
>> participating to our QA effort ?
>>
yes, I'll address that.
>> [2] Thoughts on what we've been doing up to now ?
>>
I think what you're doing is good work.  Unfortunately for small teams
(and even for larger teams), the hard work of recruiting means that you
have to constantly do more than you did yesterday.  So what you're doing
now is excellent and good.  I applaud your efforts to reach into other
OSS groups and make connections there. I think that is a good idea, and
we've talked about that before.

One thing you may wish to consider is to analyze what the impact is of
each action.  Are you recruiting in everything you're doing?  Often
times, a volunteer getting involved in a project is the difference
between showing up in a channel and looking at a blank screen versus
seeing someone speaking, or even better welcoming that new person.
You're trying to make a human to human connection, and that can't be
solved with fancy bots. Every single event, every single post, every
single word you say needs to invite people into your project and into
your events.  I haven't attended a single Thunderbird event, so I don't
know how you're doing on this metric, but it is something to evaluate
yourselves against.

Another thing to do with your current array of tactics is to analyze
each one on the barrier to entry.  Unfortunately some of our tools are
in desperate need of design revisions in order to lower their barriers
to entry (examples: Litmus and Bugzilla).  Even IRC is a high barrier to
entry.  I'm pushing the FFx QA folks to work to lower some of these
barriers over the next few months.  That will have some residual help
for Tbird as well.  But I encourage you to analyze your tactics in terms
of this metric and find places to make improvements.

>
> Big picture ... have we sought out OSS systems that are exemplary
> models? (I know I haven't.) Are there such systems? How do they attract
> and retain volunteers?
I'd love to see data on this, if anyone has it.  From my experience, I
held together a volunteer calendar QA team by personal interaction.  In
its heyday, the calendar-qa channel was more akin to a living room with
big comfy couches and a roaring fire where people hung out just as much
because they enjoyed the company as they did because they enjoyed the
project and the work we did.  And the only way to get there is to
engage, engage, engage.

I've wondered lately if one of the reasons for the success I enjoyed
there was *because* I was also a volunteer.  I can't say if that
actually had an effect, but it's interesting.  I have seen quite often
in conversations that people consider the "Community" to be an amorphous
thing "out there".  That's a divisive way to think of it.  A much better
way to think of it is that we ARE the community.  Each and every one of
us.  Thinking of ourselves in the same light as the person that joined
IRC for the first time today can only do the Project a favor.  I believe
that such thinking will change the interaction dynamic in a crucial way.
>
> One would think and hope a new start page in beta builds appealing to
> current has got to help. They obviously care enough to be testing. But,
> how to effectively motivate them out of that comfort zone? For example,
> has such a page been effective for FF? (I don't know - but I haven't
> seen a steady trickle of new FF triagers over the last year. Or, perhaps
> I'm just not seeing it.)
We did change the nightly start page not that long ago.  Jay Patel might
have more data on that since I think he was driving it.  The nightly
start page has several more "Get Involved" links and it also mentions
the QA Companion add-on. This has indeed brought a few more folks our
way, but in the numbers of tens rather than hundreds.  Either we failed
to engage those folk, or they simply went their own way after their
interest waned; I can't think of a single one of them that remained
committed for any decent length of time (say a month).
>
> For the (existing) community ... We now have -testers mailing list and
> #tb-qa which have been great. But it feels like we are ready for more
> venues for the community - maybe something at QMO, a newsgroup/mailing
> list, or something entirely new. (new =~ FF beta facebook
> http://www.facebook.com/group.php?gid=54860359570 )
This is something I would suggest you improve.  I tend to think that we
have begged, cajoled and pleaded for help on all our standard fora for
so long that people don't even realize that when we say, "if you'd like
to get involved, let us know," it is actually a sincere ask.  As an
example, take a look at the Mozilla Labs group and what they are doing
in terms of building community:

* They have opened a part of the project that was always rather opaque
previously
* They have made it easy for (and continually try to improve the way in
which) people get involved in their projects.
* They blog constantly
* They are innovating and have a very high "shiny" index.  This is going
to sound snide, and it is in no way a derision on the work of Mozilla
Labs, but being the "shiny" thing is largely a matter of message and
presentation.  I mean there are limits, but those of us on the core
products can do better than we have been to date.  How exciting are your
test day blog announcements, for example?  I loved the way that you
changed the name of Bug Days at one point to Bug Love or some such...

There is no reason we can't do the same on QA.  I'm trying to push the
FFx QA team toward thinking more out of the box and doing many of the
analyses that I mentioned above.  Your project is much smaller and need
not be burdened by history or customary habits of doing things.  You
should be able to push the envelope here and start thinking about
something new.  Get on QMO (force that site to be application
agnostic!).  Try starting up modern mailing lists (google groups, yahoo
groups?) and blogging about them.  Start talking about some of the
interesting innovations (there was that timeline thing DAscher showed
once, which was cool) going on.  Innovate on new ways to do QA: Hold a
Test Day using Facebook Chat as one of your mediums.

Start doing things you never thought would work online.  One of the
biggest successes I had on calendar-qa was trying to do a "test case
writing day" online.  And the only reason it was as big as it was is
because it was unusual, and it got picked up on Slashdot where a fierce
debate ensued about whether or not random people could write test cases.

While that argument was going on, fifty people showed up and wrote 120
test cases, of which I was able to use around 110.  Many of those folks
are still involved to this day.  In fact, one of them now leads the
Calendar project. :)

I think you're doing a great job in what you're doing. I think that the
biggest win for you will be to evaluate your barriers to entry and how
well all your events and interactions are recruiting for you.  I hope
that you can use the nimbleness of your project to your advantage and
explore crazy ideas on getting people involved.  I think the Mozilla
Project has come to an age where it must branch out and re-invent the
way it recruits if it wants to keep growing.  I believe the people are
out there.  I think we're not reaching them.

I hope you show those of us on the FFx side a thing or two. :)

Best,
Clint
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

JoeS-3
In reply to this post by Wayne Mery
On 4/9/2009 11:35 AM, Wayne Mery wrote:

> On 4/8/2009 9:11 AM, Ludovic Hirlimann wrote:
>> Hi,
>>
>> At this moment Quality assurance is handled by a very small group of
>> person (between 5 and 15) and what is mostly being done is bug triage. A
>> few more people show up during TestDays to tests the new features. We
>> have a fairly nice number of people subscribed to the
>> Thunderbird-testers mailing list. This mailing list is used to announce
>> testing events so people that don't have time to follow on a daily basis
>> - can easily be told when a testing event is organized.
>>
>> What does QA cover :
>>
>> 1) Bugzilla maintainance
>> This is where most of the work happens at the moment. This is the part
>> of work where bugs are being looked at. This means figuring out if a new
>> bug is really i) a bug, ii) not already fixed , iii) not already in
>> bugzilla iv) gathering as much information so developers can understand
>> the issue better.
>> There is a backlog of bugs that need to be taken care of. This is mostly
>> done on Bug days.
>>
>> 2) Testing
>> a) Manual
>>
>> This is organized in two ways , usually on a bug day with testing of new
>> features. And prior to a release we use the mozilla litmus tools to run
>> bug days
>>
>> b) Automatic
>> This is mostly left to devs.
>>
>> Why do we need more contributions :
>>
>> 1) Triaging
>> When Thunderbird 3 hits the streets we are going to get a significant
>> bunch of new bugs generated by the release and people trying it out. We
>> will definitively need more people to read bug and triage them. Triaging
>> is easy but takes time - you need to figure out how bugzilla works and
>> what bugs are already present in it. Ideally we would need peopel to
>> join this effort now so they can train themselves and be ready when 3.0
>> is released.
>>
>> 2) Testing
>> a) Manual
>> Manual testing takes time and there is no way to test all the
>> software/hardware configuration available out there - so the more people
>> to join these events the better as more issues can be found.
>>
>> b) Automatic
>> There are two things involved in automatic testing :
>> i) Porting existing test case from litmus and making sure they are
>> automated. Once automated the test will be run. And the time that was
>> used on running the test can be spent on another test. Doing this
>> automation needs manpower and people willing to spend some time on it.
>> ii) Adding more unit tests and writing other tests
>> We have the infrastructure - but not the man power. The more of those
>> test we have the better each releases becomes because when things break
>> they are discovered a lot faster.
>>
>> How do we do recruitment for new QA contributors :
>>
>> in bugzilla - when people are responding from time to time we ask them
>> if they want to join.
>> in #thunderbird - when helping someone when we feel they are enthusiast
>> about thunderbird we will engage them and ask them if they'd like to
>> help.
>>
>> What have we tried :
>> - Forums we use the Mozillazine forums, the mozilla news groups to
>> announce our event where people can show up an participate - this has
>> been going on for a Year or so and I think we've reached the people
>> involved in those medium.
>> - Interaction with other communities I've talked to people involved with
>> OpenSolaris (with results - but it's a small community), and the major
>> linux vendors (OpenSuse, Ubuntu and Fedora) - with less success. The
>> later is my fault and I need to reengage the linux distros.
>>
>> Things that are planned and only work in progress :
>> A writing xpcshell testscript workshop - so people that are interested
>> in writing test, learning the internals of thunderbird and writing test
>> can start in a more efficient way then just starting and learning
>> everything by themselves.
>>
>> Being more demanding when helping users in #thunderbird.
>>
>> Changing the Welcome page for nightlies - To let our users know that we
>> need more manpower in QA.
>>
>>
>> [1] Any ideas or thought on what could be done to increase the community
>> participating to our QA effort ?
>>
>> [2] Thoughts on what we've been doing up to now ?
>>
>> Ludovic
>
> As you say, we have some good improvement and we are far better off now
> than last year both in QA and in volunteer coders. But over time there
> is an ebb and flow to our volunteer support, as in any voluntary
> project. So we do need more people.
>
> I am perhaps the least useful to comment, having been so close to the
> issue. For example, established areas like mozillazine and
> mozilla.support.thunderbird have talented and dedicated people there,
> and yet response there has been sparse - why I don't know. So I too am
> very interested in others' comments.
>
> Big picture ... have we sought out OSS systems that are exemplary
> models? (I know I haven't.) Are there such systems? How do they attract
> and retain volunteers?
>
> One would think and hope a new start page in beta builds appealing to
> current has got to help. They obviously care enough to be testing. But,
> how to effectively motivate them out of that comfort zone? For example,
> has such a page been effective for FF? (I don't know - but I haven't
> seen a steady trickle of new FF triagers over the last year. Or, perhaps
> I'm just not seeing it.)
>
> For the (existing) community ... We now have -testers mailing list and
> #tb-qa which have been great. But it feels like we are ready for more
> venues for the community - maybe something at QMO, a newsgroup/mailing
> list, or something entirely new. (new =~ FF beta facebook
> http://www.facebook.com/group.php?gid=54860359570 )
>
>
> Other thoughts? New thoughts? Happy thoughts? Crazy thoughts?
>
>
>
> refs:
> - https://wiki.mozilla.org/Thunderbird:Testing
> - https://wiki.mozilla.org/Thunderbird:Bugdays_Planning

Ok
Other thoughts

Folks I know are not really "onboard" with the current development trend, so why should they contribute their time.
They have many ideas for what they envision TB3 to be, but they think their ideas fall on deaf ears.
And I guess, when "a few folks" formulate a plan, that's to be expected.
Yeah, I know TB dev is not a Democracy, but even so, let's give a listen to the minority.

And so, I propose "Bitch Day"

We could announce this in Mozilazine forums, and the newsgroups, mozilla.support.thunderbird (and others)

Topics:
Why don't you use bugzilla
Have you tried a TB3 Beta
IRC as a way to communicate



I would be willing to try to consolidate the remarks in a post here.

You might not like what you hear, but it should be a very real "Community response"

JoeS





_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Bugzilla from nth10sd@gmail.com
In reply to this post by Ludovic Hirlimann-3
I won't exactly reiterate the points covered by Ludo, Wayne and Clint
here other than sharing virtually identical sentiments - the key point
is, we would need to reach out to others "not in the process" and find
out how we can get them in.

Clint's example of being Slashdotted is a good one, though it really
depends on luck to see if we can get Slashdotted, that in itself may
turn out beneficial, or it might just even turn out flame-able.

Somehow I do agree that IRC in itself is a high level of entry, people
want simple access to help or suggest or contribute, something along the
likes of the community helping itself out, like the SUMO
help-each-other-out-web-talk thing, might work, or might not.

That said, TB QA has come quite a ways since a year plus ago - it's
about time to consider how to move on in a scalable manner, such that it
almost becomes self-sustaining.


-Gary
nth10sd
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Ludovic Hirlimann-3
In reply to this post by JoeS-3
On 4/10/09 5:36 AM, JoeS wrote:

>
> And so, I propose "Bitch Day"
>
> We could announce this in Mozilazine forums, and the newsgroups,
> mozilla.support.thunderbird (and others)
>
> Topics:
> Why don't you use bugzilla
> Have you tried a TB3 Beta
> IRC as a way to communicate
>
>
>
> I would be willing to try to consolidate the remarks in a post here.

Joe this is a great way to get feedback. Thanks for all those ideas.

Ludo
--
Ludovic Hirlimann MozillaMessaging QA lead
http://www.spreadthunderbird.com/aff/79/2
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Prescience500 (Bugzilla)
In reply to this post by Clint Talbert-3
On 4/9/2009 3:34 PM, Clint Talbert wrote:

> Ludovic, Wayne,
>
> I think these are great posts detailing the hard work of driving forward
> with QA recruitment. It's something we struggle with across the Mozilla
> Project. One thing we have to realize is that success in solving the
> volunteer recruitment problem in any area, is success in every area in
> terms of visibility, word of mouth, and progress. So, I'm always
> thinking in terms of those kinds of grandiose visions (volunteers
> everywhere!) when thinking about volunteers in the short term. That
> might help to explain some of the rationale behind my thoughts. Let me
> reply more specifically below....
> On 4/9/09 8:35 AM, Wayne Mery wrote:
>
>>>
>>> [1] Any ideas or thought on what could be done to increase the community
>>> participating to our QA effort ?
>>>
> yes, I'll address that.
>>> [2] Thoughts on what we've been doing up to now ?
>>>
> I think what you're doing is good work. Unfortunately for small teams
> (and even for larger teams), the hard work of recruiting means that you
> have to constantly do more than you did yesterday. So what you're doing
> now is excellent and good. I applaud your efforts to reach into other
> OSS groups and make connections there. I think that is a good idea, and
> we've talked about that before.
>
> One thing you may wish to consider is to analyze what the impact is of
> each action. Are you recruiting in everything you're doing? Often times,
> a volunteer getting involved in a project is the difference between
> showing up in a channel and looking at a blank screen versus seeing
> someone speaking, or even better welcoming that new person. You're
> trying to make a human to human connection, and that can't be solved
> with fancy bots. Every single event, every single post, every single
> word you say needs to invite people into your project and into your
> events. I haven't attended a single Thunderbird event, so I don't know
> how you're doing on this metric, but it is something to evaluate
> yourselves against.
>
> Another thing to do with your current array of tactics is to analyze
> each one on the barrier to entry. Unfortunately some of our tools are in
> desperate need of design revisions in order to lower their barriers to
> entry (examples: Litmus and Bugzilla). Even IRC is a high barrier to
> entry. I'm pushing the FFx QA folks to work to lower some of these
> barriers over the next few months. That will have some residual help for
> Tbird as well. But I encourage you to analyze your tactics in terms of
> this metric and find places to make improvements.
>
>>
>> Big picture ... have we sought out OSS systems that are exemplary
>> models? (I know I haven't.) Are there such systems? How do they attract
>> and retain volunteers?
> I'd love to see data on this, if anyone has it. From my experience, I
> held together a volunteer calendar QA team by personal interaction. In
> its heyday, the calendar-qa channel was more akin to a living room with
> big comfy couches and a roaring fire where people hung out just as much
> because they enjoyed the company as they did because they enjoyed the
> project and the work we did. And the only way to get there is to engage,
> engage, engage.
>
> I've wondered lately if one of the reasons for the success I enjoyed
> there was *because* I was also a volunteer. I can't say if that actually
> had an effect, but it's interesting. I have seen quite often in
> conversations that people consider the "Community" to be an amorphous
> thing "out there". That's a divisive way to think of it. A much better
> way to think of it is that we ARE the community. Each and every one of
> us. Thinking of ourselves in the same light as the person that joined
> IRC for the first time today can only do the Project a favor. I believe
> that such thinking will change the interaction dynamic in a crucial way.
>>
>> One would think and hope a new start page in beta builds appealing to
>> current has got to help. They obviously care enough to be testing. But,
>> how to effectively motivate them out of that comfort zone? For example,
>> has such a page been effective for FF? (I don't know - but I haven't
>> seen a steady trickle of new FF triagers over the last year. Or, perhaps
>> I'm just not seeing it.)
> We did change the nightly start page not that long ago. Jay Patel might
> have more data on that since I think he was driving it. The nightly
> start page has several more "Get Involved" links and it also mentions
> the QA Companion add-on. This has indeed brought a few more folks our
> way, but in the numbers of tens rather than hundreds. Either we failed
> to engage those folk, or they simply went their own way after their
> interest waned; I can't think of a single one of them that remained
> committed for any decent length of time (say a month).
>>
>> For the (existing) community ... We now have -testers mailing list and
>> #tb-qa which have been great. But it feels like we are ready for more
>> venues for the community - maybe something at QMO, a newsgroup/mailing
>> list, or something entirely new. (new =~ FF beta facebook
>> http://www.facebook.com/group.php?gid=54860359570 )
> This is something I would suggest you improve. I tend to think that we
> have begged, cajoled and pleaded for help on all our standard fora for
> so long that people don't even realize that when we say, "if you'd like
> to get involved, let us know," it is actually a sincere ask. As an
> example, take a look at the Mozilla Labs group and what they are doing
> in terms of building community:
>
> * They have opened a part of the project that was always rather opaque
> previously
> * They have made it easy for (and continually try to improve the way in
> which) people get involved in their projects.
> * They blog constantly
> * They are innovating and have a very high "shiny" index. This is going
> to sound snide, and it is in no way a derision on the work of Mozilla
> Labs, but being the "shiny" thing is largely a matter of message and
> presentation. I mean there are limits, but those of us on the core
> products can do better than we have been to date. How exciting are your
> test day blog announcements, for example? I loved the way that you
> changed the name of Bug Days at one point to Bug Love or some such...
>
> There is no reason we can't do the same on QA. I'm trying to push the
> FFx QA team toward thinking more out of the box and doing many of the
> analyses that I mentioned above. Your project is much smaller and need
> not be burdened by history or customary habits of doing things. You
> should be able to push the envelope here and start thinking about
> something new. Get on QMO (force that site to be application agnostic!).
> Try starting up modern mailing lists (google groups, yahoo groups?) and
> blogging about them. Start talking about some of the interesting
> innovations (there was that timeline thing DAscher showed once, which
> was cool) going on. Innovate on new ways to do QA: Hold a Test Day using
> Facebook Chat as one of your mediums.
>
> Start doing things you never thought would work online. One of the
> biggest successes I had on calendar-qa was trying to do a "test case
> writing day" online. And the only reason it was as big as it was is
> because it was unusual, and it got picked up on Slashdot where a fierce
> debate ensued about whether or not random people could write test cases.
>
> While that argument was going on, fifty people showed up and wrote 120
> test cases, of which I was able to use around 110. Many of those folks
> are still involved to this day. In fact, one of them now leads the
> Calendar project. :)
>
> I think you're doing a great job in what you're doing. I think that the
> biggest win for you will be to evaluate your barriers to entry and how
> well all your events and interactions are recruiting for you. I hope
> that you can use the nimbleness of your project to your advantage and
> explore crazy ideas on getting people involved. I think the Mozilla
> Project has come to an age where it must branch out and re-invent the
> way it recruits if it wants to keep growing. I believe the people are
> out there. I think we're not reaching them.
>
> I hope you show those of us on the FFx side a thing or two. :)
>
> Best,
> Clint

I agree that barriers to entry is a problem, at least for me. I'm a huge
Mozilla supporter and for a while I had been trying to get involved with
some QA stuff for Thunderbird, Firefox, and Lightning. I was eventually
discouraged because I didn't have privileges in Bugzilla to make
changes. I looked up how to get the permissions (which took me a great
deal of time to find) and I felt that the process used to get Bugzilla
permissions and everything was kind of discouraging. If there were other
ways to get Bugzilla privileges that didn't make it seem so exclusive,
was more clear on how to get them, and it was easier to find I think you
would be getting more non-programmers, like myself, to contribute.

Thanks,
Michael
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Wayne Mery
On 4/10/2009 2:29 PM, Michael Burns wrote:

> On 4/9/2009 3:34 PM, Clint Talbert wrote:
>> Ludovic, Wayne,
>>
>> I think these are great posts detailing the hard work of driving forward
>> with QA recruitment. It's something we struggle with across the Mozilla
>> Project. One thing we have to realize is that success in solving the
>> volunteer recruitment problem in any area, is success in every area in
>> terms of visibility, word of mouth, and progress. So, I'm always
>> thinking in terms of those kinds of grandiose visions (volunteers
>> everywhere!) when thinking about volunteers in the short term. That
>> might help to explain some of the rationale behind my thoughts. Let me
>> reply more specifically below....
>> On 4/9/09 8:35 AM, Wayne Mery wrote:
>>
>>>>
>>>> [1] Any ideas or thought on what could be done to increase the
>>>> community
>>>> participating to our QA effort ?
>>>>
>> yes, I'll address that.
>>>> [2] Thoughts on what we've been doing up to now ?
>>>>
>> I think what you're doing is good work. Unfortunately for small teams
>> (and even for larger teams), the hard work of recruiting means that you
>> have to constantly do more than you did yesterday. So what you're doing
>> now is excellent and good. I applaud your efforts to reach into other
>> OSS groups and make connections there. I think that is a good idea, and
>> we've talked about that before.
>>
>> One thing you may wish to consider is to analyze what the impact is of
>> each action. Are you recruiting in everything you're doing? Often times,
>> a volunteer getting involved in a project is the difference between
>> showing up in a channel and looking at a blank screen versus seeing
>> someone speaking, or even better welcoming that new person. You're
>> trying to make a human to human connection, and that can't be solved
>> with fancy bots. Every single event, every single post, every single
>> word you say needs to invite people into your project and into your
>> events. I haven't attended a single Thunderbird event, so I don't know
>> how you're doing on this metric, but it is something to evaluate
>> yourselves against.
>>
>> Another thing to do with your current array of tactics is to analyze
>> each one on the barrier to entry. Unfortunately some of our tools are in
>> desperate need of design revisions in order to lower their barriers to
>> entry (examples: Litmus and Bugzilla). Even IRC is a high barrier to
>> entry. I'm pushing the FFx QA folks to work to lower some of these
>> barriers over the next few months. That will have some residual help for
>> Tbird as well. But I encourage you to analyze your tactics in terms of
>> this metric and find places to make improvements.
>>
>>>
>>> Big picture ... have we sought out OSS systems that are exemplary
>>> models? (I know I haven't.) Are there such systems? How do they attract
>>> and retain volunteers?
>> I'd love to see data on this, if anyone has it. From my experience, I
>> held together a volunteer calendar QA team by personal interaction. In
>> its heyday, the calendar-qa channel was more akin to a living room with
>> big comfy couches and a roaring fire where people hung out just as much
>> because they enjoyed the company as they did because they enjoyed the
>> project and the work we did. And the only way to get there is to engage,
>> engage, engage.
>>
>> I've wondered lately if one of the reasons for the success I enjoyed
>> there was *because* I was also a volunteer. I can't say if that actually
>> had an effect, but it's interesting. I have seen quite often in
>> conversations that people consider the "Community" to be an amorphous
>> thing "out there". That's a divisive way to think of it. A much better
>> way to think of it is that we ARE the community. Each and every one of
>> us. Thinking of ourselves in the same light as the person that joined
>> IRC for the first time today can only do the Project a favor. I believe
>> that such thinking will change the interaction dynamic in a crucial way.
>>>
>>> One would think and hope a new start page in beta builds appealing to
>>> current has got to help. They obviously care enough to be testing. But,
>>> how to effectively motivate them out of that comfort zone? For example,
>>> has such a page been effective for FF? (I don't know - but I haven't
>>> seen a steady trickle of new FF triagers over the last year. Or, perhaps
>>> I'm just not seeing it.)
>> We did change the nightly start page not that long ago. Jay Patel might
>> have more data on that since I think he was driving it. The nightly
>> start page has several more "Get Involved" links and it also mentions
>> the QA Companion add-on. This has indeed brought a few more folks our
>> way, but in the numbers of tens rather than hundreds. Either we failed
>> to engage those folk, or they simply went their own way after their
>> interest waned; I can't think of a single one of them that remained
>> committed for any decent length of time (say a month).
>>>
>>> For the (existing) community ... We now have -testers mailing list and
>>> #tb-qa which have been great. But it feels like we are ready for more
>>> venues for the community - maybe something at QMO, a newsgroup/mailing
>>> list, or something entirely new. (new =~ FF beta facebook
>>> http://www.facebook.com/group.php?gid=54860359570 )
>> This is something I would suggest you improve. I tend to think that we
>> have begged, cajoled and pleaded for help on all our standard fora for
>> so long that people don't even realize that when we say, "if you'd like
>> to get involved, let us know," it is actually a sincere ask. As an
>> example, take a look at the Mozilla Labs group and what they are doing
>> in terms of building community:
>>
>> * They have opened a part of the project that was always rather opaque
>> previously
>> * They have made it easy for (and continually try to improve the way in
>> which) people get involved in their projects.
>> * They blog constantly
>> * They are innovating and have a very high "shiny" index. This is going
>> to sound snide, and it is in no way a derision on the work of Mozilla
>> Labs, but being the "shiny" thing is largely a matter of message and
>> presentation. I mean there are limits, but those of us on the core
>> products can do better than we have been to date. How exciting are your
>> test day blog announcements, for example? I loved the way that you
>> changed the name of Bug Days at one point to Bug Love or some such...
>>
>> There is no reason we can't do the same on QA. I'm trying to push the
>> FFx QA team toward thinking more out of the box and doing many of the
>> analyses that I mentioned above. Your project is much smaller and need
>> not be burdened by history or customary habits of doing things. You
>> should be able to push the envelope here and start thinking about
>> something new. Get on QMO (force that site to be application agnostic!).
>> Try starting up modern mailing lists (google groups, yahoo groups?) and
>> blogging about them. Start talking about some of the interesting
>> innovations (there was that timeline thing DAscher showed once, which
>> was cool) going on. Innovate on new ways to do QA: Hold a Test Day using
>> Facebook Chat as one of your mediums.
>>
>> Start doing things you never thought would work online. One of the
>> biggest successes I had on calendar-qa was trying to do a "test case
>> writing day" online. And the only reason it was as big as it was is
>> because it was unusual, and it got picked up on Slashdot where a fierce
>> debate ensued about whether or not random people could write test cases.
>>
>> While that argument was going on, fifty people showed up and wrote 120
>> test cases, of which I was able to use around 110. Many of those folks
>> are still involved to this day. In fact, one of them now leads the
>> Calendar project. :)
>>
>> I think you're doing a great job in what you're doing. I think that the
>> biggest win for you will be to evaluate your barriers to entry and how
>> well all your events and interactions are recruiting for you. I hope
>> that you can use the nimbleness of your project to your advantage and
>> explore crazy ideas on getting people involved. I think the Mozilla
>> Project has come to an age where it must branch out and re-invent the
>> way it recruits if it wants to keep growing. I believe the people are
>> out there. I think we're not reaching them.
>>
>> I hope you show those of us on the FFx side a thing or two. :)
>>
>> Best,
>> Clint
>
> I agree that barriers to entry is a problem, at least for me. I'm a huge
> Mozilla supporter and for a while I had been trying to get involved with
> some QA stuff for Thunderbird, Firefox, and Lightning. I was eventually
> discouraged because I didn't have privileges in Bugzilla to make
> changes. I looked up how to get the permissions (which took me a great
> deal of time to find) and I felt that the process used to get Bugzilla
> permissions and everything was kind of discouraging. If there were other
> ways to get Bugzilla privileges that didn't make it seem so exclusive,
> was more clear on how to get them, and it was easier to find I think you
> would be getting more non-programmers, like myself, to contribute.
>
> Thanks,
> Michael

Hi Michael. Thanks for commenting.

Perhaps you read a description that was complex, but the process is
simple and certainly not exclusive.  But it is necessary - one cites a
few good examples of what they have done in bugzilla. If you have a
bugzilla account, and have commented intelligently in some bugs, chances
are very good that you already demonstrated your abilities.

Please see the description at
https://wiki.mozilla.org/Thunderbird:Bug_Triage#Canconfirm_and_Editbugs_Privileges

Do you see anything that can be improved there?
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Magnus Melin
In reply to this post by Ludovic Hirlimann-3
On 2009-04-08 16:11, Ludovic Hirlimann wrote:
> Any ideas or thought on what could be done to increase the community
> participating to our QA effort ?
>
> Thoughts on what we've been doing up to now ?

Not specific to QA but what people want is for their work to have an impact.
It's rather obvious but is worth thinking about once in a while.

One thing I'd like for more new and old QA people to do is to comment in bug
reports also if they can't reproduce. Some bugs are simple to follow the steps
to reproduce for but it takes time. That someone tried to reproduce and
  a) were able to follow the steps
  b) reproduced/didn't reproduce

... is fairly valuable information, and is something most people with incentive
and a nightly build are able to do to help out.

  -Magnus

_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

R Kent James
On 4/15/2009 12:02 PM, Magnus Melin wrote:

>
> One thing I'd like for more new and old QA people to do is to comment in
> bug reports also if they can't reproduce. Some bugs are simple to follow
> the steps to reproduce for but it takes time. That someone tried to
> reproduce and
> a) were able to follow the steps
> b) reproduced/didn't reproduce
>
> ... is fairly valuable information, and is something most people with
> incentive and a nightly build are able to do to help out.
>

I confess that I have frequently gone to a bug, tried to reproduce it
and failed, then moved on without comment. My case is a little different
though, because I am usually looking for bugs that are clear enough for
me to actually fix.

rkent
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Nelson Bolyard-3
In reply to this post by Ludovic Hirlimann-3
Ludovic Hirlimann wrote, On 2009-04-08 06:11:

> Changing the Welcome page for nightlies - To let our users know that we
> need more manpower in QA.
>
> Any ideas or thought on what could be done to increase the community
> participating to our QA effort ?
>
> Thoughts on what we've been doing up to now ?

I used to help, but I've stopped.  It was way too discouraging.
Not enough bugs get fixed, and it takes way too long.  On some bugs, the
only activity they've seen over the course of several years is a periodic
ping ("Is this bug still reproducible?").  It's bad enough when that
happens once.  When it happens more than once ...  Then there are bugs where
the duplicates have piled on and piled up, but they just live on, with no
relief in sight.

I finally decided that it was better to just stick with an old verion and
live with its known bugs than to keep trying nightlies, in hope of getting
fixes, only to constantly encounter new issues.

:(
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Bugzilla from nth10sd@gmail.com
On 4/17/09 1:40 AM, Nelson Bolyard wrote:

> I used to help, but I've stopped.  It was way too discouraging.
> Not enough bugs get fixed, and it takes way too long.  On some bugs, the
> only activity they've seen over the course of several years is a periodic
> ping ("Is this bug still reproducible?").  It's bad enough when that
> happens once.  When it happens more than once ...  Then there are bugs where
> the duplicates have piled on and piled up, but they just live on, with no
> relief in sight.
>
> I finally decided that it was better to just stick with an old verion and
> live with its known bugs than to keep trying nightlies, in hope of getting
> fixes, only to constantly encounter new issues.
>
> :(

The situation now is much better than before, the community is
revitalizing. While I would agree that the situation was indeed bad
before, Thunderbird QA is now getting back up to speed, as well as
development.

Personally, I would encourage you to "get your feet wet" again - the TB3
betas are looking to be a release with many improvements over the TB2
series. However, as with all software in the world, there will still be
bugs and issues - please do consider reporting and helping triage again.

Regards,
-Gary
nth10sd
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Robert Kaiser
In reply to this post by Nelson Bolyard-3
Nelson Bolyard wrote:
> On some bugs, the
> only activity they've seen over the course of several years is a periodic
> ping ("Is this bug still reproducible?").

Bugs like that should probably just be closed. That could also mean
moving it to UNCO and auto-resolving UNCO to INCOMPLETE or so after some
time without activity, as some Firefox people proposed in the thread to
change Bugzilla workflow.

Robert Kaiser
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Ludovic Hirlimann-3
On 4/17/09 3:55 PM, Robert Kaiser wrote:
> Nelson Bolyard wrote:
>> On some bugs, the
>> only activity they've seen over the course of several years is a periodic
>> ping ("Is this bug still reproducible?").
>
> Bugs like that should probably just be closed. That could also mean

I disagree there. When I usually ping - it's because I can't reproduce
the bug - and that the reporter has been active - pinging is a good way
for me to know if the bug as been resolved by another commit.

> moving it to UNCO and auto-resolving UNCO to INCOMPLETE or so after some
> time without activity, as some Firefox people proposed in the thread to
> change Bugzilla workflow.

That's already being done - when the bug reporter does not react we use
the status whiteboard [closeme yyyy-mm-dd] and we try to have that date
match a Thursday so they end up being closed by Joshua when he does the
closeme rounds on bugdays.


--
Ludovic Hirlimann MozillaMessaging QA lead
http://www.spreadthunderbird.com/aff/79/2
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Ludovic Hirlimann-3
In reply to this post by Nelson Bolyard-3
On 4/16/09 7:40 PM, Nelson Bolyard wrote:

> Ludovic Hirlimann wrote, On 2009-04-08 06:11:
>
>> Changing the Welcome page for nightlies - To let our users know that we
>> need more manpower in QA.
>>
>> Any ideas or thought on what could be done to increase the community
>> participating to our QA effort ?
>>
>> Thoughts on what we've been doing up to now ?
>
> I used to help, but I've stopped.  It was way too discouraging.
> Not enough bugs get fixed, and it takes way too long.  On some bugs, the

That as changed over the course of the last year - Issue is the backlog
of bugs to fix is huge so getting down to that will take some time. You
of course know that patches are welcome :-)


> I finally decided that it was better to just stick with an old verion and
> live with its known bugs than to keep trying nightlies, in hope of getting
> fixes, only to constantly encounter new issues.

So what is your strategy when switching to major new versions ? You wait
for the final version to be released or you use beta's before the
version becomes final. Would help if you would start using TB3.0b3 for
instance.

Ludo
--
Ludovic Hirlimann MozillaMessaging QA lead
http://www.spreadthunderbird.com/aff/79/2
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Nelson Bolyard-3
Ludovic Hirlimann wrote, On 2009-04-17 08:16:

> So what is your strategy when switching to major new versions ? You wait
> for the final version to be released or you use beta's before the
> version becomes final. Would help if you would start using TB3.0b3 for
> instance.

For years, I ran trunk nightlies.  I was way ahead of the betas or even
alphas.  But over time, I became SO frustrated with it, and with the large
number of fundamental usability bugs that have gone unfixed for years,
and with the number of regressions (!), that I asked myself, why am I
torturing myself like this?

So, my strategy, in answer to your question is this.  I am still running a
nightly trunk build from late last year.  When the new release finally
releases, I will try it, and see if the regressions are still there.
(Can you "paste as quotation" from the keyboard in TB3 betas yet?
You could in all versions of TB1 and TB2.)  If not, then I will stick with
that version until its dying day.  By sticking with one version from the
days of "early adopter" to the days of "late relenquisher", I will minimize
the number of surprises.

These are a few of my favorite bugs/RFEs (notice the 5 digit ones)
      RFE
 10097 * Ability to add user to 'killfile' with a single key stroke
 28211   On Send, fail to copy to Sent folder should not lose sent msg
 43278   Crossposts (same Message-ID) not marked as read in other groups
 86607 * UI option for disabling format=flowed
106518   "create filter" from a "To:" email address in the message pane
         creates a "From" filter for the address
108650   Unread counts toggle back and forth between twisty open and
         selection.
178870 * make "Run Filters on Folder" available for newsgroups
187997   Rewrap doesn't work with selected text
191881   Rewrap adds space at beginning of quoted line when previous line
         is almost full length
264981 * support HTTP proxy connect feature for proxying mail/news protos
285399   Reply quotes entire paragraph as a single line
288700 * Handle delete/detach attachment feature with crypto-signed mails
297314 * Allow detach/delete of attachments in signed/encrypted messages
354272 * Need feature to delete & prevent duplicate messages in folders
383985   Order of threads in msg list pane should not be controlled by Date
         Sent vs Date Received
388242   SMTP Mail Server Password dialog is modal to the wrong window
446228   rebuilding summary file doesn't fix newsgroup message counts
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Peter Lairo-3
On 18.04.2009 10:59, Nelson Bolyard wrote:
> Can you "paste as quotation" from the keyboard in TB3 betas yet?

https://bugzilla.mozilla.org/show_bug.cgi?id=461117
--
Regards,

Peter Lairo

The browser you can trust:  www.Firefox.com
Reclaim Your Inbox:         www.GetThunderbird.com

Islam:  http://www.jihadwatch.org/islam101/
Israel: http://www.jewishvirtuallibrary.org/jsource/myths2/
Church of the Flying Spaghetti Monster:  http://www.venganza.org/

"So, why don't we ever talk about the sun's contribution to global
warming? Well, because we can't regulate it, tax it, or make it feel
guilty for what it's doing" (www.WhatYouOughtToKnow.com)
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Wayne Mery
In reply to this post by Ludovic Hirlimann-3
On 4/17/2009 11:02 AM, Ludovic Hirlimann wrote:

> On 4/17/09 3:55 PM, Robert Kaiser wrote:
>> Nelson Bolyard wrote:
>>> On some bugs, the
>>> only activity they've seen over the course of several years is a
>>> periodic
>>> ping ("Is this bug still reproducible?").
>>
>> Bugs like that should probably just be closed. That could also mean
>
> I disagree there. When I usually ping - it's because I can't reproduce
> the bug - and that the reporter has been active - pinging is a good way
> for me to know if the bug as been resolved by another commit.

Yes it can seem repetitive and therefore may be discouraging if there
are multiple "asks" - it may not seem helpful to just ping when a new
release comes out, or when someone (newbie or not) comes on the scene
and pings without commenting as to whether they tested.

But related to Ludo's point, and not necessarily speaking directly to
Nelson's situation since 2+ years is eons ago, time/value should be an
important factor in what happens in the bug.

For example I tend to test only in cases where I think I have decent
odds of proving whether or not it's a bug or it's reproducible. (Why
take 5-20 minutes attempting to reproduce and not succeed and therefore
prove not much?)  And in cases where I don't test two choices remain:
  1. take a pass - not comment and hope someone else helps in the next
few months (the odds of the later happening are better now than last
year, but still not fantastic), or
  2. ping reporter and see if we can't make some progress through
further reporter testing and followup

So most often I opt for #2. In 30 seconds I can write something simple,
I don't throw away the time I already invested in reading/thinking about
the bug, and in the "saved time" I can be helpful in 5+ more bugs where
I might be more productive.

Big picture ... with 40-50 (optimistically) active triagers in varying
states of activeness and tons of unique reporters per year** the greater
available manpower by far is on the reporter side. And so for that
reason alone it's reasonable to expect that many reporters will be asked
"can you reproduce doing ______."


** in 180 days ending April 10:
- 1,384 bug filers. ~1,100 filing only one bug. 275 filing 2-3 bugs. 75
4-9 bugs, 35 >10 bugs
- 2,743 new bugs filed. of those 1,291 are still open (about half
unconfirmed), 527 fixed, ~927 closed other resolutions (mostly dupes).
~300 of the 2,743 or 10% are ENH bugs
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|

Re: Increasing QA contributions

Wayne Mery
In reply to this post by Peter Lairo-3
On 4/18/2009 6:16 PM, Peter Lairo wrote:
> On 18.04.2009 10:59, Nelson Bolyard wrote:
>> Can you "paste as quotation" from the keyboard in TB3 betas yet?
>
> https://bugzilla.mozilla.org/show_bug.cgi?id=461117

That bug fails to mention the rather simple Thunderbird workaround
mentioned in bug 192330
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
12