Contributor pathways, engagement points and bug mentoring

classic Classic list List threaded Threaded
59 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Robert Kaiser
Gijs Kruitbosch schrieb:
> ** which probably means we should also think harder on the fx desktop
> team about which bugs could be such bugs, and how to make more of them!

That's surely a good idea!

KaiRo

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Gavin Sharp-3
In reply to this post by Ehsan Akhgari
On Wed, Apr 16, 2014 at 8:53 AM, Ehsan Akhgari <[hidden email]> wrote:
> To get back to the original purpose of this thread, I actually question
> Gavin's assertion about the cost effectiveness and scalability of answering
> "yes please, you can work on this bug".  As someone who has worked in two
> very different cultures, I think it's very easy to dismiss these types of
> asks before working on a bug as an indicator that the requestor is not
> serious etc but that is far from the truth, they are just trying to be
> polite within the cultural norms that they are used to.

I think you misunderstood my claim.

If all someone needs is a "yes please, go right ahead! Here are some
tips:", we should definitely provide it to them. If adding needinfos
helps with that, that's great, and I don't object to it.

But, my personal experience has shown that a high percentage of the
people who ask "can you assign this bug to me" or some variant of it,
aren't actually ready to have the bug assigned to them. I have
assigned bugs, and spent time providing helpful comments, only to get
no response or to discover that the person is so far from being able
to contribute (for various reasons) that it's not an effective use of
my time to help them with that bug. Those experiences are what led to
me "ignore" such requests, for better or worse, and the needinfo
proposal isn't going to address that problem.

So we need to tackle both sides of the problem. That's all I'm saying.

Gavin
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Chris Peterson-12
In reply to this post by Ehsan Akhgari
On 4/16/14, 11:01 AM, Gavin Sharp wrote:
> But, my personal experience has shown that a high percentage of the
> people who ask "can you assign this bug to me" or some variant of it,
> aren't actually ready to have the bug assigned to them. I have
> assigned bugs, and spent time providing helpful comments, only to get
> no response or to discover that the person is so far from being able
> to contribute (for various reasons) that it's not an effective use of
> my time to help them with that bug. Those experiences are what led to
> me "ignore" such requests, for better or worse, and the needinfo
> proposal isn't going to address that problem.

Maybe we can craft some polite canned responses to new contributor's bug
questions. If they have not met some minimum requirements like "I have
successfully built Firefox and created a test patch", then we can direct
them from Bugzilla to #introduction and the wiki.


chris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Josh Matthews-4
On 04/16/2014 02:58 PM, Chris Peterson wrote:

> On 4/16/14, 11:01 AM, Gavin Sharp wrote:
>> But, my personal experience has shown that a high percentage of the
>> people who ask "can you assign this bug to me" or some variant of it,
>> aren't actually ready to have the bug assigned to them. I have
>> assigned bugs, and spent time providing helpful comments, only to get
>> no response or to discover that the person is so far from being able
>> to contribute (for various reasons) that it's not an effective use of
>> my time to help them with that bug. Those experiences are what led to
>> me "ignore" such requests, for better or worse, and the needinfo
>> proposal isn't going to address that problem.
>
> Maybe we can craft some polite canned responses to new contributor's bug
> questions. If they have not met some minimum requirements like "I have
> successfully built Firefox and created a test patch", then we can direct
> them from Bugzilla to #introduction and the wiki.
>
>
> chris

Exactly. This is the role of #introduction; mentors should not feel
responsible for solving build environment issues that are not a direct
result of changes made in order to solve the bug under discussion. Off
the top of my head:
"Have you compiled Firefox from source? If not, please see
https://developer.mozilla.org/en/Simple_Firefox_build and direct further
questions to #introduction on http://irc.mozilla.org. When you a build
environment prepared, please feel free to start work on this bug
immediately, and I will try to answer any specific questions you have
about this task."

There is also nothing wrong with unassigning someone who stops
responding (a needinfo with a week's leniency seems polite to me).

Cheers,
Josh
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Benjamin Smedberg
On 4/16/2014 3:23 PM, Josh Matthews wrote:
>
> There is also nothing wrong with unassigning someone who stops
> responding (a needinfo with a week's leniency seems polite to me).

The recommendation at
https://wiki.mozilla.org/Contribute/Coding/Mentoring is that we don't
assign bugs to new contributors until they've produced the patch. So the
response to "I'd like to work on this bug, please assign it to me"
should for new contributors be "Please feel free to work on this bug. We
don't assign bugs to new contributors until they have created a patch,
but please feel free to ask for help here as bug comments."

--BDS
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Mike Connor-4
In reply to this post by Gavin Sharp-3

On Apr 15, 2014, at 6:59 PM, Gavin Sharp <[hidden email]> wrote:

> We need to have
> systematic solutions for avoiding expending large amounts of effort
> for minimal return, and efficient ways to direct potential
> contributors to the right opportunities.

I think this gets held up a lot, but usually in the context of people arguing that we should make calls quicker on people to avoid sinking too much effort into an individual that won’t deliver a good enough return on that investment.  My point is that we often can’t know, without investing a significant amount of time, whether someone will be an important contributor over the long run.

Largely, I think we worry a lot about the false positive case (when we invest too much in someone who wasn’t worth it), and don’t spend as much time thinking about the false negative case (when we fail to invest enough time in someone who would be an important contributor).  Perhaps more importantly, I think we value the “lost” time for the false positive far more than we value the contributions we’re losing from the false negative case.  That’s natural, but I think we can and should do better.

— Mike
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

How to best engage with and invest time in new/beginning contributors (was: Re: Contributor pathways, engagement points and bug mentoring)

Gijs Kruitbosch ("Hannibal")
In reply to this post by Gavin Sharp-3
On 21/04/2014 15:24, Michael Connor wrote:

>
> On Apr 15, 2014, at 6:59 PM, Gavin Sharp <[hidden email]> wrote:
>
>> We need to have
>> systematic solutions for avoiding expending large amounts of effort
>> for minimal return, and efficient ways to direct potential
>> contributors to the right opportunities.
>
> I think this gets held up a lot, but usually in the context of people
> arguing that we should make calls quicker on people to avoid sinking too
> much effort into an individual that won’t deliver a good enough return
> on that investment. My point is that we often can’t know, without
> investing a significant amount of time, whether someone will be an
> important contributor over the long run.

I didn't mean to hold this up in terms of "we need quicker calls" but
more in terms of "how do we communicate such a call". I find it hard to
say "no" generally, and even harder in these circumstances.

However, I think that the below point is more important:

> Largely, I think we worry a lot about the false positive case (when we
> invest too much in someone who wasn’t worth it), and don’t spend as much
> time thinking about the false negative case (when we fail to invest
> enough time in someone who would be an important contributor). Perhaps
> more importantly, I think we value the “lost” time for the false
> positive far more than we value the contributions we’re losing from the
> false negative case. That’s natural, but I think we can and should do
> better.

Two points here:
1) as I pointed out, the false positive and false negative cases are
linked in that supplies are (relatively) low, and more than a few of the
bugs in http://www.joshmatthews.net/bugsahoy/?ff=1&unowned=1 were at one
point owned and went unfixed. All the time in which bugs are assigned to
people who don't fix them is time they spend making it harder for the
"right" people to find bugs to work on.

2) How can we reduce the number of false negatives? This is intended as
a genuine question - I would love to be able to help in that department.
Could we try to e.g. get team members to identify five mentored bugs a
week or something? Send swag to people for fixing mentored bugs? Invite
them for work weeks? Organize "fix mentored bugs" competitions/days?
Make it easier for people to fix bugs by improving the "effort curve"
for contributing? Something else? Do we (does Mike) have data on some of
this?

I'm sure Mike Hoye has lots of thoughts on this subject, but it's a
while away from his original post, so I'm threadsplitting (at least
subject-wise, ymmv depending on how you're interacting with this
thread). :-)

~ Gijs

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: How to best engage with and invest time in new/beginning contributors (was: Re: Contributor pathways, engagement points and bug mentoring)

Benjamin Kerensa
On Mon, Apr 21, 2014 at 7:41 AM, Gijs Kruitbosch
<[hidden email]> wrote:
> On 21/04/2014 15:24, Michael Connor wrote:
>>
>>
>> On Apr 15, 2014, at 6:59 PM, Gavin Sharp <[hidden email]> wrote:
>>
>>> We need to have
>>> systematic solutions for avoiding expending large amounts of effort
>>> for minimal return, and efficient ways to direct potential
>>> contributors to the right opportunities.

Is there any current policies in place in how you work with
contributors and at what point you stop investing time into a
contributor or group of contributors? Do you have strict criteria you
look for currently such as code examples (Github, Launchpad etc)
before investing time in a contributor?

>>
>>
>> I think this gets held up a lot, but usually in the context of people
>> arguing that we should make calls quicker on people to avoid sinking too
>> much effort into an individual that won’t deliver a good enough return
>> on that investment. My point is that we often can’t know, without
>> investing a significant amount of time, whether someone will be an
>> important contributor over the long run.

I don't think this is something that can be easily solved and I think
all open source projects struggle with investing time in contributors
who just do not work out. On the other hand some other open source
projects do have employees who role is strictly connecting good
contributors and filtering out ones that are not going to work out.

>
>
> I didn't mean to hold this up in terms of "we need quicker calls" but more
> in terms of "how do we communicate such a call". I find it hard to say "no"
> generally, and even harder in these circumstances.
>
> However, I think that the below point is more important:
>
>> Largely, I think we worry a lot about the false positive case (when we
>> invest too much in someone who wasn’t worth it), and don’t spend as much
>> time thinking about the false negative case (when we fail to invest
>> enough time in someone who would be an important contributor). Perhaps
>> more importantly, I think we value the “lost” time for the false
>> positive far more than we value the contributions we’re losing from the
>> false negative case. That’s natural, but I think we can and should do
>> better.
>
>
> Two points here:
> 1) as I pointed out, the false positive and false negative cases are linked
> in that supplies are (relatively) low, and more than a few of the bugs in
> http://www.joshmatthews.net/bugsahoy/?ff=1&unowned=1 were at one point owned
> and went unfixed. All the time in which bugs are assigned to people who
> don't fix them is time they spend making it harder for the "right" people to
> find bugs to work on.

Do we have any tools to check bugs that are assigned for periods of
time and perhaps leave a canned message and unassign the contributor?

>
> 2) How can we reduce the number of false negatives? This is intended as a
> genuine question - I would love to be able to help in that department. Could
> we try to e.g. get team members to identify five mentored bugs a week or
> something? Send swag to people for fixing mentored bugs? Invite them for
> work weeks? Organize "fix mentored bugs" competitions/days? Make it easier
> for people to fix bugs by improving the "effort curve" for contributing?
> Something else? Do we (does Mike) have data on some of this?

I think all of those forms of recognition are good ways to recognize
contributors but I hope that some policy surrounding when each is
offered is developed. For instance I think in the case of inviting
someone to a work week they should be a core contributor to your team
that is contributing at least part time what a paid contributor might.

Also sometimes an e-mail or note is much more rewarding to
contributors then a t-shirt or swag. Just knowing that they are
helping and providing some value is pretty powerful recognition too.

- Benjamin
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Gavin Sharp-3
In reply to this post by Mike Connor-4
I worry about the "false positive" case not just because it represents
"wasted effort", but because it's a demotivating experience for the mentor,
which can build on itself and make people less likely to take a chance and
invest in a new contributor in the future. That leads directly to the
"false negative case" that you're describing.

These aren't independent problems - all I'm saying is that we shouldn't
treat them as such, and we need to address them both.

Gavin


On Mon, Apr 21, 2014 at 7:24 AM, Michael Connor <[hidden email]> wrote:

>
> On Apr 15, 2014, at 6:59 PM, Gavin Sharp <[hidden email]> wrote:
>
> > We need to have
> > systematic solutions for avoiding expending large amounts of effort
> > for minimal return, and efficient ways to direct potential
> > contributors to the right opportunities.
>
> I think this gets held up a lot, but usually in the context of people
> arguing that we should make calls quicker on people to avoid sinking too
> much effort into an individual that won’t deliver a good enough return on
> that investment.  My point is that we often can’t know, without investing a
> significant amount of time, whether someone will be an important
> contributor over the long run.
>
> Largely, I think we worry a lot about the false positive case (when we
> invest too much in someone who wasn’t worth it), and don’t spend as much
> time thinking about the false negative case (when we fail to invest enough
> time in someone who would be an important contributor).  Perhaps more
> importantly, I think we value the “lost” time for the false positive far
> more than we value the contributions we’re losing from the false negative
> case.  That’s natural, but I think we can and should do better.
>
> — Mike
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Mike Connor-4
This is an interesting perspective.  I’ve mentored plenty of people who didn’t succeed or follow through, and I don’t think I found it demotivating or made me less willing to help others, but that could be an expectations issue.  I think I always recognized that I was accepting the false positives to make sure I caught as many good hackers as I could find.  It probably helped that my initial job description had community building as 50% of my time, but it was how I was working even when I was still a volunteer.

Describing it as “taking a chance” implies there’s a risk involved for the mentor, but I’m not sure what’s being risked.  Is it reputation? Time? Self-confidence (in terms of self-belief in mentoring skills)?  Can we help people understand that failures are an expected part of the process, and a necessary evil?

I completely agree that they’re not independent problems.  I think it’s a question of “which direction are we happier failing in?”  For me, it’s false positives, because I believe that the cost of false positives will be lower than the benefit we’ll get from fewer false negatives.  It’d be good to figure out how we measure that to be sure that math holds.

— Mike

On Apr 21, 2014, at 3:01 PM, Gavin Sharp <[hidden email]> wrote:

> I worry about the "false positive" case not just because it represents "wasted effort", but because it's a demotivating experience for the mentor, which can build on itself and make people less likely to take a chance and invest in a new contributor in the future. That leads directly to the "false negative case" that you're describing.
>
> These aren't independent problems - all I'm saying is that we shouldn't treat them as such, and we need to address them both.
>
> Gavin
>
>
> On Mon, Apr 21, 2014 at 7:24 AM, Michael Connor <[hidden email]> wrote:
>
> On Apr 15, 2014, at 6:59 PM, Gavin Sharp <[hidden email]> wrote:
>
> > We need to have
> > systematic solutions for avoiding expending large amounts of effort
> > for minimal return, and efficient ways to direct potential
> > contributors to the right opportunities.
>
> I think this gets held up a lot, but usually in the context of people arguing that we should make calls quicker on people to avoid sinking too much effort into an individual that won’t deliver a good enough return on that investment.  My point is that we often can’t know, without investing a significant amount of time, whether someone will be an important contributor over the long run.
>
> Largely, I think we worry a lot about the false positive case (when we invest too much in someone who wasn’t worth it), and don’t spend as much time thinking about the false negative case (when we fail to invest enough time in someone who would be an important contributor).  Perhaps more importantly, I think we value the “lost” time for the false positive far more than we value the contributions we’re losing from the false negative case.  That’s natural, but I think we can and should do better.
>
> — Mike
>

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Gijs Kruitbosch ("Hannibal")
In reply to this post by Gavin Sharp-3
On 21/04/2014 20:22, Michael Connor wrote:
> This is an interesting perspective.  I’ve mentored plenty of people who didn’t succeed or follow through, and I don’t think I found it demotivating or made me less willing to help others, but that could be an expectations issue.

Can you give some bugs as examples? How recent was this? I didn't find
anything with the bugsahoy whiteboard tags in bugzilla (for "mconnor")
so I'm wondering if we're just seeing very different types of
contributors now than we have in the past, and so perhaps it's an
experiences rather than expectations issue. :-)

~ Gijs

> On Apr 21, 2014, at 3:01 PM, Gavin Sharp <[hidden email]> wrote:
>
>> I worry about the "false positive" case not just because it represents "wasted effort", but because it's a demotivating experience for the mentor, which can build on itself and make people less likely to take a chance and invest in a new contributor in the future. That leads directly to the "false negative case" that you're describing.
>>
>> These aren't independent problems - all I'm saying is that we shouldn't treat them as such, and we need to address them both.
>>
>> Gavin
>>
>>
>> On Mon, Apr 21, 2014 at 7:24 AM, Michael Connor <[hidden email]> wrote:
>>
>> On Apr 15, 2014, at 6:59 PM, Gavin Sharp <[hidden email]> wrote:
>>
>>> We need to have
>>> systematic solutions for avoiding expending large amounts of effort
>>> for minimal return, and efficient ways to direct potential
>>> contributors to the right opportunities.
>>
>> I think this gets held up a lot, but usually in the context of people arguing that we should make calls quicker on people to avoid sinking too much effort into an individual that won’t deliver a good enough return on that investment.  My point is that we often can’t know, without investing a significant amount of time, whether someone will be an important contributor over the long run.
>>
>> Largely, I think we worry a lot about the false positive case (when we invest too much in someone who wasn’t worth it), and don’t spend as much time thinking about the false negative case (when we fail to invest enough time in someone who would be an important contributor).  Perhaps more importantly, I think we value the “lost” time for the false positive far more than we value the contributions we’re losing from the false negative case.  That’s natural, but I think we can and should do better.
>>
>> — Mike
>>
>

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

mhoye
In reply to this post by Mike Connor-4
On 2014-04-21, 3:22 PM, Michael Connor wrote:
> This is an interesting perspective.  I’ve mentored plenty of people who didn’t succeed or follow through, and I don’t think I found it demotivating or made me less willing to help others, but that could be an expectations issue.
For what it's worth, all the developers I've interviewed last year spoke
about the failure of the relationship - not "failure to complete a bug",
mind you, but specifically the failure of the relationship - as being a
major demotivator. So while it's not universally the case, the
demoralizing-false-positive is certainly a real thing. The way I'm
trying to minimize the downside there is by asking people to get into
these relationships cautiously, and to be willing to size up
contributors and redirect enthusiastic but out-of-their-depth people to
something they'll have a good shot at.

We don't have a lot of support for the practice of mentoring in place
right now, which makes a failed mentoring relationship look (and
certainly feel) like a personal failure on the part of the engineer,
rather than the systemic problem that IMO it really represents; we're on
our way there, though, between the mentoring fields coming into Bugzilla
and some better documentation, hopefully this will make for more
functional and easy-to-manage contributor relationships.

- mhoye
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: How to best engage with and invest time in new/beginning contributors

mhoye
In reply to this post by Gijs Kruitbosch ("Hannibal")
On 2014-04-21, 10:41 AM, Gijs Kruitbosch wrote:

> On 21/04/2014 15:24, Michael Connor wrote:
>> I think this gets held up a lot, but usually in the context of people
>> arguing that we should make calls quicker on people to avoid sinking too
>> much effort into an individual that won’t deliver a good enough return
>> on that investment. My point is that we often can’t know, without
>> investing a significant amount of time, whether someone will be an
>> important contributor over the long run.
>
> I didn't mean to hold this up in terms of "we need quicker calls" but
> more in terms of "how do we communicate such a call". I find it hard
> to say "no" generally, and even harder in these circumstances.

So, I have more to say about this - when I'm back from my trip tomorrow,
one hopes! - but I think that:

- it's pretty clear when somebody is thrashing, rather than making
forward progress, and
- the real challenge is actually making "the call", rather than knowing
that you should.

With regards to what "the call" means, my theory is that having a bunch
of different options for how and where an engineer can send a strugging
contributor close to hand will make this much easier and happier
experience, so I'm working on building those out right now.


- mhoye
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Gavin Sharp-3
In reply to this post by Mike Connor-4
You can't see how mentors might be reluctant to invest time in helping
contributors if a high percentage of the their past efforts have not been
successful in any form? This seems fairly obvious to me. No one is
expecting a 100% "success" rate - but there's no getting around the fact
that a 100% (or 90%) "failure" rate is going to have an impact on your
future efforts.

I fully support encouraging people to invest more in mentoring, and it
makes sense to bias our cost/benefit analysis to the longer term rather
than focusing on short term returns. But you can't boil this down to just
"pick one direction to fail in", because failing too much in the "false
positive" case makes you fail more in the "false negative" case too.

Gavin


On Mon, Apr 21, 2014 at 12:22 PM, Michael Connor <[hidden email]>wrote:

> This is an interesting perspective.  I’ve mentored plenty of people who
> didn’t succeed or follow through, and I don’t think I found it demotivating
> or made me less willing to help others, but that could be an expectations
> issue.  I think I always recognized that I was accepting the false
> positives to make sure I caught as many good hackers as I could find.  It
> probably helped that my initial job description had community building as
> 50% of my time, but it was how I was working even when I was still a
> volunteer.
>
> Describing it as “taking a chance” implies there’s a risk involved for the
> mentor, but I’m not sure what’s being risked.  Is it reputation? Time?
> Self-confidence (in terms of self-belief in mentoring skills)?  Can we help
> people understand that failures are an expected part of the process, and a
> necessary evil?
>
> I completely agree that they’re not independent problems.  I think it’s a
> question of “which direction are we happier failing in?”  For me, it’s
> false positives, because I believe that the cost of false positives will be
> lower than the benefit we’ll get from fewer false negatives.  It’d be good
> to figure out how we measure that to be sure that math holds.
>
> — Mike
>
> On Apr 21, 2014, at 3:01 PM, Gavin Sharp <[hidden email]> wrote:
>
> > I worry about the "false positive" case not just because it represents
> "wasted effort", but because it's a demotivating experience for the mentor,
> which can build on itself and make people less likely to take a chance and
> invest in a new contributor in the future. That leads directly to the
> "false negative case" that you're describing.
> >
> > These aren't independent problems - all I'm saying is that we shouldn't
> treat them as such, and we need to address them both.
> >
> > Gavin
> >
> >
> > On Mon, Apr 21, 2014 at 7:24 AM, Michael Connor <[hidden email]>
> wrote:
> >
> > On Apr 15, 2014, at 6:59 PM, Gavin Sharp <[hidden email]> wrote:
> >
> > > We need to have
> > > systematic solutions for avoiding expending large amounts of effort
> > > for minimal return, and efficient ways to direct potential
> > > contributors to the right opportunities.
> >
> > I think this gets held up a lot, but usually in the context of people
> arguing that we should make calls quicker on people to avoid sinking too
> much effort into an individual that won’t deliver a good enough return on
> that investment.  My point is that we often can’t know, without investing a
> significant amount of time, whether someone will be an important
> contributor over the long run.
> >
> > Largely, I think we worry a lot about the false positive case (when we
> invest too much in someone who wasn’t worth it), and don’t spend as much
> time thinking about the false negative case (when we fail to invest enough
> time in someone who would be an important contributor).  Perhaps more
> importantly, I think we value the “lost” time for the false positive far
> more than we value the contributions we’re losing from the false negative
> case.  That’s natural, but I think we can and should do better.
> >
> > — Mike
> >
>
>
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: How to best engage with and invest time in new/beginning contributors

mhoye
In reply to this post by Benjamin Kerensa
On 2014-04-21, 12:27 PM, Benjamin Kerensa wrote:
> Is there any current policies in place in how you work with
> contributors and at what point you stop investing time into a
> contributor or group of contributors? Do you have strict criteria you
> look for currently such as code examples (Github, Launchpad etc)
> before investing time in a contributor?

I wrote a bunch of stuff up here about the mentor's role and expectations:

https://wiki.mozilla.org/Contribute/Coding/Mentoring

They're intended to be helpful guidelines, not policies. No policy or
criteria is going to be able to replace or do better than smart people
who care exercising their best judgement.

> I don't think this is something that can be easily solved and I think
> all open source projects struggle with investing time in contributors
> who just do not work out. On the other hand some other open source
> projects do have employees who role is strictly connecting good
> contributors and filtering out ones that are not going to work out.

Yeah, you can't "solve" people.

> Do we have any tools to check bugs that are assigned for periods of
> time and perhaps leave a canned message and unassign the contributor?

Keeping an eye on this situation is my responsibility; I'm hoping to
automate some of it, but either way.

> I think all of those forms of recognition are good ways to recognize
> contributors but I hope that some policy surrounding when each is
> offered is developed. For instance I think in the case of inviting
> someone to a work week they should be a core contributor to your team
> that is contributing at least part time what a paid contributor might.
>
> Also sometimes an e-mail or note is much more rewarding to
> contributors then a t-shirt or swag. Just knowing that they are
> helping and providing some value is pretty powerful recognition too.
I've outlined a bit of that here:
https://wiki.mozilla.org/Contribute/Coding/Pathways - it's a work in
progress, but it speaks to some of your points.

...but the situation is actually worse than you describe - after a
certain point sending a high-value contributor a t-shirt or some
stickers is kind of insulting; at that point, the rewards that matter
are involvement, recognition and greater responsibility.

- mhoye


_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: How to best engage with and invest time in new/beginning contributors (was: Re: Contributor pathways, engagement points and bug mentoring)

Mike Connor-4
In reply to this post by Gijs Kruitbosch ("Hannibal")

On Apr 21, 2014, at 10:41 AM, Gijs Kruitbosch <[hidden email]> wrote:

> On 21/04/2014 15:24, Michael Connor wrote:
>>
>> On Apr 15, 2014, at 6:59 PM, Gavin Sharp <[hidden email]> wrote:
>>
>>> We need to have
>>> systematic solutions for avoiding expending large amounts of effort
>>> for minimal return, and efficient ways to direct potential
>>> contributors to the right opportunities.
>>
>> I think this gets held up a lot, but usually in the context of people
>> arguing that we should make calls quicker on people to avoid sinking too
>> much effort into an individual that won’t deliver a good enough return
>> on that investment. My point is that we often can’t know, without
>> investing a significant amount of time, whether someone will be an
>> important contributor over the long run.
>
> I didn't mean to hold this up in terms of "we need quicker calls" but more in terms of "how do we communicate such a call". I find it hard to say "no" generally, and even harder in these circumstances.

These are very difficult conversations to have, for sure.  I think a big part of the art of mentoring is about helping people find the right path for them.  It’s not just “how do I do this thing?” but “how do I succeed?”  If they’re on the wrong path, hopefully you’ve interacted with them enough to help them find a better fit for their skills.  If not, and it’s an uncomfortable conversation for you, I’d be happy to engage with individuals to help them find a better place for their energy and passion. I’m sure mhoye and others would also be glad to help out with the hard stuff.

> However, I think that the below point is more important:
>
>> Largely, I think we worry a lot about the false positive case (when we
>> invest too much in someone who wasn’t worth it), and don’t spend as much
>> time thinking about the false negative case (when we fail to invest
>> enough time in someone who would be an important contributor). Perhaps
>> more importantly, I think we value the “lost” time for the false
>> positive far more than we value the contributions we’re losing from the
>> false negative case. That’s natural, but I think we can and should do
>> better.
>
> Two points here:
> 1) as I pointed out, the false positive and false negative cases are linked in that supplies are (relatively) low, and more than a few of the bugs in http://www.joshmatthews.net/bugsahoy/?ff=1&unowned=1 were at one point owned and went unfixed. All the time in which bugs are assigned to people who don't fix them is time they spend making it harder for the "right" people to find bugs to work on.

Are we really in a state where we’re target-constrained?  My experience with Mozilla is that there’s effectively infinite possible work.  There’s 312 firefox-backlog+ bugs assigned to nobody@, and that number will grow.  People should take a bug once they have a patch (though they can comment to say they’re trying to produce a patch.)

All that said, I’d consider it a high quality problem to have so many people wanting to fix bugs that we’re out of useful work to give them.

> 2) How can we reduce the number of false negatives? This is intended as a genuine question - I would love to be able to help in that department. Could we try to e.g. get team members to identify five mentored bugs a week or something? Send swag to people for fixing mentored bugs? Invite them for work weeks? Organize "fix mentored bugs" competitions/days? Make it easier for people to fix bugs by improving the "effort curve" for contributing? Something else? Do we (does Mike) have data on some of this?

Having done this a lot, the following three things, in order, are pretty key to encouraging and including talented people:

* Review turnaround.

If we can tag individuals as “in mentoring” (basically “newbie who’s cleared the #introduction steps”) and then prioritize responses to bugs/patches for that group, we’re winning.  Think about how hard it is when reviews get delayed and you have to work on other things, and then come back and fix stuff.  Now imagine if the context you’re swapping in is “all of Mozilla tech” and you’ve got a day job or school.  Reviews and feedback requests from people in mentoring should trump everyone else short of a chemspill.

I’d suggest “time from taking bug to bug landed and in Nightly” is the key pseudo-metric here.  Everyone wants that sense of accomplishment, and we should help them get it quickly.

* Integration into the group.

Get them on IRC, get them interacting with the rest of the team so they feel like they’re part of the project, not outsiders.  Have conversations, even somewhat crunchy ones, somewhere they can be a part of it.

* Show them the love.

Swag, work week invites, delivery of tasty things, summits and MozCamps.  These are all ways we should people they’re appreciated and important.

These are all hard, and require effort and diligence to get right.

— Mike
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Mike Connor-4
In reply to this post by Gijs Kruitbosch ("Hannibal")
On Apr 21, 2014, at 3:42 PM, Gijs Kruitbosch <[hidden email]> wrote:

> On 21/04/2014 20:22, Michael Connor wrote:
>> This is an interesting perspective.  I’ve mentored plenty of people who didn’t succeed or follow through, and I don’t think I found it demotivating or made me less willing to help others, but that could be an expectations issue.
>
> Can you give some bugs as examples? How recent was this? I didn't find anything with the bugsahoy whiteboard tags in bugzilla (for "mconnor") so I'm wondering if we're just seeing very different types of contributors now than we have in the past, and so perhaps it's an experiences rather than expectations issue. :-)

I’ve been inactive in mainline dev for a fair while.  (I keep filing the points on my head away, but they grow back.)  Largely I’m thinking 2005-2010 timeframe.  I’m also including things like ‘interns’ in this count.  Not every intern is a rock star, even perfect GPA kids from top schools.  It happens, and it’s hard to accurately predict.

I’ve seen little evidence that we aren’t attracting the same caliber of people.  It’s a bit hard to prove either way, of course.

— Mike
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: How to best engage with and invest time in new/beginning contributors (was: Re: Contributor pathways, engagement points and bug mentoring)

Mike Connor-4
In reply to this post by Benjamin Kerensa

On Apr 21, 2014, at 12:27 PM, Benjamin Kerensa <[hidden email]> wrote:

> Is there any current policies in place in how you work with
> contributors and at what point you stop investing time into a
> contributor or group of contributors?

I think this is a judgement call, for the most part.  It’s probably worth checking

> Do you have strict criteria you
> look for currently such as code examples (Github, Launchpad etc)
> before investing time in a contributor?

If we do, we’re erring all the way on the side of eliminating false positives.  If we didn’t help people without an established track record, we’d be missing a lot of people.  Including a number of people who’ve commented on this thread.  I’m not convinced that’s a path toward winning.

— Mike
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Chris Peterson-12
In reply to this post by Gijs Kruitbosch ("Hannibal")
On 4/21/14, 3:45 PM, Michael Connor wrote:
>> >Can you give some bugs as examples? How recent was this? I didn't find anything with the bugsahoy whiteboard tags in bugzilla (for "mconnor") so I'm wondering if we're just seeing very different types of contributors now than we have in the past, and so perhaps it's an experiences rather than expectations issue. :-)
> I’ve been inactive in mainline dev for a fair while.  (I keep filing the points on my head away, but they grow back.)  Largely I’m thinking 2005-2010 timeframe.  I’m also including things like ‘interns’ in this count.  Not every intern is a rock star, even perfect GPA kids from top schools.  It happens, and it’s hard to accurately predict.

Mike, do we have statistics on how contributors find us? For example,
how many come through Bugs Ahoy, school projects, or fixing a bug that
they reported?


chris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Contributor pathways, engagement points and bug mentoring

Karl Dubost
In reply to this post by Gavin Sharp-3

Le 22 avr. 2014 à 05:02, Gavin Sharp <[hidden email]> a écrit :
> expecting a 100% "success" rate - but there's no getting around the fact that a 100% (or 90%) "failure" rate is going to have an impact on your future efforts.

That would mean you have been able in advance to define what is "success" and what is "failure". My experience (not at Mozilla), but at W3C in terms of contributors to the W3C validators, spec reviews, etc. is that the return on investment is far to be even and success is a very difficult thing to assess.

* There are people who have completed the one task that they wanted to do and will never come back. It was successful because it was their initial goal.

* There are people who understood, and will not participate, BUT,  WHEN the time permits in the future, will contribute actively.

* There are people where the mentoring didn't lead to direct contributions but to them being strong advocates for the project leading other people to come. They came to learn how it was happening and in return encourage others to do.

I think in all these discussions we are having we forget about our own attitudes with regards to other projects in life, sports club, etc. We met people who gave us a bit of our time to understand an area, … sometimes we pushed further, sometimes we gave up.
It. is. ok.
The people who mentored us, shared, etc. did it out of love for what they are doing, not for productivity marks ;)

Now it is entirely possible (and acceptable) that someone doesn't want to do that :) No issue.



--
Karl Dubost, Mozilla
http://www.la-grange.net/karl/moz

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
123