Revisiting "suggested sign-off" on the dashboard

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Revisiting "suggested sign-off" on the dashboard

Axel Hecht
Hi,

I'm currently thinking about the "suggested sign-off" feature on the
sign-off pages. Right now, that's any push that's newer than the
accepted sign-off that's green.

I've come across a few cases where the latest status was actually red,
but had better translation coverage, and the old green one was just
green at the time.

I propose that we reduce the "suggested" one to only occur if the latest
compare-locales result is green.

That doesn't discourage the other revisions more than it used to, I just
think it's not encouraging sub-optimal revisions.

Opinions?

Axel
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Cédric Corazza
Le 16/02/2012 21:02, Axel Hecht a écrit :

> Hi,
>
> I'm currently thinking about the "suggested sign-off" feature on the
> sign-off pages. Right now, that's any push that's newer than the
> accepted sign-off that's green.
>
> I've come across a few cases where the latest status was actually red,
> but had better translation coverage, and the old green one was just
> green at the time.
>
> I propose that we reduce the "suggested" one to only occur if the latest
> compare-locales result is green.
>
> That doesn't discourage the other revisions more than it used to, I just
> think it's not encouraging sub-optimal revisions.
>
> Opinions?
>
> Axel

Hi,
I would say "suggested sign-off" is OK for aurora and beta imo. For
releases, I think the sign-off should be mandatory: we'we got to be sure
the product has been tested by the localizers.
But maybe we should investigate why a green state has not the sign-off.
For my part, I often miss the deadline for sign-off because I forgot the
date, because deadlines can slip, etc.
We could have a smoother process and let everybody spares time
(especially l10n staff who have to ping every locale team without
sign-off), by adding the sign-off deadline for product and branch on the
dashboard itself, which is the place every localizer looks at regularly.
That wouldn't forbid to continue announcing deadlines on the newsgroups
though.

My two cents.

Cédric
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Axel Hecht
On 16.02.12 21:48, Cédric Corazza wrote:

> Le 16/02/2012 21:02, Axel Hecht a écrit :
>> Hi,
>>
>> I'm currently thinking about the "suggested sign-off" feature on the
>> sign-off pages. Right now, that's any push that's newer than the
>> accepted sign-off that's green.
>>
>> I've come across a few cases where the latest status was actually red,
>> but had better translation coverage, and the old green one was just
>> green at the time.
>>
>> I propose that we reduce the "suggested" one to only occur if the latest
>> compare-locales result is green.
>>
>> That doesn't discourage the other revisions more than it used to, I just
>> think it's not encouraging sub-optimal revisions.
>>
>> Opinions?
>>
>> Axel
>
> Hi,
> I would say "suggested sign-off" is OK for aurora and beta imo. For
> releases, I think the sign-off should be mandatory: we'we got to be sure
> the product has been tested by the localizers.
> But maybe we should investigate why a green state has not the sign-off.
> For my part, I often miss the deadline for sign-off because I forgot the
> date, because deadlines can slip, etc.
> We could have a smoother process and let everybody spares time
> (especially l10n staff who have to ping every locale team without
> sign-off), by adding the sign-off deadline for product and branch on the
> dashboard itself, which is the place every localizer looks at regularly.
> That wouldn't forbid to continue announcing deadlines on the newsgroups
> though.

To clarify, "suggested sign-off" is a visual indication on the sign-off
page what's technically looking good to sign-off on. One thing we're
currently working on is exposing that information on the teams page.

The way I see things happening is that localizers see there's something
to do, then they do stuff, land it, tree goes green, builds pop up,
folks test and based on their testing, sign off.

Thus, it's on purpose that green status isn't immediately signed off,
but we're looking for a more obvious status.

Seems also that I should clarify what we're using in builds:

On central and aurora, the latest status of the l10n repository is used
to create the depend and nightly builds.
On beta, you get depend builds for a push, but the weekly beta builds
are taken off of the signed-off revisions.

Does that help to scope the question?

Axel
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Pavel Franc-4
>> For my part, I often miss the deadline for sign-off because I forgot the
>> date, because deadlines can slip, etc.
>> We could have a smoother process and let everybody spares time
>> (especially l10n staff who have to ping every locale team without
>> sign-off), by adding the sign-off deadline for product and branch on the
>> dashboard itself, which is the place every localizer looks at regularly.
>> That wouldn't forbid to continue announcing deadlines on the newsgroups
>> though.
>

Hi guys,
personally I have big trouble with signing.

I'm working on central, once upon a time (every six weeks) I move the
work to aurora, and I don't touch the beta at all. So, it is very easy
to forgot to sign the right revision in aurora/beta. Especially when the
dashboard is green, and everything is marked as signed (some revision
from past). Last time we were creating Firefox 10 beta builds from
revision of Firefox 9 !

Either some type of automatic signing, beta sign-off removing after
aurora->beta migration, or big warning in dashboard would be really
appreciate from my side.

Best,
Pavel Franc
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Justin Wood (Callek)-2
Pavel Franc wrote:

>>> For my part, I often miss the deadline for sign-off because I forgot the
>>> date, because deadlines can slip, etc.
>>> We could have a smoother process and let everybody spares time
>>> (especially l10n staff who have to ping every locale team without
>>> sign-off), by adding the sign-off deadline for product and branch on the
>>> dashboard itself, which is the place every localizer looks at regularly.
>>> That wouldn't forbid to continue announcing deadlines on the newsgroups
>>> though.
>>
>
> Hi guys,
> personally I have big trouble with signing.
>
> I'm working on central, once upon a time (every six weeks) I move the
> work to aurora, and I don't touch the beta at all. So, it is very easy
> to forgot to sign the right revision in aurora/beta. Especially when the
> dashboard is green, and everything is marked as signed (some revision
> from past). Last time we were creating Firefox 10 beta builds from
> revision of Firefox 9 !
>
> Either some type of automatic signing, beta sign-off removing after
> aurora->beta migration, or big warning in dashboard would be really
> appreciate from my side.
>

Well if you do all your work in central/aurora you can signoff on aurora
and have that signoff carried forward to the beta release without having
to touch beta (you of course *can*)

We should also do a better job of notifying teams when there are
outstanding signoffs for a given cycle (Pike has a tool to make that
easier from us drivers -- at least in identifying teams, someone
could/should write up a tool to contact you).

Part of that is having responsive l10n contacts (that understand
english). And the method in which to get ahold of the teams can vary as
well. Which is how I view the effort to update the L10n:Teams pages on
wiki.m.o etc.

I for one haven't been contacting people who fall behind/forget as much
as I should, and I hope to get better at that.

--
~Justin Wood (Callek)

_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Axel Hecht
In reply to this post by Pavel Franc-4
On 17.02.12 00:56, Pavel Franc wrote:

>>> For my part, I often miss the deadline for sign-off because I forgot the
>>> date, because deadlines can slip, etc.
>>> We could have a smoother process and let everybody spares time
>>> (especially l10n staff who have to ping every locale team without
>>> sign-off), by adding the sign-off deadline for product and branch on the
>>> dashboard itself, which is the place every localizer looks at regularly.
>>> That wouldn't forbid to continue announcing deadlines on the newsgroups
>>> though.
>>
>
> Hi guys,
> personally I have big trouble with signing.
>
> I'm working on central, once upon a time (every six weeks) I move the
> work to aurora, and I don't touch the beta at all. So, it is very easy
> to forgot to sign the right revision in aurora/beta. Especially when the
> dashboard is green, and everything is marked as signed (some revision
> from past). Last time we were creating Firefox 10 beta builds from
> revision of Firefox 9 !
>
> Either some type of automatic signing, beta sign-off removing after
> aurora->beta migration, or big warning in dashboard would be really
> appreciate from my side.
>
> Best,
> Pavel Franc

We're *so* close to having a team page with the interesting info right
there on one page.

The real downside right now is that it takes 13 seconds to load at least
for the french team. That's not cool, so I'm optimizing. And finding
what we're currently calling "suggested sign-off" on the sign-off page
is rather expensive, so I looked deeper, and remembered that I found
that suggestion to be wrong actually in a few cases. Thus I started the
discussion here.

Axel
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Mad Maks
In reply to this post by Pavel Franc-4
On 17-02-12 00:56, Pavel Franc wrote:

>>> For my part, I often miss the deadline for sign-off because I forgot the
>>> date, because deadlines can slip, etc.
>>> We could have a smoother process and let everybody spares time
>>> (especially l10n staff who have to ping every locale team without
>>> sign-off), by adding the sign-off deadline for product and branch on the
>>> dashboard itself, which is the place every localizer looks at regularly.
>>> That wouldn't forbid to continue announcing deadlines on the newsgroups
>>> though.
>>
>
> Hi guys,
> personally I have big trouble with signing.
>
> I'm working on central, once upon a time (every six weeks) I move the
> work to aurora, and I don't touch the beta at all. So, it is very easy
> to forgot to sign the right revision in aurora/beta. Especially when the
> dashboard is green, and everything is marked as signed (some revision
> from past). Last time we were creating Firefox 10 beta builds from
> revision of Firefox 9 !
>
> Either some type of automatic signing, beta sign-off removing after
> aurora->beta migration, or big warning in dashboard would be really
> appreciate from my side.
>
> Best,
> Pavel Franc

I have the same problem als Pavel, you don't see witch version is the
accepted signoff on the dashboard (Bug 669127).

but what i don't understand why is there a sign off for aurora, as far i
can see aurora is build an aviable even if you don't sign off. I think
it is much clearer when you only have to sign off beta.

Tim Maks
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Francesco Lodolo [:flod]
Il 17/02/12 10.09, Mad Maks ha scritto:
> but what i don't understand why is there a sign off for aurora, as far
> i can see aurora is build an aviable even if you don't sign off. I
> think it is much clearer when you only have to sign off beta.
Working on central I usually sign-off on Aurora, while I request new
sign-offs on Beta only if there are problems that need a fix and can't
wait the next cycle.
Honestly I believed this was the most common workflow, but I've just
realized it's probably not.

Francesco
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Michael Bauer-11
In reply to this post by Pavel Franc-4
Would it not make sense to assume a signoff at the deadline date if the
locale is green at that point? Maybe add a "do not release" button or
something for the odd occasion when a locale is green but does not want
a release (not that I can think of one)?

Quite a few other projects seem to run along those lines.

Michael

16/02/2012 23:56, sgrìobh Pavel Franc:
> For my part, I often miss the deadline for sign-off because I forgot the
> date, because deadlines can slip, etc.

_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Axel Hecht
In reply to this post by Pavel Franc-4
On 17.02.12 11:48, Michael Bauer wrote:
> Would it not make sense to assume a signoff at the deadline date if the
> locale is green at that point? Maybe add a "do not release" button or
> something for the odd occasion when a locale is green but does not want
> a release (not that I can think of one)?
>
> Quite a few other projects seem to run along those lines.

I'm afraid you're more optimistic about the value of those tests than I am.

We're constantly trying to make those tests better and cover more
ground. You can tell by the number of updates I released to
compare-locales just recently. You can also tell from that number that
there has been ways to break localizations that we had yet to discover.

Also, the source-based checks only cover the most simplistic error cases.

The real criteria if we should ship a particular revision is the human
testing. "Sign-off" means that the localizer is confident that the state
of the tree is good. Not that you happen to close a window when you're
trying to copy some text from the scratchpad, for example. To give a
recent example.

Firefox is unlike most open-source projects. It's huge, both from the
amount of work, and in the impact that bugs have on humans around the
globe. We need to work to get better still, so that we keep having that
impact.

On the other hand, "sign-off" isn't all that uncommon in concept. It
actually is really close to pull requests on github. It has a different
name, and your pull request isn't on a repo that you fork, but you're
requesting an upstream maintainer to look at your contribution and ship
it. We're a tad better than what you get at github in that we have some
build and test automation for you, but from a conceptual point of view,
it's pretty similar to a pull request.

Axel

> 16/02/2012 23:56, sgrìobh Pavel Franc:
>> For my part, I often miss the deadline for sign-off because I forgot the
>> date, because deadlines can slip, etc.
>

_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Rimas Kudelis
Hi,

2012.02.17 14:08, Axel Hecht rašė:
> On 17.02.12 11:48, Michael Bauer wrote:
>> Would it not make sense to assume a signoff at the deadline date if
>> the locale is green at that point? Maybe add a "do not release"
>> button or something for the odd occasion when a locale is green but
>> does not want a release (not that I can think of one)?
>>
>> Quite a few other projects seem to run along those lines.

I would want that too.

> I'm afraid you're more optimistic about the value of those tests than
> I am.
>
> We're constantly trying to make those tests better and cover more
> ground. You can tell by the number of updates I released to
> compare-locales just recently. You can also tell from that number
> that there has been ways to break localizations that we had yet to
> discover.
>
> Also, the source-based checks only cover the most simplistic error
> cases.
>
> The real criteria if we should ship a particular revision is the
> human testing. "Sign-off" means that the localizer is confident that
> the state of the tree is good. Not that you happen to close a window
> when you're trying to copy some text from the scratchpad, for
> example. To give a recent example.

This assumption that the localizer will test before signing up... it
simply can't be verified. I would guess that most of the teams who test
before signing off, would do that either way (and Michael's suggestion
does not even affect their workflow). On the other hand, the (small)
teams who don't have time or don't want to bother testing, they can sign
off untested builds anyway. So again, it's an assumption that has no
strong basis (and I've said it before).

Me, for example. I usually sign off as soon as my tree becomes green, if
I don't expect further changes (that is, if we're in a string freeze and
mobile team haven't announced that they don't care about it). And I
definately don't test everything I commit. I DO use Firefox daily, and
on the box where I'm localizing it, I have nightly builds of both
Firefox and Thunderbird running, and I do check out at least some of the
new strings (those that are unclear of otherwise interesting to me).
But, if I would happen to introduce that Scratchpad bug you mentioned as
an example, I don't think it's very likely that I would have caught it.
And yet, I would have signed that revision off, if it were green.

Signing off is good for those who do the QA homework. But for those who
don't, it's just a useless nuisance. I almost remember the "You have
checked, right?" message by heart, and yet, I can't say I have checked.
:P And by the way, either I am quite good at not making mistakes, or
compare-locales is quite good at catching them, or the users don't
really care, or there are too few users (I believe that at least the
first is true and the last is not) because I only get like one or two
bugs a year filed for my locale.



Regarding the actual subject of this thread: Axel,  your proposal
definately makes sense, I wouldn't even ask if I were you. But what does
a suggested sign-off look like? Is it just the sentence saying "There is
a sourcestamp that technically looks good to sign off on.", or is it
something more?

Regards,
Rimas
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Axel Hecht
On 17.02.12 19:08, Rimas Kudelis wrote:

> Hi,
>
> 2012.02.17 14:08, Axel Hecht rašė:
>> On 17.02.12 11:48, Michael Bauer wrote:
>>> Would it not make sense to assume a signoff at the deadline date if
>>> the locale is green at that point? Maybe add a "do not release"
>>> button or something for the odd occasion when a locale is green but
>>> does not want a release (not that I can think of one)?
>>>
>>> Quite a few other projects seem to run along those lines.
>
> I would want that too.
>
>> I'm afraid you're more optimistic about the value of those tests than
>> I am.
>>
>> We're constantly trying to make those tests better and cover more
>> ground. You can tell by the number of updates I released to
>> compare-locales just recently. You can also tell from that number
>> that there has been ways to break localizations that we had yet to
>> discover.
>>
>> Also, the source-based checks only cover the most simplistic error
>> cases.
>>
>> The real criteria if we should ship a particular revision is the
>> human testing. "Sign-off" means that the localizer is confident that
>> the state of the tree is good. Not that you happen to close a window
>> when you're trying to copy some text from the scratchpad, for
>> example. To give a recent example.
>
> This assumption that the localizer will test before signing up... it
> simply can't be verified. I would guess that most of the teams who test
> before signing off, would do that either way (and Michael's suggestion
> does not even affect their workflow). On the other hand, the (small)
> teams who don't have time or don't want to bother testing, they can sign
> off untested builds anyway. So again, it's an assumption that has no
> strong basis (and I've said it before).
>
> Me, for example. I usually sign off as soon as my tree becomes green, if
> I don't expect further changes (that is, if we're in a string freeze and
> mobile team haven't announced that they don't care about it). And I
> definately don't test everything I commit. I DO use Firefox daily, and
> on the box where I'm localizing it, I have nightly builds of both
> Firefox and Thunderbird running, and I do check out at least some of the
> new strings (those that are unclear of otherwise interesting to me).
> But, if I would happen to introduce that Scratchpad bug you mentioned as
> an example, I don't think it's very likely that I would have caught it.
> And yet, I would have signed that revision off, if it were green.
>
> Signing off is good for those who do the QA homework. But for those who
> don't, it's just a useless nuisance. I almost remember the "You have
> checked, right?" message by heart, and yet, I can't say I have checked.
> :P And by the way, either I am quite good at not making mistakes, or
> compare-locales is quite good at catching them, or the users don't
> really care, or there are too few users (I believe that at least the
> first is true and the last is not) because I only get like one or two
> bugs a year filed for my locale.

Yeah, patterns are rather different.

Maybe it helps to detail on my basic thoughts:

1) all locales are equal
2) we can't just ship what's in hg.

1) isn't really true, but it's also true. In this context, it boils down
to me not feeling good to create some process for some locales, and a
different process for others.
2) worst case example would be the german locale breaking the google
plugin. doesn't even have to be malicious edits by the ferengi localizer
taking over that revenue stream, can just be accidental.

At this point, there's no resources at moco to just verify everthing
that lands. I'm also not sure that's well spent resources. We should
incorporate the community in doing so, and I'd be happy to see
suggestions on how we can make the source review tools more accessible,
or somesuch. Also, we'll probably discuss this differently once the new
team page is up.

> Regarding the actual subject of this thread: Axel, your proposal
> definately makes sense, I wouldn't even ask if I were you. But what does
> a suggested sign-off look like? Is it just the sentence saying "There is
> a sourcestamp that technically looks good to sign off on.", or is it
> something more?

I asked, but I didn't wait with the patch 'til I got all answers ;-).

"Suggested" doesn't really mean anything more than that. It doesn't even
discourage to sign-off on things that are not suggested. For those you
should probably just be more aware of why you want that particular
change in.

Axel

PS: I did file a documentation bug referencing this thread, in case
you're wondering :-)
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Armen Zambrano G.
In reply to this post by Axel Hecht
 From release automation it would be helpful if:
* mozilla-beta would take automatically the "suggested sign-off" OR the
one signed off by the localizer

Would this be possible?

cheers,
Armen
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Rimas Kudelis
In reply to this post by Axel Hecht
Hi :)

2012.02.17 20:57, Axel Hecht rašė:

> On 17.02.12 19:08, Rimas Kudelis wrote:
>> 2012.02.17 14:08, Axel Hecht rašė:
>>> We're constantly trying to make those tests better and cover more
>>> ground. You can tell by the number of updates I released to
>>> compare-locales just recently. You can also tell from that number
>>> that there has been ways to break localizations that we had yet to
>>> discover.
>>>
>>> Also, the source-based checks only cover the most simplistic error
>>> cases.
>>>
>>> The real criteria if we should ship a particular revision is the
>>> human testing. "Sign-off" means that the localizer is confident that
>>> the state of the tree is good. Not that you happen to close a window
>>> when you're trying to copy some text from the scratchpad, for
>>> example. To give a recent example.
>>
>> This assumption that the localizer will test before signing up... it
>> simply can't be verified. I would guess that most of the teams who test
>> before signing off, would do that either way (and Michael's suggestion
>> does not even affect their workflow). On the other hand, the (small)
>> teams who don't have time or don't want to bother testing, they can sign
>> off untested builds anyway. So again, it's an assumption that has no
>> strong basis (and I've said it before).
>>
>> Me, for example. I usually sign off as soon as my tree becomes green, if
>> I don't expect further changes (that is, if we're in a string freeze and
>> mobile team haven't announced that they don't care about it). And I
>> definately don't test everything I commit. I DO use Firefox daily, and
>> on the box where I'm localizing it, I have nightly builds of both
>> Firefox and Thunderbird running, and I do check out at least some of the
>> new strings (those that are unclear of otherwise interesting to me).
>> But, if I would happen to introduce that Scratchpad bug you mentioned as
>> an example, I don't think it's very likely that I would have caught it.
>> And yet, I would have signed that revision off, if it were green.
>>
>> Signing off is good for those who do the QA homework. But for those who
>> don't, it's just a useless nuisance. I almost remember the "You have
>> checked, right?" message by heart, and yet, I can't say I have checked.
>> :P And by the way, either I am quite good at not making mistakes, or
>> compare-locales is quite good at catching them, or the users don't
>> really care, or there are too few users (I believe that at least the
>> first is true and the last is not) because I only get like one or two
>> bugs a year filed for my locale.
>
> Yeah, patterns are rather different.
>
> Maybe it helps to detail on my basic thoughts:
>
> 1) all locales are equal
> 2) we can't just ship what's in hg.
>
> 1) isn't really true, but it's also true. In this context, it boils down
> to me not feeling good to create some process for some locales, and a
> different process for others.
> 2) worst case example would be the german locale breaking the google
> plugin. doesn't even have to be malicious edits by the ferengi localizer
> taking over that revenue stream, can just be accidental.
>
> At this point, there's no resources at moco to just verify everthing
> that lands. I'm also not sure that's well spent resources. We should
> incorporate the community in doing so, and I'd be happy to see
> suggestions on how we can make the source review tools more accessible,
> or somesuch. Also, we'll probably discuss this differently once the new
> team page is up.

Hehe, we keep debating this over and over... :)

For 1), my point is that if you gave two possibilities for all locales,
they would still remain equal, but could choose the process they feel
suits them better.

For 2), again, my point is that if I "sign off" whenever I feel like
I've made my last commit, then you still get to check whether or not I
have "accidently" tried to take over the revenue stream or broken
something else. You still perform the same checks, and use the same
resources you'd use checking the last good check-on. The only difference
is the sign-off itself. I (and Michael) would simply prefer the lack of
a sign-off to be treated as an implicit sign-off on the last green
check-in. It would be just a default, nothing more. The locales who
can't imagine signing off without a thorough QA process could still
sign-off explicitly. No-one would be hurt (well, IMO).

>> Regarding the actual subject of this thread: Axel, your proposal
>> definately makes sense, I wouldn't even ask if I were you. But what does
>> a suggested sign-off look like? Is it just the sentence saying "There is
>> a sourcestamp that technically looks good to sign off on.", or is it
>> something more?
>
> I asked, but I didn't wait with the patch 'til I got all answers ;-).
>
> "Suggested" doesn't really mean anything more than that. It doesn't even
> discourage to sign-off on things that are not suggested. For those you
> should probably just be more aware of why you want that particular
> change in.

I see. :)

> PS: I did file a documentation bug referencing this thread, in case
> you're wondering :-)

What bug?

Rimas
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Michael Bauer-11
In reply to this post by Axel Hecht
This may be a case of "my bad" but while I did *a lot* of testing during
the first few releases, I mainly see the dashboard as a means of
remining if I need to translate more stuff. Unless some dramatic changes
are in the pipeline that affect the layout, I can't see that changing.
Maybe not what dashboard is intended for but hey. I do use the beta on a
daily basis on my main system but it's been a long time since I caught
anything that wasn't a typo.

I take you're point about not wanting a two class system but perhaps
that's the wrong way of thinking of it. There are "mature" locales and
"new" locales to my mind and if you've been through several releases, I
personally can't see the harm.

But I'm not going to labour the point :)

Cheers,

Michael

17/02/2012 12:08, sgrìobh Axel Hecht:
> I'm afraid you're more optimistic about the value of those tests than
> I am.

_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Axel Hecht
In reply to this post by Rimas Kudelis
On 17.02.12 21:16, Rimas Kudelis wrote:

> Hi :)
>
> 2012.02.17 20:57, Axel Hecht rašė:
>> On 17.02.12 19:08, Rimas Kudelis wrote:
>>> 2012.02.17 14:08, Axel Hecht rašė:
>>>> We're constantly trying to make those tests better and cover more
>>>> ground. You can tell by the number of updates I released to
>>>> compare-locales just recently. You can also tell from that number
>>>> that there has been ways to break localizations that we had yet to
>>>> discover.
>>>>
>>>> Also, the source-based checks only cover the most simplistic error
>>>> cases.
>>>>
>>>> The real criteria if we should ship a particular revision is the
>>>> human testing. "Sign-off" means that the localizer is confident that
>>>> the state of the tree is good. Not that you happen to close a window
>>>> when you're trying to copy some text from the scratchpad, for
>>>> example. To give a recent example.
>>>
>>> This assumption that the localizer will test before signing up... it
>>> simply can't be verified. I would guess that most of the teams who test
>>> before signing off, would do that either way (and Michael's suggestion
>>> does not even affect their workflow). On the other hand, the (small)
>>> teams who don't have time or don't want to bother testing, they can sign
>>> off untested builds anyway. So again, it's an assumption that has no
>>> strong basis (and I've said it before).
>>>
>>> Me, for example. I usually sign off as soon as my tree becomes green, if
>>> I don't expect further changes (that is, if we're in a string freeze and
>>> mobile team haven't announced that they don't care about it). And I
>>> definately don't test everything I commit. I DO use Firefox daily, and
>>> on the box where I'm localizing it, I have nightly builds of both
>>> Firefox and Thunderbird running, and I do check out at least some of the
>>> new strings (those that are unclear of otherwise interesting to me).
>>> But, if I would happen to introduce that Scratchpad bug you mentioned as
>>> an example, I don't think it's very likely that I would have caught it.
>>> And yet, I would have signed that revision off, if it were green.
>>>
>>> Signing off is good for those who do the QA homework. But for those who
>>> don't, it's just a useless nuisance. I almost remember the "You have
>>> checked, right?" message by heart, and yet, I can't say I have checked.
>>> :P And by the way, either I am quite good at not making mistakes, or
>>> compare-locales is quite good at catching them, or the users don't
>>> really care, or there are too few users (I believe that at least the
>>> first is true and the last is not) because I only get like one or two
>>> bugs a year filed for my locale.
>>
>> Yeah, patterns are rather different.
>>
>> Maybe it helps to detail on my basic thoughts:
>>
>> 1) all locales are equal
>> 2) we can't just ship what's in hg.
>>
>> 1) isn't really true, but it's also true. In this context, it boils down
>> to me not feeling good to create some process for some locales, and a
>> different process for others.
>> 2) worst case example would be the german locale breaking the google
>> plugin. doesn't even have to be malicious edits by the ferengi localizer
>> taking over that revenue stream, can just be accidental.
>>
>> At this point, there's no resources at moco to just verify everthing
>> that lands. I'm also not sure that's well spent resources. We should
>> incorporate the community in doing so, and I'd be happy to see
>> suggestions on how we can make the source review tools more accessible,
>> or somesuch. Also, we'll probably discuss this differently once the new
>> team page is up.
>
> Hehe, we keep debating this over and over... :)
>
> For 1), my point is that if you gave two possibilities for all locales,
> they would still remain equal, but could choose the process they feel
> suits them better.

My point is, I would offer two possibilities. I'd put some locales in
bucket one, and the rest in the other bucket. Locales in bucket one
could probably still opt out, but the other bucket couldn't get into
bucket one.

> For 2), again, my point is that if I "sign off" whenever I feel like
> I've made my last commit, then you still get to check whether or not I
> have "accidently" tried to take over the revenue stream or broken
> something else. You still perform the same checks, and use the same
> resources you'd use checking the last good check-on. The only difference
> is the sign-off itself. I (and Michael) would simply prefer the lack of
> a sign-off to be treated as an implicit sign-off on the last green
> check-in. It would be just a default, nothing more. The locales who
> can't imagine signing off without a thorough QA process could still
> sign-off explicitly. No-one would be hurt (well, IMO).

To be honest, you neglect how much time it takes to go through these.
Yes, I ask you guys to do some work. But that's because if you do, I
don't have to. If you feel that it's tedious that I ask you that,
imagine doing it 100 times.

>>> Regarding the actual subject of this thread: Axel, your proposal
>>> definately makes sense, I wouldn't even ask if I were you. But what does
>>> a suggested sign-off look like? Is it just the sentence saying "There is
>>> a sourcestamp that technically looks good to sign off on.", or is it
>>> something more?
>>
>> I asked, but I didn't wait with the patch 'til I got all answers ;-).
>>
>> "Suggested" doesn't really mean anything more than that. It doesn't even
>> discourage to sign-off on things that are not suggested. For those you
>> should probably just be more aware of why you want that particular
>> change in.
>
> I see. :)
>
>> PS: I did file a documentation bug referencing this thread, in case
>> you're wondering :-)
>
> What bug?

https://bugzilla.mozilla.org/show_bug.cgi?id=728066 would be the bug.

Axel

_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Axel Hecht
In reply to this post by Armen Zambrano G.
On 17.02.12 20:44, Armen Zambrano G. wrote:
>  From release automation it would be helpful if:
> * mozilla-beta would take automatically the "suggested sign-off" OR the
> one signed off by the localizer
>
> Would this be possible?
>
> cheers,
> Armen

Why would this be useful for release automation?

Axel
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Staś Małolepszy
In reply to this post by Axel Hecht

(xposting to tools-elmo)


On 02/17/2012 09:38 AM, Axel Hecht wrote:

> We're *so* close to having a team page with the interesting info right
> there on one page.
>
> The real downside right now is that it takes 13 seconds to load at least
> for the french team. That's not cool, so I'm optimizing. And finding
> what we're currently calling "suggested sign-off" on the sign-off page
> is rather expensive, so I looked deeper, and remembered that I found
> that suggestion to be wrong actually in a few cases. Thus I started the
> discussion here.

I wonder if this means that we're hitting a bottleneck that's blocking
us from releasing the new version of the teams page with all its goodness.

I see three solutions:

   1. release as-is, and optimize next.  The teams page will feel a bit
slow in some cases, but the goodness is there for all to see and benefit
from.

   2. rip off the 'suggested sign-off' feature and work on it next.  The
page will be fast and it will still provide a lot of good data in a very
good-looking and organized form.  Much better than it is right now, even
if without the suggested sign-off feature.

   3. make 'suggested sign-off' async, maybe trigger it with a click?
try to find a solution that would combine the best of the two above.

-stas



_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Axel Hecht
In reply to this post by Axel Hecht
On 21.02.12 15:30, Staś Małolepszy wrote:

>
> (xposting to tools-elmo)
>
>
> On 02/17/2012 09:38 AM, Axel Hecht wrote:
>
>> We're *so* close to having a team page with the interesting info right
>> there on one page.
>>
>> The real downside right now is that it takes 13 seconds to load at least
>> for the french team. That's not cool, so I'm optimizing. And finding
>> what we're currently calling "suggested sign-off" on the sign-off page
>> is rather expensive, so I looked deeper, and remembered that I found
>> that suggestion to be wrong actually in a few cases. Thus I started the
>> discussion here.
>
> I wonder if this means that we're hitting a bottleneck that's blocking
> us from releasing the new version of the teams page with all its goodness.
>
> I see three solutions:
>
> 1. release as-is, and optimize next. The teams page will feel a bit slow
> in some cases, but the goodness is there for all to see and benefit from.
>
> 2. rip off the 'suggested sign-off' feature and work on it next. The
> page will be fast and it will still provide a lot of good data in a very
> good-looking and organized form. Much better than it is right now, even
> if without the suggested sign-off feature.
>
> 3. make 'suggested sign-off' async, maybe trigger it with a click? try
> to find a solution that would combine the best of the two above.

.... or just land the branch that's requesting review from peterbe that
makes our perf awesome.
https://bugzilla.mozilla.org/show_bug.cgi?id=713878 for the curious.

Axel

_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Revisiting "suggested sign-off" on the dashboard

Armen Zambrano G.
In reply to this post by Axel Hecht
On 12-02-17 5:49 PM, Axel Hecht wrote:

> On 17.02.12 20:44, Armen Zambrano G. wrote:
>> From release automation it would be helpful if:
>> * mozilla-beta would take automatically the "suggested sign-off" OR the
>> one signed off by the localizer
>>
>> Would this be possible?
>>
>> cheers,
>> Armen
>
> Why would this be useful for release automation?
>
> Axel
*sigh* I removed the 2nd point by mistake. I guess this is orthogonal.

If there was a "FF11 beta" milestone (we would not need the 6 beta
milestones) and we could query the "suggested sign-off"/"signed off
revision" milestone then we could simplify the process. I think.
I guess, I am trying to see if we need the same mozilla-release sign-off
process for betas.

I assume the process for mozilla-release would still be the same.

How does the creation of milestones and sign-offs work?
Does compare-locales fail on changes that otherwise L10n-drivers would
reject?

cheers and thanks in advance,
Armen
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
12