Signing off during (long) beta cycles

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Signing off during (long) beta cycles

Rimas Kudelis
Hi,

I tried to convince Axel about this, but he didn't seem to care much.
However, he hasn't convinced me that I'm wrong either, so I'm posting my
request here, in hope to gather more support.

My problem is simple: I really hate the requirement to sign off the
changeset I want to go into a beta. I can understand why we have to do
that for releases, but for betas? Especially now, when we're having like
10 or 20 of them in a row? Come on!

My suggestion is simple: during alpha/beta cycle, I want to be able to
say "please take my latest checkin which passes compare-locales and
consider it signed off". Axel's argument was that "it may regress
something", but you know... a) if I sign it off without much testing, it
will regress anyway, and b) at least from my PoV, a revision with say
five missing strings is obviosly preferred over one with 150.

Just to make it clear: I'm not asking for this to be the default policy,
I just want it to be an option. An option that is really convenient for
small teams (think one or two persons). I believe that even if I (or
anyone else, for that matter) choose this option, it won't really make
any negative difference, compared to manual sign-off procedure, for one
simple reason I stated above: if compare-locales doesn't catch my error,
and I'm signing off without testing, then the error will be introduced
anyway. On the other hand, if I do my tests, then nothing will prevent
me from catching my error myself, so it will most probably not get into
the beta anyway. So, in my opinion, this strict requirement to sign off
is nothing but a bureaucratic annoyance right now.

Add to that the fact that in Dashboard, Firefox doesn't fill in my
password (which I don't remember), and a trully subjective fact that
Panorama has shuffled all my tabs, and you get a really pissed off me.

After all, it's a BETA for God's sake! I shall serve my duty and issue
an explicit sign off for a release, but with pre-releases every two
weeks, I would like to relax a little, instead of being entitled to
remember when the next beta is going out.

Rimas
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Signing off during (long) beta cycles

Ehsan Akhgari
Rimas, please note that incorrect l10n changes potentially have the
power of making the main browser window unusable?  Do we really want to
release something which doesn't even show the main browser window to
thousands of beta users?

I'm all for making the process of signing off easier and faster, but I
don't think that the right approach here is skipping the process altogether.

Cheers,
Ehsan
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Signing off during (long) beta cycles

Cédric Corazza
In reply to this post by Rimas Kudelis
Rimas,
I have to disagree with you: I remember the infinite signoff threads on
m.d.l10n, and this new app is definitely a great move for localizers:
l10n teams can test the nightlies, make the signoff whenever they want,
several times, and the signoffs are processed at the most, 12 hours
later (let the l10n staff have some sleep ;-) ).
I do agree with Axel and Ehsan, we don't want to have regressions or
crashes, or whatever.
It happened that we made pushes and realized it broke the package, so we
really want make a signoff, not to have many bug reports.
This app obviously spares time for both l10n staff and l10n teams.
Could we make a better stuff? Maybe, but the solution you proposed is
not an enhancement imo.

Regards

Cédric
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Signing off during (long) beta cycles

Rimas Kudelis-4
In reply to this post by Rimas Kudelis
2010.09.23 23:39, Ehsan Akhgari rašė:
> Rimas, please note that incorrect l10n changes potentially have the
> power of making the main browser window unusable?  Do we really want to
> release something which doesn't even show the main browser window to
> thousands of beta users?
>
> I'm all for making the process of signing off easier and faster, but I
> don't think that the right approach here is skipping the process altogether.

Well, like I said:
1) I'm only talking about betas. A beta is a beta, its testers know that
it may have bugs.
2) I think most YSoD causes are now caught by compare-locales, no?
3) The requirement to sign off doesn't change much. After all, the
sign-off process is just a tool in our hands that can help us avoid
bugs. But it really can't force us to. I don't have to go through all
dialogs in the browser, or even run that particular build of it to be
able to click that button.

Furthermore, I don't want these options (sign off or go with the latest
check in) to be exclusive, I see them as complements. To visualize it a
little, here's a mockup of what the dashboard could look like:

+-------------------------------------------------------
| Timeline ....
|-------------+-------------+---------------+-----------
| Commit 1    | Commit 2    | Commit 3      | Commit 4
| 5 missing   | 10 missing  | 25 missing    | 1 ERROR!
|             |             |               |
| o Sign off  | o Sign off  | * Signed off  | Rejected
|             |             |               |
+-------------+-------------+-----------------------------------
| o Use the latest push that passes tests (not recommended!)
+---------------------------------------------------------------

With this, if let's say I found a bug in my latest commit, and I don't
have time for further commits until the next beta, I could just log into
the dashboard and sign off Commit 2 or Commit 3.

Rimas
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Signing off during (long) beta cycles

Rimas Kudelis-4
In reply to this post by Cédric Corazza
2010.09.24 00:04, Cédric Corazza rašė:
> Rimas,
> I have to disagree with you: I remember the infinite signoff threads on
> m.d.l10n, and this new app is definitely a great move for localizers:
> l10n teams can test the nightlies, make the signoff whenever they want,
> several times, and the signoffs are processed at the most, 12 hours
> later (let the l10n staff have some sleep ;-) ).

Yep, the app itself is a great evolutionary step over what we had. My
intention is not to take it away from you.

> I do agree with Axel and Ehsan, we don't want to have regressions or
> crashes, or whatever.
> It happened that we made pushes and realized it broke the package, so we
> really want make a signoff, not to have many bug reports.

Again, I don't want to take it away from you. I just don't like the fact
that my only choice is to either not have an up to date beta at all or
to go log in and press the sign off button before the beta is branched,
which means I have to follow the beta cycle and I have to remember my
LDAP password, both of which I'd rather not do. I'm still running my
nightlies, daily, and I plan to keep doing so. When I spot a bug that
slipped in, I fix it, and the Sign off button has nothing to do with it.

> This app obviously spares time for both l10n staff and l10n teams.
> Could we make a better stuff? Maybe, but the solution you proposed is
> not an enhancement imo.

I think you're imagining something else than I want you to. See my reply
to Ehsan.

Rimas
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Signing off during (long) beta cycles

Seth Bindernagel


Rimas Kudelis wrote:
> Again, I don't want to take it away from you. I just don't like the
> fact that my only choice is to either not have an up to date beta at
> all or to go log in and press the sign off button before the beta is
> branched, which means I have to follow the beta cycle and I have to
> remember my LDAP password, both of which I'd rather not do.

NB: My intentions is not to start a tangential debate about what we
should and shouldn't do...

...but not wanting to remember your password and not wanting to follow
the beta cycle seem like weak arguments.

If we were chatting in person where we could have a quicker
back-and-forth exchange, I'd ask you why you don't want to remember a
password (Is it that hard to do?) and why you don't want to follow a
beta cycle when you are the primary contact point for Lithuanian beta
users of Firefox.
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Signing off during (long) beta cycles

Axel Hecht
In reply to this post by Rimas Kudelis
Here's a few comments from me:

password manager is https://bugzilla.mozilla.org/show_bug.cgi?id=566290.
It doesn't annoy me and probably the majority of other folks as with
default cookie settings, you just stay logged in for two weeks. Not sure
how much pageload time we'd hit when fixing this. If you're fixing with
custom cookie settings, well, yeah, I can see that getting really
annoying, but that doesn't really bump that bug into a different triage
bucket, tbh.

On sign-offs:

Localizers signing off on a revision doesn't mean that we're shipping
that revision. It means that I (in case of fx, sea and tb is not me) go
through a complete diff and check for issues. If I find some, I reject,
usually with filing a bug on the issue to resolve. In some cases I
accept and file a bug, depends on the issue. Here's a few samples of
common issues I find:

-- regressions in search/region.properties
-- YSOD. We still have those, and they're not all machine-fixable (*)
-- misplaced translations. <!ENTITY foo.style "Breite: fünf Centimeter">
-- bad translations. en-US string changed, but l10n didn't. Recent
example was the offline rewording, where "File menu" was still there.
-- to a lesser extent bad params to printf

(*) YSOD, the nitty gritty details. I'm currently hacking on tests for
the new compare-locales version that will find much more YSODs than the
current one, BUT. Some errors need human intervention to see if they're
errors. Those are entity references to "unknown" entities. &rdquo; for
example. Is that known or not? Depends. Bad people could use the same
DTD file in an XHTML and XUL file, and once it's OK, and the second time
it's an error. I can find <!ENTITY foo.accesskey "&"> with the new
version, though.

So yes, we'll continue to work on making the bots be better, and we
intend to even add runtime testing, too. But, there's no way to catch
everything and sometimes it takes a more technical person to look at it.


One more point: Signing off should be something done responsibly.
Depending on how big a change you landed, and how much review you can do
yourself on the source, you may want to take a build with your latest
changes and smoketest it for a few. That said, I personally don't see a
requirement for the locale owner to do the sign-off. In your case, you
submitted 5 or 6 contributors and we could clearly find a way to let one
of them sign off. (not ad-hoc trivial, but still.)


I meant to ask, and didn't yet. I've been at times sending out mails to
people who apparently had more recent pushes than signed off on, but
I've never made up my mind on when it'd be a good time to send those
out. I wonder how many of the folks just read those and felt a "yeah,
right, I know, get off my lawn".


A few notes on the Beta program, too: Firstly, and most sadly, 2 weeks
my ass. Our beta cycle would be much easier for everybody if it actually
had a cadence.
Also, we might not have had the time back then to clearly word that the
beta program before the branch is pretty high-touch. Those locales
starting beta post-branch will have an easier tree to deal with, though
in less time.
If you'd be happier with just occasionally working on the nightlies and
join the beta program post-branch, I don't know. You do have a pretty
solid head count of 350 on the lt 4 betas.
As for upcoming releases, it's too early now to judge if the things we
expected the quick-cadence beta program to do turned out at least
somewhat like we hoped. All our previous plans didn't deliver on time,
either. We'll see when we shipped.



Overall, I think you've been overly optimistic on how good our automated
tools are. I'll take that as a compliment and I'll try to get there,
with the caveats above. The login-issue is something that you could work
around. Also, there seems to be a somewhat different expectation on what
a beta is in your point of view, and what we see users out there seeing.
In particular as more and more web apps out there are beta 'til end-of-life.


So, did I manage to write a more lengthy post? I guess, but I tricked
with whitespace, too.

HTH

Axel



On 23.09.10 15:22, Rimas Kudelis wrote:

> Hi,
>
> I tried to convince Axel about this, but he didn't seem to care much.
> However, he hasn't convinced me that I'm wrong either, so I'm posting my
> request here, in hope to gather more support.
>
> My problem is simple: I really hate the requirement to sign off the
> changeset I want to go into a beta. I can understand why we have to do
> that for releases, but for betas? Especially now, when we're having like
> 10 or 20 of them in a row? Come on!
>
> My suggestion is simple: during alpha/beta cycle, I want to be able to
> say "please take my latest checkin which passes compare-locales and
> consider it signed off". Axel's argument was that "it may regress
> something", but you know... a) if I sign it off without much testing, it
> will regress anyway, and b) at least from my PoV, a revision with say
> five missing strings is obviosly preferred over one with 150.
>
> Just to make it clear: I'm not asking for this to be the default policy,
> I just want it to be an option. An option that is really convenient for
> small teams (think one or two persons). I believe that even if I (or
> anyone else, for that matter) choose this option, it won't really make
> any negative difference, compared to manual sign-off procedure, for one
> simple reason I stated above: if compare-locales doesn't catch my error,
> and I'm signing off without testing, then the error will be introduced
> anyway. On the other hand, if I do my tests, then nothing will prevent
> me from catching my error myself, so it will most probably not get into
> the beta anyway. So, in my opinion, this strict requirement to sign off
> is nothing but a bureaucratic annoyance right now.
>
> Add to that the fact that in Dashboard, Firefox doesn't fill in my
> password (which I don't remember), and a trully subjective fact that
> Panorama has shuffled all my tabs, and you get a really pissed off me.
>
> After all, it's a BETA for God's sake! I shall serve my duty and issue
> an explicit sign off for a release, but with pre-releases every two
> weeks, I would like to relax a little, instead of being entitled to
> remember when the next beta is going out.
>
> Rimas

_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Signing off during (long) beta cycles

Zbigniew Braniecki-3
In reply to this post by Rimas Kudelis
Hi Rimas,

Rimas Kudelis wrote:
> Hi,
>
> I tried to convince Axel about this, but he didn't seem to care much.

Attributing bad intentions to someone who spends his days and nights on
the projects doesn't sound very friendly to be honest.


> My suggestion is simple: during alpha/beta cycle, I want to be able to
> say "please take my latest checkin which passes compare-locales and
> consider it signed off". Axel's argument was that "it may regress
> something", but you know... a) if I sign it off without much testing, it
> will regress anyway, and b) at least from my PoV, a revision with say
> five missing strings is obviosly preferred over one with 150.

As others pointed out, the main issue here is that we really don't know
if your revision works until we all test it. By all I mean, you, me,
Pike, our try-servers.

The good news is that as we progress in the direction of L20n we are
actually talking about reducing the risk. I honestly believe that our
l20n builds will be bullet-proof.
One of the goals I have for l20n in Firefox is that localization cannot
break the build. It's tricky, I know, especially once we talk about
localization injected into JS code, but hey, L20n is a bold goal by
definition. :>

So, if we succeed, and I see less reasons why we might not (we have a
branch with Firefox 4 running on L20n exclusively now!), I'll be happy
to revisit your suggestion.
Why not before?


> After all, it's a BETA for God's sake! I shall serve my duty and issue
> an explicit sign off for a release, but with pre-releases every two
> weeks, I would like to relax a little, instead of being entitled to
> remember when the next beta is going out.

Ah. Here's the thing. Beta. So, as much as I love the concept of betas,
I believe that no one has ever build a clear definition of the term and
each person uses his own. For you, it's "just beta", and "we have plenty
of them" and things can break from time to time, no worry.
Well, for me, we're talking about hundreds of thousands of users who we
explicitly try to attract to beta builds so that we have a better
testing sample, more input, also on localization quality and
intersection between new features and localization. Imagine user
experience of a user who wants to try beta version, or updates to the
next beta and loads YSOD instead. Or cannot click on anything.

I believe that our beta builds are public, and as a Mozilla project we
put a sticker on it - "ready to be tested" and we want people to feel
that we made sure it's ready. It's beta, yeah, new tab can sometimes
open in a wrong place, or theme may not be finalized, but it should work.

Saying that, I understand your concern and I believe that in the future
we should be able to allow you to opt-in for "take the latest signoff"
but that should go hand-in-hand with "I commit only after testing" rule
cause it moves the "signoff" act into hg push, and with localization not
be able to break builds.

Hope that makes sense.

Cheers,
g.
--

Mozilla (http://www.mozilla.org)
_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n
Reply | Threaded
Open this post in threaded view
|

Re: Signing off during (long) beta cycles

Rimas Kudelis
In reply to this post by Axel Hecht
2010.09.24 17:49, Axel Hecht rašė:
> Here's a few comments from me:
>
> password manager is https://bugzilla.mozilla.org/show_bug.cgi?id=566290.
> It doesn't annoy me and probably the majority of other folks as with
> default cookie settings, you just stay logged in for two weeks. Not sure
> how much pageload time we'd hit when fixing this. If you're fixing with
> custom cookie settings, well, yeah, I can see that getting really
> annoying, but that doesn't really bump that bug into a different triage
> bucket, tbh.

I don't think I use custom settings. Maybe you could at least increase
cookie lifetime? I was about to suggest something like two months, but
then thought that it could actually be more. Why not?

> On sign-offs:
>
> Localizers signing off on a revision doesn't mean that we're shipping
> that revision. It means that I (in case of fx, sea and tb is not me) go
> through a complete diff and check for issues. If I find some, I reject,
> usually with filing a bug on the issue to resolve. In some cases I
> accept and file a bug, depends on the issue.

Yeah, what I ment was that you could as well just go through the diff
against the latest revision regardless of whether or not I signed it off
explicitly. The workload is pretty much the same (unless I submit a
newer rev later, but then again, if I submit and sign it off, you get to
double-check anyway). Btw none of the replies here has touched this
particular aspect yet.

> So yes, we'll continue to work on making the bots be better, and we
> intend to even add runtime testing, too. But, there's no way to catch
> everything and sometimes it takes a more technical person to look at it.

Agreed. BTW I remember your recently filed bug where an entity reference
within a string was fixed in US, but I missed that fix. But I don't
think anything would have been different back then if it was the latest
revision and not the signed off one.

> One more point: Signing off should be something done responsibly.
> Depending on how big a change you landed, and how much review you can do
> yourself on the source, you may want to take a build with your latest
> changes and smoketest it for a few. That said, I personally don't see a
> requirement for the locale owner to do the sign-off. In your case, you
> submitted 5 or 6 contributors and we could clearly find a way to let one
> of them sign off. (not ad-hoc trivial, but still.)

Most of them are not contributors anymore. We have a 10 year old history
of Mozilla's localization; what I submitted was that history. I don't
even know what exactly most of those people did, and some of them I
don't know at all. Today, I'm the only really active contributor for lt.

> I meant to ask, and didn't yet. I've been at times sending out mails to
> people who apparently had more recent pushes than signed off on, but
> I've never made up my mind on when it'd be a good time to send those
> out. I wonder how many of the folks just read those and felt a "yeah,
> right, I know, get off my lawn".

Thanks for those reminders, though I don't think I've seen them in my
Inbox lately, did you stop sending them? Wrt timing, if we have string
freezes then I guess the best option would be to send one soon after we
hit the string freeze, and another one not long before cutting (or
however you call it).

> A few notes on the Beta program, too: Firstly, and most sadly, 2 weeks
> my ass. Our beta cycle would be much easier for everybody if it actually
> had a cadence.
> Also, we might not have had the time back then to clearly word that the
> beta program before the branch is pretty high-touch. Those locales
> starting beta post-branch will have an easier tree to deal with, though
> in less time.
> If you'd be happier with just occasionally working on the nightlies and
> join the beta program post-branch, I don't know. You do have a pretty
> solid head count of 350 on the lt 4 betas.

I'd just prefer to assume my fault in case I break something, but feel
more relaxed about the betas. Perhaps that would even help me attract
more people to form the community... :) Anyway, you know, maybe I'm just
too confident in myself, but I don't think I produce syntax errors. :)
Or at least I believe I hit that wall really rarely.

> Overall, I think you've been overly optimistic on how good our automated
> tools are. I'll take that as a compliment and I'll try to get there,
> with the caveats above. The login-issue is something that you could work
> around. Also, there seems to be a somewhat different expectation on what
> a beta is in your point of view, and what we see users out there seeing.
> In particular as more and more web apps out there are beta 'til
> end-of-life.

Yeah. I think I saw a comic or something exactly about that some time
ago... However, at least from my personal experience, our latest betas
aren't as stable as I would want them to be.

> So, did I manage to write a more lengthy post? I guess, but I tricked
> with whitespace, too.

I guess so. And it's pretty obvious that noone else is on my side.
Should I give up already?

Rimas

_______________________________________________
dev-l10n mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-l10n