New QE-Verify Flag in Bugzilla

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

New QE-Verify Flag in Bugzilla

Marc Schifer
  A short while ago we had a discussion about how to stream line some QA processes (Desktop QA - Bugzilla keywords: Proposal on dev-quality) and reduce the bug spam generated by setting keywords and whiteboard tags (Bugspam generated by the new process on firefox-dev). The final resolution of this discussion was to have a new Bugzilla flag created, qe-verify, that would replace the current use of QA+/- whiteboard tags and the verifyme keyword. This will give you the ability to easily filter out any activity on the flag instead of needing to have multiple filters for keywords and whiteboard activity as well simplify our queries for tracking work to be done in QE. The use of other keywords (qawanted, qaurgent, steps-wanted, regression-window-wanted, testcase-wanted) will remain the same.

Currently this is only implemented for Firefox QE, other teams are more then welcome to adopt if so desired.

Details:
 New Flag: qe-verify[?|+|-]
  qe-verify[?] = triage request
  qe-verify[+] = bug to be verified
  qe-verify[-] = bug will not/can not be verified

 Deprecate use of:
  Whiteboard tag: QA[?|+|-]
  Keyword       : verifyme  

 The component list the flag can be set for is:

  Core                   -- Any --
  Firefox                -- Any --
  Firefox for Android    -- Any --
  Firefox for Metro      -- Any --
  Firefox Health Report  -- Any --
  Firefox OS             -- Any --
  Loop                   Client
  Loop                   General
  Mozilla Localizations  -- Any --
  Mozilla QA             Mozmill Tests
  Mozilla Services       -- Any --
  NSPR                   -- Any --
  NSS                    -- Any --
  Plugins                -- Any --
  Snippets               -- Any --
  Toolkit                -- Any --

If there is a component that I missed please let me or the bugzilla team know so we can fix it.

This message was posted to dev-quality and has been crossposted to on firefox-dev and dev-platform. Please follow up on dev-quality.

Thank You.
 Marc S.
Firefox Q.E. Manager

_______________________________________________
dev-quality mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-quality
Reply | Threaded
Open this post in threaded view
|

Re: New QE-Verify Flag in Bugzilla

Liz Henry
Marc,

Sounds good! I'll change the documentation on my test plan.

I guess people who don't want to see bugmail from qa triaging can filter
out changes to this field as they could with the QA Whiteboard field.

I'm also hoping we can make some small changes in Bugzilla to help with
this workflow. Right now having to expand the "Flags" field on every
bug, then click again on qe-verify, then click a 3rd time to select +,
-, or ? is a bit laborious for something I'm going to need to do on
hundreds of bugs. My suggestions are that it would help to keep "Flags"
expanded (or allow a preference setting to have it expanded by default),
and to also allow batch editing on the qe-verify flag (Bug 281791).


Best,


Liz


On 8/14/14 2:45 PM, Marc Schifer wrote:

> A short while ago we had a discussion about how to stream line some
> QA processes (Desktop QA - Bugzilla keywords: Proposal on
> dev-quality) and reduce the bug spam generated by setting keywords
> and whiteboard tags (Bugspam generated by the new process on
> firefox-dev). The final resolution of this discussion was to have a
> new Bugzilla flag created, qe-verify, that would replace the current
> use of QA+/- whiteboard tags and the verifyme keyword. This will give
> you the ability to easily filter out any activity on the flag instead
> of needing to have multiple filters for keywords and whiteboard
> activity as well simplify our queries for tracking work to be done in
> QE. The use of other keywords (qawanted, qaurgent, steps-wanted,
> regression-window-wanted, testcase-wanted) will remain the same.
>
> Currently this is only implemented for Firefox QE, other teams are
> more then welcome to adopt if so desired.
>
> Details: New Flag: qe-verify[?|+|-] qe-verify[?] = triage request
> qe-verify[+] = bug to be verified qe-verify[-] = bug will not/can not
> be verified
>
> Deprecate use of: Whiteboard tag: QA[?|+|-] Keyword       : verifyme
>
>
> The component list the flag can be set for is:
>
> Core                   -- Any -- Firefox                -- Any --
> Firefox for Android    -- Any -- Firefox for Metro      -- Any --
> Firefox Health Report  -- Any -- Firefox OS             -- Any --
> Loop                   Client Loop                   General Mozilla
> Localizations  -- Any -- Mozilla QA             Mozmill Tests Mozilla
> Services       -- Any -- NSPR                   -- Any -- NSS
> -- Any -- Plugins                -- Any -- Snippets               --
> Any -- Toolkit                -- Any --
>
> If there is a component that I missed please let me or the bugzilla
> team know so we can fix it.
>
> This message was posted to dev-quality and has been crossposted to on
> firefox-dev and dev-platform. Please follow up on dev-quality.
>
> Thank You. Marc S. Firefox Q.E. Manager
>
> _______________________________________________ firefox-dev mailing
> list [hidden email]
> https://mail.mozilla.org/listinfo/firefox-dev
>


--
Liz Henry
[hidden email]
_______________________________________________
dev-quality mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-quality
Reply | Threaded
Open this post in threaded view
|

Re: New QE-Verify Flag in Bugzilla

glob
Liz Henry wrote:
> Marc,
>
> Sounds good! I'll change the documentation on my test plan.
>
> I guess people who don't want to see bugmail from qa triaging can filter
> out changes to this field as they could with the QA Whiteboard field.
this isn't possible yet (bug 1054138).
> My suggestions are that it would help to keep "Flags" expanded
i'm not keen on this change; it makes the ui more complicated for little
benefit to most users.
a preference would be possible, however it's unlikely to be made a priority.

perhaps a user-script would be more suitable?

--
byron jones - :glob - bugzilla.mozilla.org team -

_______________________________________________
dev-quality mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-quality
Reply | Threaded
Open this post in threaded view
|

Re: New QE-Verify Flag in Bugzilla

Robert Kaiser
In reply to this post by Marc Schifer
Marc Schifer schrieb:
>   New Flag: qe-verify[?|+|-]

Note that my prototype dashboard - currently residing on
https://crash-analysis.mozilla.com/rkaiser/crash-report-tools/qa/ -
includes this flag in its queries already.

Also note that this change does away with the [qa!] state, which can be
seen through the bug itself and/or the relevant train status flags
getting set to verified (in the case the bug gets, it's good that
qe-verify+ stays on it so we know we need to take a look again when it
gets fixed).

KaiRo
_______________________________________________
dev-quality mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-quality
Reply | Threaded
Open this post in threaded view
|

Re: New QE-Verify Flag in Bugzilla

Jared Wein
In reply to this post by Marc Schifer
Hi Marc,

Can you shed some background on the change in name from QA to QE? Does QE stand for Quality Engineering, whereas QA is Quality Assurance? What are the differences between the two? Do we now have two different teams/departments?

Thanks,
Jared

----- Original Message -----

> From: "Marc Schifer" <[hidden email]>
> To: "dev-quality" <[hidden email]>
> Cc: [hidden email], "dev-platform" <[hidden email]>, [hidden email]
> Sent: Thursday, August 14, 2014 5:45:52 PM
> Subject: New QE-Verify Flag in Bugzilla
>
> A short while ago we had a discussion about how to stream line some QA
> processes (Desktop QA - Bugzilla keywords: Proposal on dev-quality) and
> reduce the bug spam generated by setting keywords and whiteboard tags
> (Bugspam generated by the new process on firefox-dev). The final resolution
> of this discussion was to have a new Bugzilla flag created, qe-verify, that
> would replace the current use of QA+/- whiteboard tags and the verifyme
> keyword. This will give you the ability to easily filter out any activity on
> the flag instead of needing to have multiple filters for keywords and
> whiteboard activity as well simplify our queries for tracking work to be
> done in QE. The use of other keywords (qawanted, qaurgent, steps-wanted,
> regression-window-wanted, testcase-wanted) will remain the same.
>
> Currently this is only implemented for Firefox QE, other teams are more then
> welcome to adopt if so desired.
>
> Details:
>  New Flag: qe-verify[?|+|-]
>   qe-verify[?] = triage request
>   qe-verify[+] = bug to be verified
>   qe-verify[-] = bug will not/can not be verified
>
>  Deprecate use of:
>   Whiteboard tag: QA[?|+|-]
>   Keyword       : verifyme
>
>  The component list the flag can be set for is:
>
>   Core                   -- Any --
>   Firefox                -- Any --
>   Firefox for Android    -- Any --
>   Firefox for Metro      -- Any --
>   Firefox Health Report  -- Any --
>   Firefox OS             -- Any --
>   Loop                   Client
>   Loop                   General
>   Mozilla Localizations  -- Any --
>   Mozilla QA             Mozmill Tests
>   Mozilla Services       -- Any --
>   NSPR                   -- Any --
>   NSS                    -- Any --
>   Plugins                -- Any --
>   Snippets               -- Any --
>   Toolkit                -- Any --
>
> If there is a component that I missed please let me or the bugzilla team know
> so we can fix it.
>
> This message was posted to dev-quality and has been crossposted to on
> firefox-dev and dev-platform. Please follow up on dev-quality.
>
> Thank You.
>  Marc S.
> Firefox Q.E. Manager
>
> _______________________________________________
> firefox-dev mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/firefox-dev
>
_______________________________________________
dev-quality mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-quality
Reply | Threaded
Open this post in threaded view
|

Re: New QE-Verify Flag in Bugzilla

Clint Talbert-3
Hello all,

I know that there is a bunch of confusion in the wake of the recent reorganization of b2g, so let me set things straight. There is no new QE department. The QA team is still the same folks, although part of it is in the B2G org, part of it is in the Services org, and the core QA team are still in the Platform org.

This little "QE" moniker that was used here is indicating the aspirational shift that we'd like to make to move Quality Analysis at Mozilla toward a more "Quality Engineering" orientation. This means we are trying to shift toward deep diving into the technology behind the features we're testing, making better use of automation, being more proactive, being more data & metrics driven, being more experimental in ways that we do the work we do (with community, trying new approaches), etc.

This "engineering" orientation toward Quality will be a hallmark of the unified voice for Quality at Mozilla that we are striving to create. I know that many developers across the org are also interested in this, and we invite you to be a part of that too.

There will be much more to come on all of this, and you should start to see some small things changing in the Quality teams as we move toward that "Engineering" direction. We aren't going to change the names of all the "QA" things right now, we are going to focus on earning the "engineering" moniker first before we change anything else.

Hope that clears up the confusion.

Clint

----- Original Message -----
From: "Jared Wein" <[hidden email]>
To: "Marc Schifer" <[hidden email]>
Cc: [hidden email], "dev-platform" <[hidden email]>, "dev-quality" <[hidden email]>, [hidden email]
Sent: Tuesday, August 19, 2014 11:03:58 AM
Subject: Re: New QE-Verify Flag in Bugzilla

Hi Marc,

Can you shed some background on the change in name from QA to QE? Does QE stand for Quality Engineering, whereas QA is Quality Assurance? What are the differences between the two? Do we now have two different teams/departments?

Thanks,
Jared

----- Original Message -----

> From: "Marc Schifer" <[hidden email]>
> To: "dev-quality" <[hidden email]>
> Cc: [hidden email], "dev-platform" <[hidden email]>, [hidden email]
> Sent: Thursday, August 14, 2014 5:45:52 PM
> Subject: New QE-Verify Flag in Bugzilla
>
> A short while ago we had a discussion about how to stream line some QA
> processes (Desktop QA - Bugzilla keywords: Proposal on dev-quality) and
> reduce the bug spam generated by setting keywords and whiteboard tags
> (Bugspam generated by the new process on firefox-dev). The final resolution
> of this discussion was to have a new Bugzilla flag created, qe-verify, that
> would replace the current use of QA+/- whiteboard tags and the verifyme
> keyword. This will give you the ability to easily filter out any activity on
> the flag instead of needing to have multiple filters for keywords and
> whiteboard activity as well simplify our queries for tracking work to be
> done in QE. The use of other keywords (qawanted, qaurgent, steps-wanted,
> regression-window-wanted, testcase-wanted) will remain the same.
>
> Currently this is only implemented for Firefox QE, other teams are more then
> welcome to adopt if so desired.
>
> Details:
>  New Flag: qe-verify[?|+|-]
>   qe-verify[?] = triage request
>   qe-verify[+] = bug to be verified
>   qe-verify[-] = bug will not/can not be verified
>
>  Deprecate use of:
>   Whiteboard tag: QA[?|+|-]
>   Keyword       : verifyme
>
>  The component list the flag can be set for is:
>
>   Core                   -- Any --
>   Firefox                -- Any --
>   Firefox for Android    -- Any --
>   Firefox for Metro      -- Any --
>   Firefox Health Report  -- Any --
>   Firefox OS             -- Any --
>   Loop                   Client
>   Loop                   General
>   Mozilla Localizations  -- Any --
>   Mozilla QA             Mozmill Tests
>   Mozilla Services       -- Any --
>   NSPR                   -- Any --
>   NSS                    -- Any --
>   Plugins                -- Any --
>   Snippets               -- Any --
>   Toolkit                -- Any --
>
> If there is a component that I missed please let me or the bugzilla team know
> so we can fix it.
>
> This message was posted to dev-quality and has been crossposted to on
> firefox-dev and dev-platform. Please follow up on dev-quality.
>
> Thank You.
>  Marc S.
> Firefox Q.E. Manager
>
> _______________________________________________
> firefox-dev mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/firefox-dev
>
_______________________________________________
dev-platform mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-platform
_______________________________________________
dev-quality mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-quality