Locking bugs temporarily?

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Locking bugs temporarily?

Gervase Markham
I have an extension which integrates Bugzilla with another system. At
the moment, it's possible for me to get a Bug object and, while I am
working out how to change it as part of the sync, for someone else to
update the bug. My changes then overwrite theirs. Obviously, there's a
fairly small time window, but occurrences of this have been noted.

I can't really implement conflict resolution logic - that needs a human.
So I'd really like to lock the bug when I get a copy, make my update
e.g. 2 seconds later, unlock it, for their change to then try and be
committed and for _them_ (the human) to get a mid-air that they can
resolve however they like.

Is it possible to temporarily lock a bug? Either within Bugzilla or
using a direct database command on a table row? Would that screw things
up? Is there a better way of solving this?

We are using MySQL and, at the moment, Bugzilla 3.6.

Thanks,

Gerv
_______________________________________________
dev-apps-bugzilla mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-bugzilla
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

David Marshall-5
I don't like hard locks such as what you get with SELECT ... FOR UPDATE
because they're better for database integrity than for mutual exclusion.

Fortunately, you can use MySQL as a source of mutual exclusion with the
GET_LOCK function:

http://dev.mysql.com/doc/refman/5.1/en/miscellaneous-functions.html#function
_get-lock

SELECT GET_LOCK("bug:12345", 10) [or whatever]

Of course, that's an advisory lock only! It's up to you to identify exactly
which transactions should respect the lock.

PostgreSQL implements advisory locks somewhat differently:

http://www.postgresql.org/docs/9.1/static/explicit-locking.html#ADVISORY-LOC
KS

A truly enterprising chap could add subroutines Bugzilla::Object::lock,
Bugzilla:Object::is_locked, and Bugzilla::Object::release_lock that take
advantage of subroutines in Bugzilla::DB.

I'm guessing that the overhead in fiddling with advisory locks is pretty
low, albeit non-zero. I haven't though about it enough to say whether it's
worth it for objects to deal with them by default.

At Yahoo!, in addition to bug mid-airs, we also implement our own product
mid-airs. With 4832 products (as of this morning), we have "product
administrators" who can modify products. Some products have dozens of
administrators, who can and do occasionally try to modify the same thing at
the same time.

On 2/24/12 6:30 AM, "Gervase Markham" <[hidden email]> wrote:

> I have an extension which integrates Bugzilla with another system. At
> the moment, it's possible for me to get a Bug object and, while I am
> working out how to change it as part of the sync, for someone else to
> update the bug. My changes then overwrite theirs. Obviously, there's a
> fairly small time window, but occurrences of this have been noted.
>
> I can't really implement conflict resolution logic - that needs a human.
> So I'd really like to lock the bug when I get a copy, make my update
> e.g. 2 seconds later, unlock it, for their change to then try and be
> committed and for _them_ (the human) to get a mid-air that they can
> resolve however they like.
>
> Is it possible to temporarily lock a bug? Either within Bugzilla or
> using a direct database command on a table row? Would that screw things
> up? Is there a better way of solving this?
>
> We are using MySQL and, at the moment, Bugzilla 3.6.
>
> Thanks,
>
> Gerv
> _______________________________________________
> dev-apps-bugzilla mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-apps-bugzilla
> -
> To view or change your list settings, click here:
> <http://bugzilla.org/cgi-bin/mj_wwwusr?user=dmarshal@...>

_______________________________________________
dev-apps-bugzilla mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-bugzilla
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

Max Kanat-Alexander
In reply to this post by Gervase Markham
        My original plan for the API on this was to allow people to pass a
parameter to Bug.update, perhaps "check_conflicts => (time)", which
would then return an error if there had been changes since the specified
time. The error would include the changes that had been made, in enough
detail that an external app could re-create the "mid-air collision" page
that Bugzilla currently has in its UI.

        One of the pre-requisites to this would be moving the mid-air collision
logic into something re-usable in Bugzilla::Bug or something.

        -Max

On 02/24/2012 06:30 AM, Gervase Markham wrote:

> I have an extension which integrates Bugzilla with another system. At
> the moment, it's possible for me to get a Bug object and, while I am
> working out how to change it as part of the sync, for someone else to
> update the bug. My changes then overwrite theirs. Obviously, there's a
> fairly small time window, but occurrences of this have been noted.
>
> I can't really implement conflict resolution logic - that needs a human.
> So I'd really like to lock the bug when I get a copy, make my update
> e.g. 2 seconds later, unlock it, for their change to then try and be
> committed and for _them_ (the human) to get a mid-air that they can
> resolve however they like.
>
> Is it possible to temporarily lock a bug? Either within Bugzilla or
> using a direct database command on a table row? Would that screw things
> up? Is there a better way of solving this?
>
> We are using MySQL and, at the moment, Bugzilla 3.6.
>
> Thanks,
>
> Gerv
> _______________________________________________
> dev-apps-bugzilla mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-apps-bugzilla
> -
> To view or change your list settings, click here:
> <http://bugzilla.org/cgi-bin/mj_wwwusr?user=mkanat@...>


--
Max Kanat-Alexander
Chief Architect, Community Lead, and Release Manager
Bugzilla Project
http://www.bugzilla.org/
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

Gervase Markham
In reply to this post by Gervase Markham
Hi Max,

Thanks for taking the time - I know you are busy :-)

On 26/02/12 04:50, Max Kanat-Alexander wrote:
> My original plan for the API on this was to allow people to pass a
> parameter to Bug.update, perhaps "check_conflicts => (time)", which
> would then return an error if there had been changes since the specified
> time. The error would include the changes that had been made, in enough
> detail that an external app could re-create the "mid-air collision" page
> that Bugzilla currently has in its UI.

At the moment, if you pass a modification_time, the BzAPI will do
collision checking and return an error if there's a collision. (It's up
to the client to get a new copy of the bug, figure out how to merge the
changes, perhaps with user assistance, and resubmit the request.) If you
don't pass a modification_time, it does an unconditional overwrite.

However, this doesn't solve the problem - my extension does not have the
ability to talk to a "user" to resolve any mid-airs. Therefore, the
mid-air needs to appear on the screen of the _other_ guy trying to
change the bug - the real person. That means, I need to have a "lock" or
something on the bug while I'm changing it, so that when the lock is
released, _their_ change fails. I don't want users normally using the
web interface to get a lock, only to respect my lock! :-))

Any thoughts on how to do that?

Gerv
_______________________________________________
dev-apps-bugzilla mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-bugzilla
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

Gervase Markham
In reply to this post by Gervase Markham
On 24/02/12 16:47, David Marshall wrote:
> Fortunately, you can use MySQL as a source of mutual exclusion with the
> GET_LOCK function:

Interesting :-)

So the way to do it would be to change process.cgi and put a GET_LOCK
call before getting data about each bug, and a RELEASE_LOCK call after
updating it. Then, I could do the same in my extension. That way, only
one bit of code could be updating a bug at once. Is that the idea?

Do we currently have race conditions in process_bug? It grabs copies of
all the necessary bugs (around line 112), checks for mid-airs (around
line 150), but it only then opens a database transaction a bit later (on
line 543). Surely some other process could perhaps update the bug in
between, in which case some stale data might get written back to the bug?

Gerv
_______________________________________________
dev-apps-bugzilla mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-bugzilla
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

Frédéric Buclin
In reply to this post by Gervase Markham
Le 27. 02. 12 10:41, Gervase Markham a écrit :
> mid-air needs to appear on the screen of the _other_ guy trying to
> change the bug - the real person. That means, I need to have a "lock" or
> something on the bug while I'm changing it, so that when the lock is
> released, _their_ change fails.

I don't see why BzAPI should have a higher priority on someone else's
changes. You are not the only one to use remote clients to interact with
Bugzilla, and the other client would have the same problem as you. So if
you come after the other user, *you* should get the midair collision,
not the other user.

About GET_LOCK, depending on how it's used, you could easily block all
changes to a single bug, if this lock isn't released for some reason. It
sounds like a come back of LOCK TABLES to me, though not exactly the
same thing.


LpSolit
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

Gervase Markham
In reply to this post by Gervase Markham
On 27/02/12 10:41, Frédéric Buclin wrote:

> Le 27. 02. 12 10:41, Gervase Markham a écrit :
>> mid-air needs to appear on the screen of the _other_ guy trying to
>> change the bug - the real person. That means, I need to have a "lock" or
>> something on the bug while I'm changing it, so that when the lock is
>> released, _their_ change fails.
>
> I don't see why BzAPI should have a higher priority on someone else's
> changes. You are not the only one to use remote clients to interact with
> Bugzilla, and the other client would have the same problem as you. So if
> you come after the other user, *you* should get the midair collision,
> not the other user.

There appears to be some confusion here. My question is not about BzAPI,
although part of the discussion with Max turned to what the BzAPI does.

My question is more about my Sync extension. The reason the changes made
by this extension need to have priority is that they are automatic
syncings from another system, not in real time. I can't go back to that
other system and say "you know the guy who committed a change 5 minutes
ago? Get him back from his tea break, put him in front of a computer and
get him to resolve this conflict". However, I can get the guy who is
making a change in the 1 or 2 seconds between me requesting a copy of
the bug data and me writing it - because he's still sitting at his
computer, in front of a system I control (Bugzilla).

So Sync changes need to always succeed, which means the real-life user
has to get the mid-air.

> About GET_LOCK, depending on how it's used, you could easily block all
> changes to a single bug, if this lock isn't released for some reason. It
> sounds like a come back of LOCK TABLES to me, though not exactly the
> same thing.

I wonder if it's possible to make the locks automatically time out after
a certain length of time?

Gerv

_______________________________________________
dev-apps-bugzilla mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-bugzilla
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=$MSGRCPT>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

Marc Schumann
2012/2/27 Gervase Markham <[hidden email]>

> My question is more about my Sync extension. The reason the changes made
> by this extension need to have priority is that they are automatic
> syncings from another system, not in real time. I can't go back to that
> other system and say "you know the guy who committed a change 5 minutes
> ago? Get him back from his tea break, put him in front of a computer and
> get him to resolve this conflict". However, I can get the guy who is
> making a change in the 1 or 2 seconds between me requesting a copy of
> the bug data and me writing it - because he's still sitting at his
> computer, in front of a system I control (Bugzilla).
>
> So Sync changes need to always succeed, which means the real-life user
> has to get the mid-air.
>

In order to avoid a mid-air, you need to lock as soon as changes start
happening on your syncing system. Otherwise, changes might conflict. And if
you say that change might have happened 5 minutes ago, then that's way too
long a time for a lock imho.

On a related note, what do you want to happen if two sync users conflict?

   Marc
_______________________________________________
dev-apps-bugzilla mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-bugzilla
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

Max Kanat-Alexander
In reply to this post by Gervase Markham
On 02/27/2012 01:41 AM, Gervase Markham wrote:
> However, this doesn't solve the problem - my extension does not have the
> ability to talk to a "user" to resolve any mid-airs. Therefore, the
> mid-air needs to appear on the screen of the _other_ guy trying to
> change the bug - the real person. That means, I need to have a "lock" or
> something on the bug while I'm changing it, so that when the lock is
> released, _their_ change fails. I don't want users normally using the
> web interface to get a lock, only to respect my lock! :-))

        Okay. I'm not sure why the user isn't getting a mid-air. Are you not
updating delta_ts? It's theoretically possible for Bugzilla to have
conflicting transactions that both go through and both happen after the
mid-air check, but the probability is extremely small. Other than that,
as long as you're updating delta_ts, there's no way for two changes to
happen without mid-airs unless both of them are API clients who aren't
respecting the mid-air protection.

        -Max
--
Max Kanat-Alexander
Chief Architect, Community Lead, and Release Manager
Bugzilla Project
http://www.bugzilla.org/
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

Max Kanat-Alexander
In reply to this post by Gervase Markham
On 02/24/2012 06:30 AM, Gervase Markham wrote:
> At
> the moment, it's possible for me to get a Bug object and, while I am
> working out how to change it as part of the sync, for someone else to
> update the bug. My changes then overwrite theirs.

        Ah, I see. That must be an abnormally long time window in order for
this to happen any noticeable amount. What amount of time are we talking
about, and what's the nature of the plugin that causes it to take that
much time to make the updating decision? Should it perhaps re-fetch the
bug immediately before issuing an update() to make sure its logic is
still sound?

        -Max
--
Max Kanat-Alexander
Chief Architect, Community Lead, and Release Manager
Bugzilla Project
http://www.bugzilla.org/
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

Gervase Markham
In reply to this post by Gervase Markham
On 27/02/12 13:39, Marc Schumann wrote:
> In order to avoid a mid-air, you need to lock as soon as changes start
> happening on your syncing system. Otherwise, changes might conflict. And if
> you say that change might have happened 5 minutes ago, then that's way too
> long a time for a lock imho.

Sure; I'm only looking at locking things between the time I get a copy
of the bug for modification, and the time I write one back - a matter of
a second or two.

Sync errors of the sort you describe are easier to spot using timestamps.

> On a related note, what do you want to happen if two sync users conflict?

The updates from the remote system arrive periodically in one big bunch,
as a set of "current states", not a set of "deltas"; there are no
conflicts among them. (I.e. any problems at the other end have already
been sorted out.)

Gerv

_______________________________________________
dev-apps-bugzilla mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-bugzilla
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

Gervase Markham
In reply to this post by Gervase Markham
On 28/02/12 08:09, Max Kanat-Alexander wrote:
> Ah, I see. That must be an abnormally long time window in order for
> this to happen any noticeable amount. What amount of time are we talking
> about, and what's the nature of the plugin that causes it to take that
> much time to make the updating decision? Should it perhaps re-fetch the
> bug immediately before issuing an update() to make sure its logic is
> still sound?

I could shrink the window still further by implementing some sort of
final-delta_ts_check-and-retry-if-its-changed logic, which I may end up
doing, but I was wondering if there was a good locking system I could
use to shut the window entirely.

Given that the window inside Bugzilla isn't _totally_ shut, perhaps I'll
go with the above style solution.

Gerv
_______________________________________________
dev-apps-bugzilla mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-bugzilla
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

Eric Olson-4
In reply to this post by David Marshall-5
> I don't like hard locks such as what you get with SELECT ... FOR UPDATE
> because they're better for database integrity than for mutual exclusion.

I'm not sure I get why the application lock is preferred over the SELECT FOR UPDATE lock. It seems to me that method has several disadvantages. Since you're presumably obtaining a lock on the database row anyway (though maybe implicitly when you do an update), why not just use that mechanism?
_______________________________________________
dev-apps-bugzilla mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-bugzilla
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>
Reply | Threaded
Open this post in threaded view
|

Re: Locking bugs temporarily?

Max Kanat-Alexander
In reply to this post by Gervase Markham
On 02/28/2012 04:12 AM, Gervase Markham wrote:
> I could shrink the window still further by implementing some sort of
> final-delta_ts_check-and-retry-if-its-changed logic, which I may end up
> doing, but I was wondering if there was a good locking system I could
> use to shut the window entirely.
>
> Given that the window inside Bugzilla isn't _totally_ shut, perhaps I'll
> go with the above style solution.

        Yeah, I think in general you'll probably want to look at the sorts of
solutions that distributed systems use to probabilistically avoid conflicts.

        -Max
--
Max Kanat-Alexander
Chief Architect, Community Lead, and Release Manager
Bugzilla Project
http://www.bugzilla.org/
-
To view or change your list settings, click here:
<http://bugzilla.org/cgi-bin/mj_wwwusr?user=lists@...>