hg diff getting slower ? (Debian GNU/linux kernel 4.7.0-1)

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

hg diff getting slower ? (Debian GNU/linux kernel 4.7.0-1)

ISHIKAWA,chiaki
Hi,

I have used hg (mercurial) satisfactorily until about a month ago or so.
Somehow hg diff has become very slow.
I am using it on two Debian GNU/Linux PC (64-bit).
It is possible that latest OK kernel update in Debian may have changed  
something like file cache policy, etc. But user visible change like this  
is rather unlikely.

I am using fsmonitor extension, and maybe there could have been changes  
related inotify in the recent kernels offered by Debian.
My version is linux 4.7.0-1
uname -a
Linux ip030 4.7.0-1-amd64 #1 SMP Debian 4.7.8-1 (2016-10-19) x86_64  
GNU/Linux

(Come to think of it, I should get a newer fsmonitor from facebook site
for 4.7.0-1 or later kernel.)

Or maybe the change in the kernel might have caused the changes in the  
optimal setting for
sysctl fs.inotify.max_user_watches= ???
sysctl fs.inotify.max_queued_events= ???
sysctl fs.inotify.max_user_instances= ???

Has anyone experienced such strange slowdown recently under Debian  
GNU/linux with 4.7.x kernel?
Or can someone suggest a modern optimal set of values of s.inotify.max*  ?


TIA




_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds
Reply | Threaded
Open this post in threaded view
|

Re: hg diff getting slower ? (Debian GNU/linux kernel 4.7.0-1)

Gregory Szorc-3
On Sat, Jan 7, 2017 at 9:31 AM, ISHIKAWA,chiaki <[hidden email]> wrote:
Hi,

I have used hg (mercurial) satisfactorily until about a month ago or so.
Somehow hg diff has become very slow.
I am using it on two Debian GNU/Linux PC (64-bit).
It is possible that latest OK kernel update in Debian may have changed something like file cache policy, etc. But user visible change like this is rather unlikely.

I am using fsmonitor extension, and maybe there could have been changes related inotify in the recent kernels offered by Debian.
My version is linux 4.7.0-1
uname -a
Linux ip030 4.7.0-1-amd64 #1 SMP Debian 4.7.8-1 (2016-10-19) x86_64 GNU/Linux

(Come to think of it, I should get a newer fsmonitor from facebook site
for 4.7.0-1 or later kernel.)

Or maybe the change in the kernel might have caused the changes in the optimal setting for
sysctl fs.inotify.max_user_watches= ???
sysctl fs.inotify.max_queued_events= ???
sysctl fs.inotify.max_user_instances= ???

Has anyone experienced such strange slowdown recently under Debian GNU/linux with 4.7.x kernel?
Or can someone suggest a modern optimal set of values of s.inotify.max*  ?

Running any `hg` command with `--profile` will invoke a Python profiler and print what's taking so long. I can help you decode that.

Keep in mind that if a pager is configured, time spent waiting for pager to exit will skew results. So you can disable pager by redirecting stdout to /dev/null or setting `--pager=false` Alternatively, setting HGRCPATH to an empty value will disable loading of your global config file, disabling extensions (like pager) and isolating behavior of vanilla Mercurial, which can help isolate performance issues as caused by extensions. In this case, fsmonitor *should* make things faster, so you could do something like `HGRCPATH= hg --config extensions.fsmonitor=` to disable everything then load fsmonitor explicitly.

It's not unheard of for fsmonitor to slow things down in some scenarios. This can happen on first run or if inotify is hitting limits. I use the following limits without any major problems:

fs.inotify.max_queued_events = 65536
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 512000


_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds
Reply | Threaded
Open this post in threaded view
|

Re: hg diff getting slower ? (Debian GNU/linux kernel 4.7.0-1)

ISHIKAWA,chiaki
On 2017/01/10 8:37, Gregory Szorc wrote:

> On Sat, Jan 7, 2017 at 9:31 AM, ISHIKAWA,chiaki <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi,
>
>     I have used hg (mercurial) satisfactorily until about a month ago or so.
>     Somehow hg diff has become very slow.
>     I am using it on two Debian GNU/Linux PC (64-bit).
>     It is possible that latest OK kernel update in Debian may have
>     changed something like file cache policy, etc. But user visible
>     change like this is rather unlikely.
>
>     I am using fsmonitor extension, and maybe there could have been
>     changes related inotify in the recent kernels offered by Debian.
>     My version is linux 4.7.0-1
>     uname -a
>     Linux ip030 4.7.0-1-amd64 #1 SMP Debian 4.7.8-1 (2016-10-19) x86_64
>     GNU/Linux
>
>     (Come to think of it, I should get a newer fsmonitor from facebook site
>     for 4.7.0-1 or later kernel.)
>
>     Or maybe the change in the kernel might have caused the changes in
>     the optimal setting for
>     sysctl fs.inotify.max_user_watches= ???
>     sysctl fs.inotify.max_queued_events= ???
>     sysctl fs.inotify.max_user_instances= ???
>
>     Has anyone experienced such strange slowdown recently under Debian
>     GNU/linux with 4.7.x kernel?
>     Or can someone suggest a modern optimal set of values of
>     s.inotify.max*  ?
>
>
> Running any `hg` command with `--profile` will invoke a Python profiler
> and print what's taking so long. I can help you decode that.
>
> Keep in mind that if a pager is configured, time spent waiting for pager
> to exit will skew results. So you can disable pager by redirecting
> stdout to /dev/null or setting `--pager=false` Alternatively, setting
> HGRCPATH to an empty value will disable loading of your global config
> file, disabling extensions (like pager) and isolating behavior of
> vanilla Mercurial, which can help isolate performance issues as caused
> by extensions. In this case, fsmonitor *should* make things faster, so
> you could do something like `HGRCPATH= hg --config
> extensions.fsmonitor=` to disable everything then load fsmonitor explicitly.
>
> It's not unheard of for fsmonitor to slow things down in some scenarios.
> This can happen on first run or if inotify is hitting limits. I use the
> following limits without any major problems:
>
> fs.inotify.max_queued_events = 65536
> fs.inotify.max_user_instances = 128
> fs.inotify.max_user_watches = 512000
>
>
>

I did both of the following .

1 - compiled the latest version of fsmonitor (after I fetched the latest
source).
    I compiled because the underlying kernel has been updated, and in so
    doing I thought using the latest fsmonitor was way to go.
   (I used fsmonitor under Debian GNU/Linux).

2  Also, I increased the limits per your suggestion.
    I am not sure if the value once set survives the rebooting.
   Maybe that was the reason for strange slow down.
   Anyway, max_queued_events and max_user_watches were way smaller than your
suggested value.

Now, git operation is as fast as it was.

Thank you again.



_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds
Reply | Threaded
Open this post in threaded view
|

Re: hg diff getting slower ? (Debian GNU/linux kernel 4.7.0-1)

Gregory Szorc-3
On Fri, Jan 27, 2017 at 6:55 AM, ISHIKAWA,chiaki <[hidden email]> wrote:
On 2017/01/10 8:37, Gregory Szorc wrote:
On Sat, Jan 7, 2017 at 9:31 AM, ISHIKAWA,chiaki <[hidden email]
<mailto:[hidden email]>> wrote:

    Hi,

    I have used hg (mercurial) satisfactorily until about a month ago or so.
    Somehow hg diff has become very slow.
    I am using it on two Debian GNU/Linux PC (64-bit).
    It is possible that latest OK kernel update in Debian may have
    changed something like file cache policy, etc. But user visible
    change like this is rather unlikely.

    I am using fsmonitor extension, and maybe there could have been
    changes related inotify in the recent kernels offered by Debian.
    My version is linux 4.7.0-1
    uname -a
    Linux ip030 4.7.0-1-amd64 #1 SMP Debian 4.7.8-1 (2016-10-19) x86_64
    GNU/Linux

    (Come to think of it, I should get a newer fsmonitor from facebook site
    for 4.7.0-1 or later kernel.)

    Or maybe the change in the kernel might have caused the changes in
    the optimal setting for
    sysctl fs.inotify.max_user_watches= ???
    sysctl fs.inotify.max_queued_events= ???
    sysctl fs.inotify.max_user_instances= ???

    Has anyone experienced such strange slowdown recently under Debian
    GNU/linux with 4.7.x kernel?
    Or can someone suggest a modern optimal set of values of
    s.inotify.max*  ?


Running any `hg` command with `--profile` will invoke a Python profiler
and print what's taking so long. I can help you decode that.

Keep in mind that if a pager is configured, time spent waiting for pager
to exit will skew results. So you can disable pager by redirecting
stdout to /dev/null or setting `--pager=false` Alternatively, setting
HGRCPATH to an empty value will disable loading of your global config
file, disabling extensions (like pager) and isolating behavior of
vanilla Mercurial, which can help isolate performance issues as caused
by extensions. In this case, fsmonitor *should* make things faster, so
you could do something like `HGRCPATH= hg --config
extensions.fsmonitor=` to disable everything then load fsmonitor explicitly.

It's not unheard of for fsmonitor to slow things down in some scenarios.
This can happen on first run or if inotify is hitting limits. I use the
following limits without any major problems:

fs.inotify.max_queued_events = 65536
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 512000




I did both of the following .

1 - compiled the latest version of fsmonitor (after I fetched the latest source).
   I compiled because the underlying kernel has been updated, and in so
   doing I thought using the latest fsmonitor was way to go.
  (I used fsmonitor under Debian GNU/Linux).

2  Also, I increased the limits per your suggestion.
   I am not sure if the value once set survives the rebooting.

If you just run `sysctl`, the changes will *not* survive a reboot. You'll need to add the modification to a file somewhere. /etc/sysctl.conf is a popular location for said file.
 
  Maybe that was the reason for strange slow down.
  Anyway, max_queued_events and max_user_watches were way smaller than your
suggested value.

Now, git operation is as fast as it was.

Thank you again.





_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds