Large memory requirement for spidermonkey 1.8.5

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Large memory requirement for spidermonkey 1.8.5

Manoj-11
Hi,
We are moving from js-1.7 to js-1.8.5. Our scripts typically process millions of objects. Observation is that js 1.8.5 takes huge runtime memory as compared to js-1.7. Often it crashes. We had to increase runtime memory 4 times so that scripts run smoothly. Is this normal? Is there any way to bring down memory requirement?


Manoj
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: Large memory requirement for spidermonkey 1.8.5

Kannan Vijayan
Hi Manoj,

SpiderMonkey 1.8.5 is already pretty dated.  Have you tried a more
recent release such as SpiderMonkey 17?  There have been significant
improvements in garbage collection and the optimizing JITs since 1.8.5.

Kannan

On 13-07-24 4:33 PM, Manoj wrote:
> Hi,
> We are moving from js-1.7 to js-1.8.5. Our scripts typically process millions of objects. Observation is that js 1.8.5 takes huge runtime memory as compared to js-1.7. Often it crashes. We had to increase runtime memory 4 times so that scripts run smoothly. Is this normal? Is there any way to bring down memory requirement?
>
>
> Manoj
> _______________________________________________
> dev-tech-js-engine mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-tech-js-engine

_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: Large memory requirement for spidermonkey 1.8.5

Manoj-11
In reply to this post by Manoj-11
On Wednesday, July 24, 2013 5:04:34 PM UTC-4, Kannan Vijayan wrote:

> Hi Manoj,
>
>
>
> SpiderMonkey 1.8.5 is already pretty dated.  Have you tried a more
>
> recent release such as SpiderMonkey 17?  There have been significant
>
> improvements in garbage collection and the optimizing JITs since 1.8.5.
>
>
>
> Kannan
>
>
>
> On 13-07-24 4:33 PM, Manoj wrote:
>
> > Hi,
>
> > We are moving from js-1.7 to js-1.8.5. Our scripts typically process millions of objects. Observation is that js 1.8.5 takes huge runtime memory as compared to js-1.7. Often it crashes. We had to increase runtime memory 4 times so that scripts run smoothly. Is this normal? Is there any way to bring down memory requirement?
>
> >
>
> >
>
> > Manoj
>
> > _______________________________________________
>
> > dev-tech-js-engine mailing list
>
> > [hidden email]
>
> > https://lists.mozilla.org/listinfo/dev-tech-js-engine

Will going to try spidermonkey 17. But looking at release notes of 1.8.5 I found that there are modifications in the structure of jsval. It's no longer a 32 bit. It has become a structure. Because of this the memory requirement definitely increase. Question is by what factor?
Our test case takes 4 times more memory on redhat and 2 times more memory on suse. Is there any justification for that?
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: Large memory requirement for spidermonkey 1.8.5

Wes Garland
jsvals are 64-bits now, but there will also be memory penalties due to the
fact that this is now a JIT compiler instead of an interpreter.

Even newer spidermonkeys do not include the precompiler, either, so they
may need to source code in RAM for function toStringing etc.

Wes

On 25 July 2013 12:27, Manoj <[hidden email]> wrote:

> On Wednesday, July 24, 2013 5:04:34 PM UTC-4, Kannan Vijayan wrote:
> > Hi Manoj,
> >
> >
> >
> > SpiderMonkey 1.8.5 is already pretty dated.  Have you tried a more
> >
> > recent release such as SpiderMonkey 17?  There have been significant
> >
> > improvements in garbage collection and the optimizing JITs since 1.8.5.
> >
> >
> >
> > Kannan
> >
> >
> >
> > On 13-07-24 4:33 PM, Manoj wrote:
> >
> > > Hi,
> >
> > > We are moving from js-1.7 to js-1.8.5. Our scripts typically process
> millions of objects. Observation is that js 1.8.5 takes huge runtime memory
> as compared to js-1.7. Often it crashes. We had to increase runtime memory
> 4 times so that scripts run smoothly. Is this normal? Is there any way to
> bring down memory requirement?
> >
> > >
> >
> > >
> >
> > > Manoj
> >
> > > _______________________________________________
> >
> > > dev-tech-js-engine mailing list
> >
> > > [hidden email]
> >
> > > https://lists.mozilla.org/listinfo/dev-tech-js-engine
>
> Will going to try spidermonkey 17. But looking at release notes of 1.8.5 I
> found that there are modifications in the structure of jsval. It's no
> longer a 32 bit. It has become a structure. Because of this the memory
> requirement definitely increase. Question is by what factor?
> Our test case takes 4 times more memory on redhat and 2 times more memory
> on suse. Is there any justification for that?
> _______________________________________________
> dev-tech-js-engine mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-tech-js-engine
>



--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: Large memory requirement for spidermonkey 1.8.5

Manoj-11
In reply to this post by Manoj-11
I understand. But the script is small. Hardly 15 lines. Shouldn't take much RAM.
Also 64 bit jsval doesn't justify consumption of 4 times more memory. Is there anything else that we don't know?


_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: Large memory requirement for spidermonkey 1.8.5

Jeff Walden-2
In reply to this post by Manoj-11
On 07/25/2013 09:27 AM, Manoj wrote:> But looking at release notes of 1.8.5 I found that there are modifications in the structure of jsval. It's no longer a 32 bit. It has become a structure. Because of this the memory requirement definitely increase.

All else equal, sure.  But not all else is equal.  Other things have decreased in size: objects particularly, stack frames as well.  Don't assume that just because the size of jsval changed, memory usage increased -- particularly not with exact 32-bits-per-value correspondence.

Jeff
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: Large memory requirement for spidermonkey 1.8.5

Manoj-11
On Thursday, July 25, 2013 6:41:09 PM UTC-4, Jeff Walden wrote:
> On 07/25/2013 09:27 AM, Manoj wrote:> But looking at release notes of 1.8.5 I found that there are modifications in the structure of jsval. It's no longer a 32 bit. It has become a structure. Because of this the memory requirement definitely increase.
>
>
>
> All else equal, sure.  But not all else is equal.  Other things have decreased in size: objects particularly, stack frames as well.  Don't assume that just because the size of jsval changed, memory usage increased -- particularly not with exact 32-bits-per-value correspondence.
>
>
>
> Jeff

Then what's the reason behind the increase in memory by 4 times? Also we tried with JS-17. It gives the same results. No improvements.
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: Large memory requirement for spidermonkey 1.8.5

Manoj-11
In reply to this post by Manoj-11
To elaborate more below is the script that we are running -
With 1 GB memory set for JS Runtime the script runs for 56 millions iterations on 1.7.0 before it crashes. With 1.8.5 and js - 17 it couldn't go beyond 9 million iterations.

var EmpObjects = new Object();//Global Object
var j = 0;
try
{
    for(var i = 0; i < 100000000; i++, j++)
    {
        if(j % 10000 == 0)
        {
            print("j == " + j );
        }
        var employee=new Object(); //Creating Employee Object
        employee.Name="John";
        employee.Surname="Methew";
        employee.Salary="10000";
        EmpObjects[i] = employee; //Pushing it to EmpObjects
    }
}
catch(e)
{
    print("j == " + j );
    print(e);
}
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: Large memory requirement for spidermonkey 1.8.5

Jeff Walden-2
In reply to this post by Manoj-11
On 07/26/2013 01:11 PM, Manoj wrote:
> Then what's the reason behind the increase in memory by 4 times? Also we tried with JS-17. It gives the same results. No improvements.

I don't know.  My point was that simply looking at the change in size of JS::Value and extrapolating an impact upon memory usage is highly misleading.  The only way to conclusively answer the question would be to debug a bunch and figure out exactly what's the cause.

With SpiderMonkey 17 in maintenance mode, it's extremely unlikely we'd backport any fixes for any bugs that might or might not be demonstrated, so we're very unlikely to attempt to debug this.  Not to put you on a treadmill or anything, but investigating how things work against trunk SpiderMonkey and filing a bug is the best way to figure out what's up.  (This may involve a bit more porting work, of course; we've started adding handles to the JSAPI, so many functions' signatures have begun to change in surface-incompatible ways.)  Even still, there's no guarantee of investigation: no free lunch, as they say, just a better chance of it, depending how closely it overlaps with existing JS team priorities.

Jeff
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: Large memory requirement for spidermonkey 1.8.5

Wes Garland
OP - Does the JS code run in the browser by any chance?

On 30 July 2013 19:10, Jeff Walden <[hidden email]> wrote:

> On 07/26/2013 01:11 PM, Manoj wrote:
> > Then what's the reason behind the increase in memory by 4 times? Also we
> tried with JS-17. It gives the same results. No improvements.
>
> I don't know.  My point was that simply looking at the change in size of
> JS::Value and extrapolating an impact upon memory usage is highly
> misleading.  The only way to conclusively answer the question would be to
> debug a bunch and figure out exactly what's the cause.
>
> With SpiderMonkey 17 in maintenance mode, it's extremely unlikely we'd
> backport any fixes for any bugs that might or might not be demonstrated, so
> we're very unlikely to attempt to debug this.  Not to put you on a
> treadmill or anything, but investigating how things work against trunk
> SpiderMonkey and filing a bug is the best way to figure out what's up.
>  (This may involve a bit more porting work, of course; we've started adding
> handles to the JSAPI, so many functions' signatures have begun to change in
> surface-incompatible ways.)  Even still, there's no guarantee of
> investigation: no free lunch, as they say, just a better chance of it,
> depending how closely it overlaps with existing JS team priorities.
>
> Jeff
> _______________________________________________
> dev-tech-js-engine mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-tech-js-engine
>



--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: Large memory requirement for spidermonkey 1.8.5

Manoj-11
In reply to this post by Manoj-11
On Wednesday, July 24, 2013 4:33:34 PM UTC-4, Manoj wrote:
> Hi,
>
> We are moving from js-1.7 to js-1.8.5. Our scripts typically process millions of objects. Observation is that js 1.8.5 takes huge runtime memory as compared to js-1.7. Often it crashes. We had to increase runtime memory 4 times so that scripts run smoothly. Is this normal? Is there any way to bring down memory requirement?
>
>
>
>
>
> Manoj

Thanks all for your reply's. I think we have got to the root of the problem. Since the data type of memory parameter in uint32, theoretically we cannot set more than 4GB memory to JS_Runtime. The script runs on a server, not in a browser. The memory requirement is more than 4 GB. In versions 1.7 and before the spidermonkey used to go beyond what was set. Looks like this issue is fixed in 1.8.5, but this fix is causing the issue. We need to do small modification in spidermonkey code so that we can set greater than 4GB memory to JS_Runtime.
Also 1.8.5 takes lesser memory than 1.7.

Thanks
Manoj
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: Large memory requirement for spidermonkey 1.8.5

Jeff Walden-2
In reply to this post by Wes Garland
On 07/30/2013 08:02 PM, Wes Garland wrote:
> OP - Does the JS code run in the browser by any chance?

That's a good point, I should have made it.  If your JS is reasonably independent of embedder-provided objects and classes and such, such that it can be converted to a plain-JS testcase, that helps us investigate it much more easily as well.  That you get all the C++ API goodness with SpiderMonkey makes it more flexible, but it also makes it harder for us to reproduce issues, or to accurately guess what might be at fault without a bunch of investigation.  :-\  Such is life with C++ and cutting-edge craziness like runtime code generation, I guess.

Jeff
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine