Which is semi-meaningless, since they don't ever explain how they
compute those numbers.
> which means that FF 3.1b3 is only ~1/2 as fast as safari 4.0
No, it means their scoring function, which they never explain, assigned
it half as high a score. What that means in practice depends on what
that function is. If you can get them to tell you that, I'd love to
know the results.
> I tried this test on my mac pro (2,4ghz, 4gb) and I got the following
> Safari 3.2.1 791points
> FF3.1beta3 592points
Which means what?
> Has anyone an explaination why FF3.1 performs so poor in this test
It's pretty easy to write tests on which FF3.1 performs worse than
Safari 4 (e.g. dromaeo). If you want me to explain numbers that are
being computed by some hidden formula, I think you're out of luck. ;)
For what it's worth, I spent a few hours on Friday trying to get this
test suite into a shape where I could run one test at a time (and hence
profile the tests to see where we spend time). I gave up when I
couldn't even get any of the tests but the array tests from their web
server: either they send different tests for the same URI based on
cookies, or their code is cleverly obfuscated beyond my ability to
comprehend. Or both.
Given that they don't publish sub-scores for the parts of the benchmark,
don't provide their benchmark's tests in a way that can be read (much
less run individually), and don't explain how their final score is
computed from the sub-scores, the credibility of these folks is about 0
for me right now. They could just be making all the numbers up using a
random number generator; I have no way to tell.