If you look at the Perfherder build metrics, there are some things you need to know about.
As of 950d65906200 (bug 1315041) landing on autoland a few minutes ago, metrics will now be segmented by execution platform instead of being lumped into a single bucket. Before, TaskCluster and Buildbot stored their data under the same "keys." Within each platform, builds could execute on different EC2 instance types. It was common for data to be bimodal and multimodal as a result. In the new world, metrics are segmented according to the execution environment. e.g. "taskcluster-c4.4xlarge" or "buildbot-r3.2xlarge." Data should be much easier to "read" and automatic regression detection should work better.
The main downside of this change is the metrics have all been effectively renamed, so there will be a clean split in the data :/ It /might/ be possible for a Treeherder admin to run some SQL to copy things.
Also, TaskCluster will soon be switching their Windows builders from c3.2xlarge to c4.4xlarge. This should make Windows builds significantly faster. Expect that to happen sometime next week. Send kudos to Rob for his great work on improving Windows in automation.