Google Explains Why Field Data is More Reliable Than Lab Data

Google’s Martin Splitt recently gave a detailed explanation of the difference between lab data and field data as it relates to testing tools.

Field data is more reliable, Splitt says, as it’s an indicator of how real users experience and interact with a website.

This topic was brought up during a Google Webmasters JavaScript SEO hangout held on June 10th.

A site owner submitted a question asking why there’s a difference between field data and lab data when measuring page speed.

Here is what Splitt had to say in response:

“Field data is coming from real users, whereas lab data comes from a quite strong machine with probably good internet from somewhere around the world. So you might not see the same results.”

Field Data vs. Lab Data

Splitt offers a hypothetical example of testing a site with a tool that uses lab data from a country where the site’s server is also based.

In this example, the lab data would likely determine the site loads quite fast. That’s because the site is hosted in the same country where the test is being run, so the physical time it takes to send data is minimal.

Let’s say the country is also relatively well covered with high-speed internet access. That would further increase site speed measured by the lab data.

However, that speed drops considerably when the site’s visitors are based in another country. The speed drops even further if visitors do not have a high-speed internet connection where they live.

That’s the key difference between lab data and field data.

Why Field Data is More Reliable

Lab data is measured with a controlled test under ideal conditions, while field data is measured according to how real-life users interact with a site under a variety of conditions.

As Martin Splitt puts it:

“In that case, my lab data looks amazing. And then my field data, because my users are sitting on the other end of the world, looks terrible.

That’s what happens because the lab data is only synthetic. It’s only an approximation of what happens in the real world. And then you have field data which, surprisingly, gives you the actual thing that people are experiencing.

So if you know that you are serving users on slow, old phones. Or rural, spotty, internet connections. This is what happens. This is when the field data looks a lot worse than what you see in lab data.

That’s a general problem – developers usually work with fantastic internet connections on modern laptops. And then the real world user is on an old iPhone 2, somewhere in rural, northern Germany where you have Edge internet connection if you are lucky. And then everything looks terrible for them…

Field data is probably a better indicator for how real users are experiencing your website than lab data. Because lab data is literally just someone’s server making a request to your thing. And then if that server happens to be quite beefy then you get pretty good looking numbers, but then the real world isn’t as beefy and nice.”

Hear Splitt’s full response in the video below: