Traffic measurement #3: the NYT on site publishers vs. ComScore and Nielsen/NetRatings

One way to sell a Web ad is to charge based on the number of times it's displayed. That's perhaps the simplest approach (though we'll cover several of the difficulties at some point). Another method is to charge by the number of different people who see the ad, taking into account that one person may see it several times. That approach is tricky, e.g. if someone deletes cookies between visits, views the site from both work and home, or is behind a connection that masks individual visitors (perhaps a large ISP; perhaps a company network). But advertisers want data, even if different sources yield quite different results.

The New York Times provided some examples today:

Condé Nast is up first:

How many people visited Style.com, the online home of Vogue and W magazines, last month?
...
Condé Nast’s internal count (1.8 million) was much higher than the tally by ComScore (421,000) or Nielsen/NetRatings (497,000), whose numbers are used to help set advertising rates

(A minor gripe: unlike in the data below, the NYT didn't specify whether the company estimate was US or worldwide.)

From Jim Spanfeller, president and chief executive of Forbes.com (emphasis added):

Forbes.com had 11.6 million United States visitors last month, far more than the 7.5 million estimated by Nielsen/NetRatings and 5.8 million from ComScore.

ComScore, Nielsen/NetRatings and (not mentioned) Hitwise use panels of "representative audience members" -- a holdover from TV where there was no hard data on who watched a specific program or even channel. One key limitation of panels: the office.

online publishers say that their systems drastically undercount people who use the Web during work hours, particularly in offices where corporate software makes the wanderings invisible to the tracking systems

Panels also have trouble covering niches, whether "the wealthier people whom Condé Nast says frequent many of its sites" or "students on college campuses" (see our why comScore got Facebook wrong) or "Hispanics and other demographic groups".

One thing that has changed compared to TV, site publishers are no longer at the mercy of these panels:

But the Internet has given publishers a new form of ammunition: raw server data with precise numbers of site visits and page views. This data does not correlate directly to the number of visitors, but it does give them ballpark figures that they say are far more accurate than the extrapolations drawn by ratings companies based on panel samplings.

(For a large site, the "ballpark figures" are generally based on data from an in-house Web analytics team. I suspect they have a pretty strong incentive to count as accurately as they can in the face of several technical challenges.)

Consistent with the journalistic formula, quotes from the tracking companies are up next:

“It’s in their interest to make their audience look as big as possible,” said Gian M. Fulgoni, chairman of ComScore.

Manish Bhatia, executive vice president of global operations and U.S. Sales for Nielsen/NetRatings, agreed.

“If the panel numbers were higher, I don’t think you’d be having this debate,” he said. “People would love us.”

Later in the article:

The ratings company say they have improved their panels, and point fingers back at the Web publishers, accusing them of mixing international and domestic traffic and of double-counting people who visit a site from home and from the office.

To make matters more complicated, consumers who delete cookies — small bits of computer code that track their online wanderings — are also overcounted by publishers’ servers, by most accounts.

(Another article gripe: the more substantive responses were left to the end. Apparently the NYT thinks the ad hominem attacks are more important. And, were the ratings companies really using language such as "point fingers" and "accusing", or is that just the reporter's spin?)

My take: it's not easy to track unique visitors. Domestic vs. international is probably not much of an issue; I suspect that the IP addresses in log files provide a reasonable view there. (And, note the large difference even though Forbes quoted their US figures.) Home vs. work is a big issue, though affects both sides. Cookies is clearly a problem, though very difficult to quantify. Panels may play a role when used to filter log files, but I doubt they can ever yield better results on their own. But the bottom line: what do the advertisers think?

(Final MSM gripe: someone should tell the NYT headline writers that nobody counts "hits" anymore, and in any case the article was about "unique visitors". So much for the paper of record.)

Hat tip: Techmeme.

Glossary: MSM = MainStream Media