Google Analytics Reliability Doubts

Posted August 16th, 2006 by Max Chirkov
in
There is an interesting post at SEOmoz.org called How Reliable is Google Analytics? by Michael Martinez. Michael makes a very strong point that Google Analytics may not be as reliable as we think it is. He compares some numbers between Google Analytics and his own web log data for one of his web sites (Xenite.Org):
For July, Google Analytics says the same page received only 700 visitors and 900 views -- but my server logs indicate over 8,000 views.
The discrepancies vary across the selection of pages. For Xenite's home page, we show over 12,000 views in January but Google shows 2200. In July, Google thinks we only received about 800 page views on the main page. Our server logs indicate 3,000. Maybe the robot filtering doesn't work the way I think it should on my server logs -- I have changed statistics programs enough through the years that I occasionally get a little confused about how to interpret the results. But these kinds of discrepancies just don't make sense.
Our server logs indicate we had 97,000 visitors in January and 81,000 visitors in July. So while a greater proportion of our summer traffic comes from non-Google sources, we didn't lose almost 50,000 visitors -- which the Google Analytics data suggests we should have.
I've been with Google Analytics myself since as early as I got a beta testing invitation before it went public. I liked all the bells and whistles and still running it on all of my web sites, but I noticed, I haven't looked at the Analytics stats for at least a couple of months now - instead, I'm using desktop based software that parses my web logs. I never questioned reliability of Google Analytics directly, but I noticed that the data was very different from the one I was getting myself and somewhere at the bottom of my mind I didn't trust the GA numbers. So I keep using the desktop based web log analyzer. That's why this article was really interesting to me and it really raises good question.
August 17th, 2006
Michael Martinez -
Some of the respondents in the comments section raised good points about possible issues with my server log analysis. Nonetheless, I feel I have enough experience at identifying false data to know when something doesn't quite add up. I think there is a technical problem on Google's side, a view that appears to be borne out by numerous other people's experience. I hope they can resolve the issue to everyone's (or nearly everyone's) satisfaction.
August 17th, 2006
Max Chirkov - Phoenix, AZ
Photo of Max Chirkov.
I absolutely agree that there is a problem, I just never really looked any deeper into it, since I never really doubt in Google (funny huh) but I see huge differences in my own statistics. I know that differen log analytics software show a little different numbers, but GA is way too low especially on page views - difference in thousands of view - not even hundreds.
September 26th, 2006
gautam -
HI... .. while i was always apprehensive about using google analytics... I wanted to dig it out and find real some things to support my claims.While searching for it on google i just found out a very good posting by mr martini.... i went through al the post and the comments. I think tag build reports are less reliable compared to log build reports.I think google wants to control each an every aspect of online business,be it search,mail,messenger or a community... To me google is too fast to be the undisputed leader in the online world.But what ever google offers are all beta an all free and that makes it even more popular to masses.... Lets chalk down what google wants to do... lets concentrate on GA.In my study or rather in my analysis... google analystics.. is not consistent with its reports when 3 or 4 months reports are taken in and compared with the log reports. There are lots of technical flaws in calculating data....
January 5th, 2007
Phil Nickel -
Our online e-zine for real estate professionals has used SmarterStats for over a year. In July of 2006 I began to realize that their was something funny about our numbers. I discovered that this "professional" site stat package counts every hit as a visit. (Multiple hits in a 20-minute period are counted as one visit.) However, this GREATLY OVERCOUNTS our true visits. Every spiderbot is counted as a visitor! Also, when we send emails with graphic links or have a graphic on a reciprocating link, those views of graphics on our site are also counted as "visits"! In Nov. 2006 I installed Google Analytics and have found the same problem that everyone else here has mentioned. The counts from google are way (6 to 10 times) less than my most conservative estimates from smarterstats. I was wondering if smarterstats had some really gross overcount that I was not aware of. But it looks like google has an unbelievable undercount.
May 29th, 2007
James Briere -
I have noticed this too, that Google Analytics's numbers are always much less. But another thing that i noticed is that Google Analytics seem to say that I had more clicks coming from the google search engine, when i don't even place in the top pages of Google with the keywords that it say people found my website. Has anyone else noticed this.
February 27th, 2008
Pinehurst REALTOR -
There are many issues with all data we receive no matter if it is free or we pay. Trends and results are trackable and that is what we need to focus on. The data we could get from various places a decade ago seemed more in line with what the counts actually were but no matter what the counts tell us now that still doesn't mean that our results based on the trends are different. GA is a pretty useful tool especially for us liteweights in the field, but if you are in the business where the actual numbers are the gauge from which your pay is derived then something else needs to be used. Trends work well for me, hehe, I am just as bad or good following the trends no matter what the data says. As far hits from google search, see how long they stayed on your site, I get google, yahoo, msn hits all the time from really weird phrases, but when they don't find what they are looking for in 1-4 seconds ~ they are gone. There are just too many unintentional keywords and phrases we all rank well on other then the ones we intended. Just plot your numbers from each data source and the resulting graphs will look similar from just about any reliable source. It also helps to be able to extract the ip addresses so you can remove the spiders for statistical purposes, there are days I think the whole world visited my pages when it turns out it was 2 or 3 bots indexing the entire site. Great Rush while it lasts though.