Transcript

Two Faces of Googlebot

The two faces of Dr. Crawlit A bot that listens to the 'little guy Googlebot Mr. Hack Scanner, spammer & DDoSing imposter On average a website will be visited by 187 Googlebots a day. For every 24 Googlebot visits a website will also be visited by a Fake Googlebot. Googlebot listens to the 'little guy'. Tests show only a 1.4% correlation between the number of Google crawls and human visits. Over 23% of Fake Googlebots are used for DDOS 10.8% are malicious Scrapers, Spammers and Scanners. 65.7% 23.5% 5.3% 3.8% 1.7% Spamming Vulnerability Scanning DDOS Scraping Other Google's average Fake Googlebot is the 3rd most common bot crawl rate is 4 pages per visit. used for DD0S attacks. Mr. Congenialit Google Where Googlebot comes from: Where Fake Googlebot comes from: accounts for 60% of all search engine page crawls, more than all others combined. UK 1.17% France 0.38% China 15.61% Turkey 14.71% > bing 24.5% Majestic ***** 12 3% USA 98.12% Belgium 0.16% Denmark 0.09% China 0.07% USA 25.16% Brazil 13.49% Indie 8.4% Others 22.63% Bai du 4.4% Yandex 2.3% Study based on 400+ million search engine visits to 10,000 sites, resulting in 2.19+ billion page crawls, over a 30 day period. Information about Googlebot impostors (a.k.a. Fake Googlebots) comes from inspection of 50+ million Googlebot impostor visits. Brought to you by Incapsula An Imperva Company

Two Faces of Googlebot

shared by Incapsula on Aug 04
204 views
1 shares
0 comments
Over the years, many efforts have been made to better understand Google’s behavior and motives. Today, we want to share with you some of our insights into Googlebot’s behavior, based on what we th...

Publisher

Igal Zeifman

Category

Technology
Did you work on this visual? Claim credit!

Embed Code

For hosted site:

Click the code to copy

For wordpress.com:

Click the code to copy
Customize size