Apple details its Web crawler, possibly indicating a future push into search

Apple has quietly added a new page to its support site that details its first Web crawler. A crawler is a bot, used by search engines such as Google and Bing, that automatically scans the Web for pages and information to add to an index and provides information when pages change. Apple’s crawler, which existed previously but was never detailed by the company, identifies itself as “Applebot” and could indicate a future push into full Web search. Right now, Apple says its crawler is used to add data such as movie information to Siri and Spotlight. You can exclude Applebot from…

This story continues at The Next Web

from The Next Web http://ift.tt/1IfO9nX
via IFTTT