After a long speculation, apply finally accepted that they have Apple Web Crawler and the same crawler is used for Siri and Spotlight as well.
The user agent will follow the string which contains “AppleBot”. They also throw some light on robots.txt rules say meta tags. As per AppleBot if you want to block AppleBot you have to block AppleBot and GoogleBot individually. If some time you face any unusual activity, approach Apple-NOC “at” apple.com directly to resolve the issue.
Seems Apple is trying to complete Google on search but not sure. As Apple is trying to expand the internal search and creating the version for web search via spotlight so it can be assumed.
Find the help document here.
