Can I use a Google Appliance/Mini to crawl and index sites I don't own?
Posted
by
SkippyFire
on Server Fault
See other posts from Server Fault
or by SkippyFire
Published on 2010-07-23T18:55:31Z
Indexed on
2011/01/05
22:55 UTC
Read the original article
Hit count: 285
Maybe this is a stupid question, but...
I am working with this company and they said they needed to get "permission" to crawl other people's sites. They have a Google Search Appliance And some Google Minis and want to point them at other sites to aggregate content. The end result will be something like a targeted search engine. (All the indexed sites relate to a specific topic)
The only thing they will be doing is:
- Indexing Content from the other sites/domains
- Providing search functionality on their own site that searches the indexed content (like Google, displaying summaries and not the full content)
- The search results will provide links back to the original content
Their intent is not malicious in nature, and is to provide a single site/resource for people to reference on their given topic.
Is there anything illegal or fishy about this process?
© Server Fault or respective owner