Cross-domain jQuery using YQL gives robots.txt error
Posted
by Jens Roland
on Stack Overflow
See other posts from Stack Overflow
or by Jens Roland
Published on 2010-03-23T14:10:55Z
Indexed on
2010/03/23
14:13 UTC
Read the original article
Hit count: 679
On the page http://qxlapps.dk/test.htm I am trying to perform an Ajax load from another domain, qxlapp.dk. I am using James Padolsey's xdomainajax.js plugin from:
http://james.padolsey.com/javascript/cross-domain-requests-with-jquery/
When I open my test page, I get no output, but FireBug shows the JSON result, including the error message: "forbidden":"robots.txt for the domain disallows crawling for url: http://qxlapp.dk/projects/dagens_kup/show.php".
The robots.txt on the qxlapp.dk domain contains the following:
User-agent: Yahoo Pipes 2.0
Allow: /
User-agent: *
Allow: /
So I don't see what the problem is? Shouldn't it pull the page just fine with those settings?
© Stack Overflow or respective owner