apache robots.txt with SSL

Posted by user224013 on Server Fault See other posts from Server Fault or by user224013
Published on 2014-06-12T14:35:27Z Indexed on 2014/06/12 15:27 UTC
Read the original article Hit count: 190

Filed under:

apache-2.2

|

.htaccess

|

robots.txt

I have an .htaccess file with a rewrite rule to get a redirect of every HTTP request to HTTPS.

But now I have a problem that my robots.txt is not recognized by some online checker. If I remove the redirect from the .htaccess file the robots.txt is recognized correctly.

Maybe I should exclude that the robots.txt is redirects to an HTTPS connection?

This is the part of .htaccess for redirecting to HTTPS

RewriteCond %{SERVER_PORT} !^443$            
RewriteRule (.*) https://%{HTTP_HOST}/$1 [L]

© Server Fault or respective owner

Related posts about apache-2.2

Web site not responding

as seen on Super User - Search for 'Super User'
I have website working fine before. But now its not able to connect to the server(I believe that is the problem). But its strange that the message not able to connect to the server is not coming and its keep connecting... for infinite time. Here is the screenshot. Here are some of the useful… >>> More
SVN Error 403 Forbidden

as seen on Stack Overflow - Search for 'Stack Overflow'
I can't figure this out. I try to import a new project into a svn repository from Netbeans and get 403 Forbidden. I just setup svn on my serverbox today. I can get to it through a browser just fine, though its empty as I haven't imported my project yet. Apache's path for html files is /var/www I… >>> More
How can I setup dependencies for Axis2 / Axiom on Maven2

as seen on Stack Overflow - Search for 'Stack Overflow'
I've tried the following settings on pom.xml to use Axis2 wsdl2code: <dependencies> <dependency> <groupId>org.apache.axis2</groupId> <artifactId>axis2</artifactId> <version>1.5.1</version> </dependency> </dependencies> ... <build> … >>> More
Problem compiling hive with ant

as seen on Stack Overflow - Search for 'Stack Overflow'
I compiling with Solaris 10 SPARC, jdk 1.6 from Sun, Ant 1.7.1 from OpenCSW. I have no problem running hadoop 0.17.2.1 However, I have problem compiling/integrating hive with the error 'cannot find symbol', although I followed the tutorial. I have the hive source code from SVN exactly from tutorial… >>> More
War deployment error related to classloading

as seen on Stack Overflow - Search for 'Stack Overflow'
hello when i am deploying my war file and run it it will give error like org.springframework.instrument.classloading.tomcat.TomcatInstrumentableClassLoader Jan 6, 2011 3:16:04 PM org.apache.catalina.startup.HostConfig deployDescriptor INFO: Deploying configuration descriptor servlet.xml Jan… >>> More

Related posts about .htaccess

Seeking htaccess help: Converting multiple subdomains (both http and https) to www.domain.com using .htaccess

as seen on Pro Webmasters - Search for 'Pro Webmasters'
I've been trying to get an answer to this question on other forums (the folks at SuperUser thought this was the place I needed to post) and via my connections, but I haven't gotten very far. Hopefully you guys can help me find an answer: I've got a dozen old subdomains that have been indexed by Google… >>> More
Disable .htaccess or disable some rules from .htaccess on specific URL

as seen on Server Fault - Search for 'Server Fault'
I have Kerberos-based authentication and I want to disable it on only root url: mysite.com/. And I want it to works fine on any other page like mysite.com/page1. I have such things in my .htaccess: AuthType Kerberos AuthName "Domain login" KrbAuthRealms DOMAIN.COM KrbMethodK5Passwd on Krb5KeyTab… >>> More
.htaccess Permission denied. Unable to check htaccess file

as seen on Server Fault - Search for 'Server Fault'
Hi, I have a strange problem when adding a sub-domain to our virtual server. I have done similar sub-domains before and they have worked fine. When I try to access the sub-domain I get an 403 Forbidden error. I checked the error logs and have the following error: pcfg_openfile: unable to check… >>> More
.htaccess Permission denied. Unable to check htaccess file

as seen on Server Fault - Search for 'Server Fault'
I have a strange problem when adding a sub-domain to our virtual server. I have done similar sub-domains before and they have worked fine. When I try to access the sub-domain I get an 403 Forbidden error. I checked the error logs and have the following error: pcfg_openfile: unable to check htaccess… >>> More
Permission denied: /home/.htaccess pcfg_openfile: unable to check htaccess file

as seen on Server Fault - Search for 'Server Fault'
This domain was working this morning, now I get a 403 error and the message above in my error log. I'm not using .htaccess files but I have been doing some copy on the server so may have messed things up but no changes to this domain (unless by accident!). What is this pcfg_openfile thing anyway? Done… >>> More