Search engine bots accessing strange URLs

Posted by casasoft on Pro Webmasters See other posts from Pro Webmasters or by casasoft
Published on 2013-10-29T11:20:39Z Indexed on 2013/10/29 16:13 UTC
Read the original article Hit count: 240

Filed under:

search-engines

|

indexing

|

robots

We have ELMAH enabled on our site and get errors whenever a Page Not Found error is triggered on the website.

We have recently redesigned a new website and so we understand that search engine robots might have previously indexed pages which they try to access and result in a Page Not Found errors. For this reason, we have set up permanent redirects for such previously indexed pages to the respective new pages.

The website in mention is www.chambercollege.com and for example, a previously indexed URL was www.chambercollege.com/special-offers.aspx. This page is no longer accessible so we have created the necessary permanent redirect to redirect to the respective page on www.chambercollege.com/en/content/special-offers-161/.

Now we are starting to receive Page Not Found errors of search engine bots (e.g. MSN bot) trying to access the URL www.chambercollege.com/special-offers.aspx/images/shadow_right.jpg/.

Any idea how could a search engine make up that strange URL and whether you have any suggestions of what to do best?

© Pro Webmasters or respective owner

Related posts about search-engines

Your Content Can Get Visibility Through Search Engines - Part 2

as seen on Ezine Articles - Search for 'Ezine Articles'
Your content plays an important role in defining the success of your search engine marketing plan and when you talk about SEO in India, and then you will easily understand different kinds of discoveries and how to use them for your benefit. After you have understood the traditional discovery and some… >>> More
Your Content Can Get Visibility Through Search Engines - Part 1

as seen on Ezine Articles - Search for 'Ezine Articles'
Search engine marketing aims at getting higher search engine rankings for your website but for that you need to get yourself into the search engine Index first. Your content designed for your SEO activity should be accessible to millions of internet users and if your content is not found through the… >>> More
Most Popular Internet Search Engines - Four Tips to Raise Your Website's Search Engine Ranking

as seen on Ezine Articles - Search for 'Ezine Articles'
When you've got a website, one of the most important factors you need to consider is search engine rankings. If you don't have a great ranking position, you won't be easy to find. Many people will only look through the first five pages of search engine results. For that reason, your website must be… >>> More
Tip/Trick: Fix Common SEO Problems Using the URL Rewrite Extension

as seen on ASP.net Weblogs - Search for 'ASP.net Weblogs'
Search engine optimization (SEO) is important for any publically facing web-site. A large % of traffic to sites now comes directly from search engines, and improving your site’s search relevancy will lead to more users visiting your site from search engine queries. This can directly or… >>> More
How Does WordPress Blog Search Engines?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello, If you go to wordpress admin-settings-privacy, there are two options asking you whether you want to allow your blog to be searched though by seach engines and this option: I would like to block search engines, but allow normal visitors How does wordpress actually block search bots/crawlers… >>> More

Related posts about indexing

IIS6 Indexing Service indexing asp.net codebehind (.aspx.cs) files

as seen on Server Fault - Search for 'Server Fault'
I've setup a few catalogs on an Windows Server 2003 IIS6 install, each tracking files within a website. In the Properties - Generation Dialog for each catalog, 'Index files with unknown extensions' is turned OFF. 'Inherit above settings from Service' in that dialog is also turned off. However, the… >>> More
Outlook 2007 OST File Indexing and OneNote 2007 Indexing are Broken

as seen on Super User - Search for 'Super User'
I'm running Outlook 2007 under Windows 7 Home Premium RTM. My OST file was previously being properly indexed but eventually searches significantly slowed down so I suspected a problem. Searching and indexing appears broken in OneNote 2007 as well as search time is now significantly longer. I brought… >>> More
Indexing vs. no indexing when inserting records

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a few questions about whether or not it would be best to not use indexing. BACKGROUND: My records have a timestamp attribute, and the records will be inserted in order of their timestamps (i.e., inserted chronologically). QUESTIONS: If I DON'T use indexing is it typical for the database… >>> More
SQL SERVER – Transcript of Learning SQL Server Performance: Indexing Basics – Interview of Vinod Kumar by Pinal Dave

as seen on SQL Authority - Search for 'SQL Authority'
Recently I just wrote a blog post on about Learning SQL Server Performance: Indexing Basics and I received lots of request that if we can share some insight into the course. Here is 200 seconds interview of Vinod Kumar I took right after completing the course. We have few free codes to watch the course… >>> More
Can the Windows Indexing Service restart an app pool if it doesn't index the Web.Config?

as seen on Stack Overflow - Search for 'Stack Overflow'
I am having a slight debate with a colleague of mine on this subject. Scenario: A web site has a sub directory that is indexed using the Windows Indexing Service. There is not a web.config in this directory and the indexing service is not indexing the parent directory which is the site root. Something… >>> More