Directing crawlers to content in language per language sub-domain

Posted by Noam on Pro Webmasters See other posts from Pro Webmasters or by Noam
Published on 2013-04-21T10:08:06Z Indexed on 2013/06/28 10:30 UTC
Read the original article Hit count: 268

Filed under:

seo

|

google

|

search-engines

|

googlebot

|

multilingual

I have a site with multilingual website with many pages (40M). The site has UGC, and each translation is actually for the titles. Each sub-domain points to the same content with different titles per language. As far as I understand, each sub-domain should be indexed by search engines, meaning they will actually need to crawl 40M x supported-languages. So I thought it might be best to direct each subdomain crawler, to pages that are fully in that language (titles + UGC).

Is there a way to do this? Should search engines understand this on their own?

© Pro Webmasters or respective owner

Related posts about seo

Do i need a full time seo employee? [closed]

as seen on Pro Webmasters - Search for 'Pro Webmasters'
I need seo done for 1 site that's still new. It's a niche listings/ecommerce site. And i need to make a hiring decision for this - whether there's a need for a full time employee dedicated to seo or is part time or a contract sufficient? Here's some info and assumptions : 1)The site's dynamic -… >>> More
1 SEO Article Vs 5 SEO Articles?

as seen on Ezine Articles - Search for 'Ezine Articles'
You've probably clicked on this article because you think it's a interesting article but the answer seems very easy or straight forward. If you think 5 SEO articles wins or is more beneficial to your business, then your like me. >>> More
SEO Training OR SEO Outsourcing

as seen on Ezine Articles - Search for 'Ezine Articles'
There is a lot of focus on outsourcing search optimization work, but this may not always be the best option. This article looks at why SEO Training can often be a better option because it results in more unique content which is better for search engines. >>> More
Get Professional SEO Service From SEO Experts

as seen on Ezine Articles - Search for 'Ezine Articles'
SEO service is imperative to online business. Business today has gone digital. With its power to perform international promotion, almost every brand is on a bid to establish an online presence. However, it's not easy to gain online presence without proper SEO service. >>> More
The Chinese SEO in the Internet SEO Formula

as seen on Ezine Articles - Search for 'Ezine Articles'
Most ecommerce websites are designed in order to drive the Chinese internet users to the website thus delivering a good user experience that converts the users into customers. According to a statistical report on the internet development in China, conducted during the year 2009, the Chinese SEO of… >>> More

Related posts about google

Removing malware of a particular kind

as seen on Super User - Search for 'Super User'
I need to remove some malware from my computer. It is a trojan, and very annoying. It blocks access to Google and search sites. The trojan, with its name spelled out on each line cause it seems to block sites when i reference it in a url, is a r t (some text to mess it up) e m (more text i s First… >>> More
Trouble installing Matlab

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
So whenever I try to install Matlab, the first image appears, but then goes away and that's the end of it. Here's what I'm doing- user@host$> cd ~/mathworks_downloads user@host$> unzip matlab_R2012a_student_glnx86_installer.zip user@host$> ./install The Matlab logo and image pop up here… >>> More
Google chrome is always searching in local google domain instead of Google.com

as seen on Super User - Search for 'Super User'
I have changed in the searched preferences to google.com but still when I do search from the address bar (instant or non-instant) it will go to google.co.kr. Even though I change "Google.com in English", still same... The only way is to open google.com website first, then do search in it. So the… >>> More
Google I/O 2010: Google TV Keynote - Introducing Google TV

as seen on Google Code - Search for 'Google Code'
Google I/O 2010: Google TV Keynote - Introducing Google TV Due to licensing and permissions issues, we are unable to show the full Google TV demonstration from the Day 2 keynote at Google I/O. Until we are able to get these permissions, please check out these clips. For Google I/O session videos… >>> More
Google I/O 2010: Google TV Keynote - Android Apps On Google TV

as seen on Google Code - Search for 'Google Code'
Google I/O 2010: Google TV Keynote - Android Apps On Google TV Due to licensing and permissions issues, we are unable to show the full Google TV demonstration from the Day 2 keynote at Google I/O. Until we are able to get these permissions, please check out these clips. For Google I/O session videos… >>> More