Some time back I created a data base driven ASP.NET Resource Provider along with some tools that make it easy to edit ASP.NET resources interactively in a Web application. One of
the small helper features of
the interactive resource admin tool is
the ability to do simple translations using both Google Translate and Babelfish. Here's what this looks like in
the resource administration form: When a resource is displayed,
the user can click a Translate button and it will show
the current resource text and then lets you set
the source and target languages to translate.
The Go button fires
the translation for both Google and Babelfish and displays them - pressing use then changes
the language of
the resource to
the target language and sets
the resource value to
the newly translated value. It's a nice and quick way to get a quick translation going. Ch… Ch… Changes Originally, both implementations basically did some screen scraping of
the interactive Web sites and retrieved translated text out of result HTML. Screen scraping is always kind of an iffy proposition as content can be changed easily, but surprisingly that code worked for many years without fail. Recently however, Google at least changed their input pages to use AJAX callbacks and
the page updates no longer worked
the same way. End result:
The Google translate code was broken. Now, Google does have an official API that you can access, but
the API is being deprecated and you actually need to have an API key. Since I have public samples that people can download
the API key is an issue if I want people to have
the samples work out of
the box -
the only way I could even do this is by sharing my API key (not allowed). However, after a bit of spelunking and playing around with
the public site however I found that Google's interactive translate page actually makes callbacks using plain public access without an API key. By intercepting some of those AJAX calls and calling them directly from code I was able to get translation back up and working with minimal fuss, by parsing out
the JSON these AJAX calls return. I don't think this particular Warning: This is hacky code, but after a fair bit of testing I found this to work very well with all sorts of languages and accented and escaped text etc. as long as you stick to small blocks of translated text. I thought I'd share it in case anybody else had been relying on a screen scraping mechanism like I did and needed a non-API based replacement. Here's
the code: /// <summary>
/// Translates a string into another language using Google's translate API JSON calls.
/// <seealso>Class TranslationServices</seealso>
/// </summary>
/// <param name="Text">Text to translate. Should be a single word or sentence.</param>
/// <param name="FromCulture">
/// Two letter culture (en of en-us, fr of fr-ca, de of de-ch)
/// </param>
/// <param name="ToCulture">
/// Two letter culture (as for FromCulture)
/// </param>
public string TranslateGoogle(string text, string fromCulture, string toCulture)
{
fromCulture = fromCulture.ToLower();
toCulture = toCulture.ToLower();
// normalize
the culture in case something like en-us was passed
// retrieve only en since Google doesn't support sub-locales
string[] tokens = fromCulture.Split('-');
if (tokens.Length > 1)
fromCulture = tokens[0];
// normalize ToCulture
tokens = toCulture.Split('-');
if (tokens.Length > 1)
toCulture = tokens[0];
string url = string.Format(@"http://translate.google.com/translate_a/t?client=j&text={0}&hl=en&sl={1}&tl={2}",
HttpUtility.UrlEncode(text),fromCulture,toCulture);
// Retrieve Translation with HTTP GET call
string html = null;
try
{
WebClient web = new WebClient();
// MUST add a known browser user agent or else response encoding doen't return UTF-8 (WTF Google?)
web.Headers.Add(HttpRequestHeader.UserAgent, "Mozilla/5.0");
web.Headers.Add(HttpRequestHeader.AcceptCharset, "UTF-8");
// Make sure we have response encoding to UTF-8
web.Encoding = Encoding.UTF8;
html = web.DownloadString(url);
}
catch (Exception ex)
{
this.ErrorMessage = Westwind.Globalization.Resources.Resources.ConnectionFailed + ": " +
ex.GetBaseException().Message;
return null;
}
// Extract out trans":"...[Extracted]...","from
the JSON string
string result = Regex.Match(html, "trans\":(\".*?\"),\"", RegexOptions.IgnoreCase).Groups[1].Value;
if (string.IsNullOrEmpty(result))
{
this.ErrorMessage = Westwind.Globalization.Resources.Resources.InvalidSearchResult;
return null;
}
//return WebUtils.DecodeJsString(result);
// Result is a JavaScript string so we need to deserialize it properly
JavaScriptSerializer ser = new JavaScriptSerializer();
return ser.Deserialize(result, typeof(string)) as string;
}
To use
the code is straightforward enough - simply provide a string to translate and a pair of two letter source and target languages:
string result = service.TranslateGoogle("Life is
great and one is spoiled when it goes on and on and on", "en", "de");
TestContext.WriteLine(result);
How it works
The code to translate is fairly straightforward. It basically uses
the URL I snagged from
the Google Translate Web Page slightly changed to return a JSON result (&client=j) instead of
the funky nested PHP style JSON array that
the default returns.
The JSON result returned looks like this:
{"sentences":[{"trans":"Das Leben ist großartig und man wird verwöhnt, wenn es weiter und weiter und weiter geht","orig":"Life is
great and one is spoiled when it goes on and on and on","translit":"","src_translit":""}],"src":"en","server_time":24}
I use WebClient to make an HTTP GET call to retrieve
the JSON data and strip out part of
the full JSON response that contains
the actual translated text. Since this is a JSON response I need to deserialize
the JSON string in case it's encoded (for upper/lower ASCII chars or quotes etc.).
Couple of odd things to note in this code:
First note that a valid user agent string must be passed (or at least one starting with a common browser identification - I use Mozilla/5.0). Without this Google doesn't encode
the result with UTF-8, but instead uses a ISO encoding that .NET can't easily decode. Google seems to ignore
the character set header and use
the user agent instead which is - odd to say
the least.
The other is that
the code returns a full JSON response. Rather than use
the full response and decode it into a custom type that matches Google's result object, I just strip out
the translated text. Yeah I know that's hacky but avoids an extra type and firing up
the JavaScript deserializer. My internal version uses a small DecodeJsString() method to decode Javascript without
the overhead of a full JSON parser.
It's obviously not rocket science but as mentioned above what's nice about it is that it works without an Google API key. I can't vouch on how many translates you can do before there are cut offs but in my limited testing running a few stress tests on a Web server under load I didn't run into any problems.
Limitations
There are some restrictions with this: It only works on single words or single sentences - multiple sentences (delimited by .) are cut off at
the
".". There is also a length limitation which appears to happen at around 220 characters or so. While that may not sound like much for typical word or phrase translations this this is plenty of length.
Use with a grain of salt - Google seems to be trying to limit their exposure to usage of
the Translate APIs so this code might break in
the future, but for now at least it works.
FWIW, I also found that Google's translation is not as good as Babelfish, especially for contextual content like sentences. Google is faster, but Babelfish tends to give better translations. This is why in my translation tool I show both Google and Babelfish values retrieved. You can check out
the code for this in
the West Wind West Wind Web Toolkit's TranslationService.cs file which contains both
the Google and Babelfish translation code pieces. Ironically
the Babelfish code has been working forever using screen scraping and continues to work just fine today. I think it's a good idea to have multiple translation providers in case one is down or changes its format, hence
the dual display in my translation form above.
I hope this has been helpful to some of you - I've actually had many small uses for this code in a number of applications and it's sweet to have a simple routine that performs these operations for me easily.
Resources
Live Localization Sample
Localization Resource Provider Administration form that includes options to translate text using Google and Babelfish interactively.
TranslationService.cs
The full source code in
the West Wind West Wind Web Toolkit's Globalization library that contains
the translation code.
© Rick Strahl, West Wind Technologies, 2005-2011Posted in CSharp HTTP
Tweet
(function() {
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true;
po.src = 'https://apis.google.com/js/plusone.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
