Some time back I created a data base driven ASP.NET Resource Provider along with some tools that make it easy
to edit ASP.NET resources interactively in a Web application. One of
the small helper features of
the interactive resource admin tool is
the ability
to do simple translations using both Google Translate and Babelfish. Here's what this looks like in
the resource administration form: When a resource is displayed,
the user can click a Translate button and it will show
the current resource text and then lets you set
the source and target languages
to translate.
The Go button fires
the translation for both Google and Babelfish and displays them - pressing use then changes
the language of
the resource
to the target language and sets
the resource value
to the newly translated value. It's a nice and quick way
to get a quick translation going. Ch… Ch… Changes Originally, both implementations basically did some
screen scraping of
the interactive Web sites and retrieved translated text out of result HTML.
Screen scraping is always kind of an iffy proposition as content can be changed easily, but surprisingly that code worked for many years without fail. Recently however, Google at least changed their input pages
to use AJAX callbacks and
the page updates no longer worked
the same way. End result:
The Google translate code was broken. Now, Google does have an official API that you can access, but
the API is being deprecated and you actually need
to have an API key. Since I have public samples that people can download
the API key is an issue if I want people
to have
the samples work out of
the box -
the only way I could even do this is by sharing my API key (not allowed). However, after a bit of spelunking and playing around with
the public site however I found that Google's interactive translate page actually makes callbacks using plain public access without an API key. By intercepting some of those AJAX calls and calling them directly from code I was able
to get translation back up and working with minimal fuss, by parsing out
the JSON these AJAX calls return. I don't think this particular Warning: This is hacky code, but after a fair bit of testing I found this
to work very well with all sorts of languages and accented and escaped text etc. as long as you stick
to small blocks of translated text. I thought I'd share it in case anybody else had been relying on a
screen scraping mechanism like I did and needed a non-API based replacement. Here's
the code: /// <summary>
/// Translates a string into another language using Google's translate API JSON calls.
/// <seealso>Class TranslationServices</seealso>
/// </summary>
/// <param name="Text">Text
to translate. Should be a single word or sentence.</param>
/// <param name="FromCulture">
/// Two letter culture (en of en-us, fr of fr-ca, de of de-ch)
/// </param>
/// <param name="ToCulture">
/// Two letter culture (as for FromCulture)
/// </param>
public string TranslateGoogle(string text, string fromCulture, string toCulture)
{
fromCulture = fromCulture.ToLower();
toCulture = toCulture.ToLower();
// normalize
the culture in case something like en-us was passed
// retrieve only en since Google doesn't support sub-locales
string[] tokens = fromCulture.Split('-');
if (tokens.Length > 1)
fromCulture = tokens[0];
// normalize ToCulture
tokens = toCulture.Split('-');
if (tokens.Length > 1)
toCulture = tokens[0];
string url = string.Format(@"http://translate.google.com/translate_a/t?client=j&text={0}&hl=en&sl={1}&tl={2}",
HttpUtility.UrlEncode(text),fromCulture,toCulture);
// Retrieve Translation with HTTP GET call
string html = null;
try
{
WebClient web = new WebClient();
// MUST add a known browser user agent or else response encoding doen't return UTF-8 (WTF Google?)
web.Headers.Add(HttpRequestHeader.UserAgent, "Mozilla/5.0");
web.Headers.Add(HttpRequestHeader.AcceptCharset, "UTF-8");
// Make sure we have response encoding
to UTF-8
web.Encoding = Encoding.UTF8;
html = web.DownloadString(url);
}
catch (Exception ex)
{
this.ErrorMessage = Westwind.Globalization.Resources.Resources.ConnectionFailed + ": " +
ex.GetBaseException().Message;
return null;
}
// Extract out trans":"...[Extracted]...","from
the JSON string
string result = Regex.Match(html, "trans\":(\".*?\"),\"", RegexOptions.IgnoreCase).Groups[1].Value;
if (string.IsNullOrEmpty(result))
{
this.ErrorMessage = Westwind.Globalization.Resources.Resources.InvalidSearchResult;
return null;
}
//return WebUtils.DecodeJsString(result);
// Result is a JavaScript string so we need
to deserialize it properly
JavaScriptSerializer ser = new JavaScriptSerializer();
return ser.Deserialize(result, typeof(string)) as string;
}
To use
the code is straightforward enough - simply provide a string
to translate and a pair of two letter source and target languages:
string result = service.TranslateGoogle("Life is great and one is spoiled when it goes on and on and on", "en", "de");
TestContext.WriteLine(result);
How it works
The code
to translate is fairly straightforward. It basically uses
the URL I snagged from
the Google Translate Web Page slightly changed
to return a JSON result (&client=j) instead of
the funky nested PHP style JSON array that
the default returns.
The JSON result returned looks like this:
{"sentences":[{"trans":"Das Leben ist großartig und man wird verwöhnt, wenn es weiter und weiter und weiter geht","orig":"Life is great and one is spoiled when it goes on and on and on","translit":"","src_translit":""}],"src":"en","server_time":24}
I use WebClient
to make an HTTP GET call
to retrieve
the JSON data and strip out part of
the full JSON response that contains
the actual translated text. Since this is a JSON response I need
to deserialize
the JSON string in case it's encoded (for upper/lower ASCII chars or quotes etc.).
Couple of odd things
to note in this code:
First note that a valid user agent string must be passed (or at least one starting with a common browser identification - I use Mozilla/5.0). Without this Google doesn't encode
the result with UTF-8, but instead uses a ISO encoding that .NET can't easily decode. Google seems
to ignore
the character set header and use
the user agent instead which is - odd
to say
the least.
The other is that
the code returns a full JSON response. Rather than use
the full response and decode it into a custom type that matches Google's result object, I just strip out
the translated text. Yeah I know that's hacky but avoids an extra type and firing up
the JavaScript deserializer. My internal version uses a small DecodeJsString() method
to decode Javascript without
the overhead of a full JSON parser.
It's obviously not rocket science but as mentioned above what's nice about it is that it works without an Google API key. I can't vouch on
how many translates you can do before there are cut offs but in my limited testing running a few stress tests on a Web server under load I didn't run into any problems.
Limitations
There are some restrictions with this: It only works on single words or single sentences - multiple sentences (delimited by .) are cut off at
the
".". There is also a length limitation which appears
to happen at around 220 characters or so. While that may not sound like much for typical word or phrase translations this this is plenty of length.
Use with a grain of salt - Google seems
to be trying
to limit their exposure
to usage of
the Translate APIs so this code might break in
the future, but for now at least it works.
FWIW, I also found that Google's translation is not as good as Babelfish, especially for contextual content like sentences. Google is faster, but Babelfish tends
to give better translations. This is why in my translation tool I show both Google and Babelfish values retrieved. You can check out
the code for this in
the West Wind West Wind Web Toolkit's TranslationService.cs file which contains both
the Google and Babelfish translation code pieces. Ironically
the Babelfish code has been working forever using
screen scraping and continues
to work just fine today. I think it's a good idea
to have multiple translation providers in case one is down or changes its format, hence
the dual display in my translation form above.
I hope this has been helpful
to some of you - I've actually had many small uses for this code in a number of applications and it's sweet
to have a simple routine that performs these operations for me easily.
Resources
Live Localization Sample
Localization Resource Provider Administration form that includes options
to translate text using Google and Babelfish interactively.
TranslationService.cs
The full source code in
the West Wind West Wind Web Toolkit's Globalization library that contains
the translation code.
© Rick Strahl, West Wind Technologies, 2005-2011Posted in CSharp HTTP
Tweet
(function() {
var po = document.createElement('script'); po.type = 'text/javascript'; po.async = true;
po.src = 'https://apis.google.com/js/plusone.js';
var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(po, s);
})();
