IDN aware tools to encode/decode human readable IRI to/from valid URI

Posted by Denis Otkidach on Stack Overflow See other posts from Stack Overflow or by Denis Otkidach
Published on 2010-05-14T09:19:30Z Indexed on 2010/05/14 9:24 UTC
Read the original article Hit count: 437

Filed under:

html

|

idn

|

iri

|

python

|

JavaScript

Let's assume a user enter address of some resource and we need to translate it to:

<a href="valid URI here">human readable form</a>

HTML4 specification refers to RFC 3986 which allows only ASCII alphanumeric characters and dash in host part and all non-ASCII character in other parts should be percent-encoded. That's what I want to put in href attribute to make link working properly in all browsers. IDN should be encoded with Punycode.

HTML5 draft refers to RFC 3987 which also allows percent-encoded unicode characters in host part and a large subset of unicode in both host and other parts without encoding them. User may enter address in any of these forms. To provide human readable form of it I need to decode all printable characters. Note that some parts of address might not correspond to valid UTF-8 sequences, usually when target site uses some other character encoding.

An example of what I'd like to get:

<a href="http://xn--80aswg.xn--p1ai/%D0%BF%D1%83%D1%82%D1%8C?%D0%B7%D0%B0%D0%BF%D1%80%D0%BE%D1%81">
http://????.??/???????????</a>

Are there any tools to solve these tasks? I'm especially interested in libraries for Python and JavaScript.

© Stack Overflow or respective owner

Related posts about html

Install usblib package - Ubuntu

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I need the package libusb for another package I am installing. I tried the following which seemed to install the package, sudo apt-get install libusb-dev but when I try to install the other package I get, configure: error: Package requirements (libusb-1.0 >= 0.9.1) were not met: No package… >>> More
Prevent malicious vulnerability scan increasing load on a server

as seen on Server Fault - Search for 'Server Fault'
Hi all, this week we have been suffering some malicious vulnerability scans to our servers, increasing the load on them, making them nearly unusable. The attack is easy to defend, just blocking the offending ip, but only after discovering it. Is there any form of prevent it? Is it normal that… >>> More
can't install psycopg2 in my env on mac os x lion

as seen on Server Fault - Search for 'Server Fault'
I tried install psycopg2 via pip in my virtual env, but got this error: ld: library not found for -lpq (full log here: http://pastebin.com/XdmGyJ4u ) I tried install postgres 9.1 from .dmg and via port, (gksks)iMac-Alexander:~ lorddaedra$ locate libpq /Developer/SDKs/MacOSX10.7.sdk/usr/include/libpq /Developer/SDKs/MacOSX10… >>> More
Bitnami redmine error SVN

as seen on Server Fault - Search for 'Server Fault'
I'm installing the Bitnami Redmine stack (redmine + subversion). Firstly I install configure and test it locally (Ubuntu 14.04 LTS). And everything is OK. I install Bitnami stack on server (Red Hat 4.4.7-4) and configure SVN. I commit files into SVN and connect project into Redmine with SVN repository… >>> More
Can the .htaccess file slow down a website to a crawl? If so, are there better ways to solve these problems with different rewrite rules and such?

as seen on Pro Webmasters - Search for 'Pro Webmasters'
here is my htaccess file...... RewriteCond %{REQUEST_URI} ^/patients/billing/FAQ_billing\.html$ [OR] RewriteCond %{REQUEST_URI} ^/patients/billing/getintouch\.html$ RewriteRule ^patients/billing/(.*)\.html$ $1.php [L,NC] RewriteCond %{REQUEST_URI} ^/patients/findadoctor/a\.html$ [OR] RewriteCond… >>> More

Related posts about idn

IDN and HTTP_HOST

as seen on Server Fault - Search for 'Server Fault'
So, when I want to link my users to a specific page I always use (in php): "http://" . $_SERVER["HTTP_HOST"] . "/page.php", to be sure that the link points to the page they're currently surfing (and not one of the server aliases). But with IDN names, HTTP_HOST is set to "xn--hemmabst-5za.net" (for… >>> More
Allowed unicode characters in IDN host labels

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi all, Im currently working on a "proper" URI validator and currently it all comes down to hostname validation, the rest isnt that tricky. Im stuck at IDN hostname labels (e.g. containing unicode; possible punycode encoded strings have been decoded at this point). My first idea was basicly a regex… >>> More
IDN aware tools to encode/decode human readable IRI to/from valid URI

as seen on Stack Overflow - Search for 'Stack Overflow'
Let's assume a user enter address of some resource and we need to translate it to: <a href="valid URI here">human readable form</a> HTML4 specification refers to RFC 3986 which allows only ASCII alphanumeric characters and dash in host part and all non-ASCII character in other parts… >>> More
idn domain not showing correct in firefox

as seen on Server Fault - Search for 'Server Fault'
I have bought this domain bilvärdering.nu http://xn--bilvrdering-o8a.nu/ in firefox adressbar it show up like http://xn--bilvrdering-o8a.nu/ but in chrome and IE it show up with the correct characters http://bilvärdering.nu/ How can I make firefox show correct characters in its adressbar? >>> More
idn_to_utf8 alternative in PHP 5

as seen on Stack Overflow - Search for 'Stack Overflow'
Is there any alternative for idn_to_utf8 function in PHP 5 < 5.3.0 ? >>> More