python UTF16LE file to UTF8 encoding

Posted by Qiao on Stack Overflow See other posts from Stack Overflow or by Qiao
Published on 2011-01-08T10:30:50Z Indexed on 2011/01/08 10:53 UTC
Read the original article Hit count: 247

Filed under:

python

|

encodings

I have big file with utf16le (BOM) encoding.
Is it possible to convert it to usual UTF8 by python?

Something like

file_old = open('old.txt', mode='r', encoding='utf_16_le')
file_new = open('new.txt', mode='w', encoding='utf-8')

text = file_old.read()

file_new.write(text.encode('utf-8'))

http://docs.python.org/release/2.3/lib/node126.html (-- utf_16_le UTF-16LE)

Not working. Can't understand "TypeError: must be str, not bytes" error.
python 3

© Stack Overflow or respective owner

Related posts about python

unmet dependencies in Ubuntu 12.04

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I tried today to install a dvb-card on my Ubuntu 12.04 (Linux blauhai-linux 3.2.0-25-generic #40-Ubuntu SMP Wed May 23 20:30:51 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux ). The installation failed with an error. After that, i tried to install python (it was already installed but i got this error): linux:~$… >>> More
How can I get sikuli-ide to work?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I installed sikuli-ide with sudo apt-get install sikuli-ide Everything was fine until I tried to start it from the terminal. I typed sikuli-ide But the only response I got was [info] locale: en_US The application was not started, furthermore there is no desktop file and sikuli-ide does not… >>> More
Getting PATH right for python after MacPorts install

as seen on Super User - Search for 'Super User'
I can't import some python libraries (PIL, psycopg2) that I just installed with MacPorts. I looked through these forums, and tried to adjust my PATH variable in $HOME/.bash_profile in order to fix this but it did not work. I added the location of PIL and psycopg2 to PATH. I know that Terminal is… >>> More
call python with system() in R to run a python script emulating the python console

as seen on Stack Overflow - Search for 'Stack Overflow'
I want to pass a chunk of Python code to Python in R with something like system('python ...'), and I'm wondering if there is an easy way to emulate the python console in this case. For example, suppose the code is "print 'hello world'", how can I get the output like this in R? >>> print… >>> More
Python - Calling a non python program from python?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I am currently struggling to call a non python program from a python script. I have a ~1000 files that when passed through this C++ program will generate ~1000 outputs. Each output file must have a distinct name. The command I wish to run is of the form: program_name -input -output -o1 -o2… >>> More

Related posts about encodings

search PDFs with non-standard character encodings

as seen on Super User - Search for 'Super User'
Some PDF files produce garbage ("mojibake") when you copy text. This makes it impossible to search them (whatever you search for will not match the garbage). Does anyone have an easy workaround? An example: TEAC TV manual EU2816STF BTW: I am using Adobe Reader - perhaps an alternative viewer might… >>> More
mod_deflate Supported Encodings for Compression

as seen on Stack Overflow - Search for 'Stack Overflow'
It seems to me, that mod_deflate in Apache 2.2 will always return: Content-Encoding: gzip and never: Content-Encoding: deflate It was explained to me, that although there may be a deflate algorithm, mod_deflate is named after a file-format, in which the algorithm could be any of: gzip, bzip.… >>> More
List of Character Encodings

as seen on Stack Overflow - Search for 'Stack Overflow'
Is There A Book or Site That Teaches And Also Includes A Complete List of Character Encoding's That Includes Hexadecimal, Decimal and Name Versions? If you can name a couple of books and sites, that would be very helpful thank you. >>> More
Java Charset problem on linux

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, problem: I have a string containing special characters which i convert to bytes and vice versa..the conversion works properly on windows but on linux the special character is not converted properly.the default charset on linux is UTF-8 as seen with Charset.defaultCharset.getdisplayName() however… >>> More
Python encoding ISO to UTF8

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello everyone, I am trying to read my emails using a Python script (Python 2.5 and PyPy) Some of my results are not in ASCII and i get strings like this: =?ISO-8859-7?B?0OXm7/Dv8d/hIPP07+0gyuno4enx/u3h?=' Is there any way to decode it and convert to utf-8 so that i can process it? I tried … >>> More