sql and web encoding problem

Posted by Marki on Server Fault See other posts from Server Fault or by Marki
Published on 2010-02-18T02:37:02Z Indexed on 2010/03/29 17:03 UTC
Read the original article Hit count: 421

Filed under:

utf8

|

encoding

|

mysql

|

php

Guys, I've got an encoding problem I believe.

I have upgraded from phpBB2 to phpBB3.

The old databases were in latin1, the new ones have utf8 encoding.

Already during the upgrade process some rows of the DB were only read partly into the new version, because of strange characters as it turned out.

When I use PHP's mb_convert_encoding() function to convert those strings to UTF8 they end up e.g. as 0x0093, i.e. they must have been some kind of double quotes.

Even after doing this conversion, they still show up as 0x0093 in the browser (the squares with 0093 in them when the browser does not know what to display).

Can someone explain the problem here?

I'm a little confused and afraid I don't see all the dependencies that need to work to have the correct encodings and the correct display thereof...

© Server Fault or respective owner

Related posts about utf8

How can I install new locale to Ubuntu?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
$ locale -a get output like this: C C.UTF-8 en_AG en_AG.utf8 en_AU.utf8 en_BW.utf8 en_CA.utf8 en_DK.utf8 en_GB.utf8 en_HK.utf8 en_IE.utf8 en_IN en_IN.utf8 en_NG en_NG.utf8 en_NZ.utf8 en_PH.utf8 en_SG.utf8 en_US.utf8 en_ZA.utf8 en_ZM en_ZM.utf8 en_ZW.utf8 POSIX zh_CN.utf8 zh_SG.utf8 How can I… >>> More
Strange display language in gnome shell

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I logged in gnome-shell, and found that the display language is set to some strange asian language (I think) without my prompt. I tried to change the locale settings but found that the default language is English (how?) despite of that strange language. Here's a snapshot, See the strange word instead… >>> More
gVim characters unreadable at random times

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
Screenshot - Anyone know what causes it and how to fix? It only started happening today, while I've been using gVim for a couple of months now. Update: Output of locale LANG=en_US.utf8 LC_CTYPE="en_US.utf8" LC_NUMERIC="en_US.utf8" LC_TIME="en_US.utf8" LC_COLLATE="en_US.utf8" LC_MONETARY="en_US… >>> More
utf8 problem with Perl and XML::Parser

as seen on Stack Overflow - Search for 'Stack Overflow'
I encountered a problem dealing with utf8, XML and Perl. The following is the smallest piece of code and data in order to reproduce the problem. Here's an XML file that needs to be parsed: <?xml version="1.0" encoding="utf-8"?> <test> <words>???????????? ??????? ????????? ??… >>> More
Applying languages / locale selectively: is it possible?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I am a Dutch user and prefer the my local date & time format, system wide. I have no trouble speaking or understanding English and find it very useful to have the rest of my system configured in English to make my life easier when I need to Google a term, for example. Is it possible to apply… >>> More

Related posts about encoding

<?xml version=“1.0” encoding=“UTF-8”?> not <?xml version='1.0' encoding='UTF-8'?>

as seen on Stack Overflow - Search for 'Stack Overflow'
I am using lxml with tree.write(xmlFileOut, pretty_print = True, xml_declaration = True, encoding='UTF-8' to write out my opened and edited xml file, but I absolutely need to have the xml declaration as <?xml version=“1.0” encoding=“UTF-8”?> and NOT <?xml version='1.0' encoding='UTF-8'… >>> More
Ivar definitions show 'long' type encoding as 'long long' type encoding

as seen on Stack Overflow - Search for 'Stack Overflow'
I've found what I think may be a bug with Ivar and Objective-C runtime. I'm using XCode 3.2.1 and associated libraries, developing a 64 bit app on X86_64 (MacBook Pro). Where I would expect the type encoding for the following "longVal" to be 'l', the Ivar encoding is showing a 'q' (which is a 'long… >>> More
How to avoid encoding the key of request parameters being encoding

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm trying to send a http request using WS.url() with a action receive a custom class parameter like public static void add(@Valid MyPage info) {...} There is a Map in MyPage @Required public Map<String, String> content = new HashMap<String, String>(); But When I try to send a request… >>> More
C# Check if character exists in encoding

as seen on Stack Overflow - Search for 'Stack Overflow'
I am writing a program that a part renders a bitmap font in CP437. In a function that renders the text with I want to be able to check whether a char is available in CP437 before the encoding conversion, like: public static void DrawCharacter(this Graphics g, char c) { if (char_exist_in_encoding(Encoding… >>> More
How to detect the character encoding of a text file?

as seen on Stack Overflow - Search for 'Stack Overflow'
I try to detect which character encoding is used in my file. I try with this code to get the standard encoding public static Encoding GetFileEncoding(string srcFile) { // *** Use Default of Encoding.Default (Ansi CodePage) Encoding enc = Encoding.Default; // *** Detect byte… >>> More