What are the commonly confused encodings that may result in identical test data?

Posted by makerofthings7 on Programmers See other posts from Programmers or by makerofthings7
Published on 2012-06-30T18:42:22Z Indexed on 2012/06/30 21:24 UTC
Read the original article Hit count: 252

Filed under:

character-encoding

|

utf-8

|

ascii

I'm fixing code that is using ASCIIEncoding in some places and UTF-8 encoding in other functions.

Since we aren't using the UTF-8 features, all of our unit tests passed, but I want to create a heightened awareness of encodings that produce similar results and may not be fully tested.

I don't want to limit this to just UTF-8 vs ASCII, since I think issue with code that handles ASN.1 fields and other code working with Base64.

So, what are the commonly confused encodings that may result in identical test data?

© Programmers or respective owner

Related posts about character-encoding

Harmonizing Character Encoding Between Imported Data and MySQL

as seen on Internet.com - Search for 'Internet.com'
MySQL's Latin-1 default encoding combined with MySQL 4.1.12's (or greater) UTF8 encoding allows the maximum number of characters codes, however incoming data with different character encoding can still present problems. Rob Gravelle shows you how to avoid problems before a lot of work is required… >>> More
Harmonizing Character Encoding Between Imported Data and MySQL

as seen on Internet.com - Search for 'Internet.com'
MySQL's Latin-1 default encoding combined with MySQL 4.1.12's (or greater) UTF8 encoding allows the maximum number of characters codes, however incoming data with different character encoding can still present problems. Rob Gravelle shows you how to avoid problems before a lot of work is required… >>> More
How to cross-reference many character encodings with ASCII OR UTFx?

as seen on Programmers - Search for 'Programmers'
I'm working with a binary structure, the goal of which is to index the significance of specific bits for any character encoding so that we may trigger events while doing specific checks against the profile. Each character encoding scheme has an associated system record. This record's leading value… >>> More
Determining default character set of platform in Java

as seen on Stack Overflow - Search for 'Stack Overflow'
I am programming in Java I have the code as: byte[] b = test.getBytes(); In the api it is specified that if we do not specify character encoding it takes the default platform character encoding. What is meant by "default platform character encoding" ? Does it mean the Java encoding or the OS… >>> More
Perl character encoding

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi People, I have an environment variable set in Windows as TEST=abc£ which uses Windows-1252 code page. Now when I run a perl program - 'test.pl', this environment value comes properly. When I call another perl code - 'test2.pl' from 'test1.pl' either by system(..) or Win32::process(..), the environment… >>> More

Related posts about utf-8

Why can't I change the AU_AU locale to en_US?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
/bin/bash: warning: setlocale: LC_ALL: cannot change locale ( (unset)) Generating locales... en_US.ISO-8859-1... /usr/sbin/locale-gen: line 177: warning: setlocale: LC_ALL: cannot change locale ( (unset)) done Generation complete. ganesha@ubuntu:~$ sudo update_locale LANG=en_US sudo: update_locale:… >>> More
Reading a plist utf-8 value as utf-16

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm working on an iphone app that needs to display superscripts and subscripts. I'm using a picker to read in data from a plist but the unicode values aren't being displayed corretly in the pickerview. Subscripts and superscripts are not being recognized. I'm assuming this is due to the encoding… >>> More
Forcing a mixed ISO-8859-1 and UTF-8 multi-line string into UTF-8

as seen on Stack Overflow - Search for 'Stack Overflow'
Consider the following problem: A multi-line string $junk contains some lines which are encoded in UTF-8 and some in ISO-8859-1. I don't know a priori which lines are in which encoding, so heuristics will be needed. I want to turn $junk into pure UTF-8 with proper re-encoding of the ISO-8859-1… >>> More
How can I tell if a CSV is in UTF-7 or UTF-8

as seen on Stack Overflow - Search for 'Stack Overflow'
Excel seems to save CSV files in (what I think is) UTF-7, despite the fact that most information I have read suggest that in general, you should not UTF-7. Indeed, other applications (Text pad, which lets me choose) save things in UTF-8 (or Unicode etc, but UTF-7 is not even an option). Using .NET… >>> More
Confused about C++'s std::wstring, UTF-16, UTF-8 and displaying strings in a windows GUI

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm working on a english only C++ program for Windows where we were told "always use std::wstring", but it seems like nobody on the team really has much of an understanding beyond that. I already read the question titled "std::wstring VS std::string. It was very helpful, but I still don't quite… >>> More