Get Python 2.7's 'json' to not throw an exception when it encounters random byte strings

Posted by Chris Dutrow on Stack Overflow See other posts from Stack Overflow or by Chris Dutrow
Published on 2012-09-07T03:36:59Z Indexed on 2012/09/07 3:37 UTC
Read the original article Hit count: 199

Trying to encode a a dict object into json using Python 2.7's json (ie: import json).

The object has some byte strings in it that are "pickled" data using cPickle, so for json's purposes, they are basically random byte strings. I was using django.utils's simplejson and this worked fine. But I recently switched to Python 2.7 on google app engine and they don't seem to have simplejson available anymore.

Now that I am using json, it throws an exception when it encounters bytes that aren't part of UTF-8. The error that I'm getting is:

UnicodeDecodeError: 'utf8' codec can't decode byte 0x80 in position 0: invalid start byte

It would be nice if it printed out a string of the character codes like the debugging might do, ie: \u0002]q\u0000U\u001201. But I really don't much care how it handles this data just as long as it doesn't throw an exception and continues serializing the information that it does recognize.

How can I make this happen?

Thanks!

© Stack Overflow or respective owner

Related posts about JSON

Related posts about google-app-engine