Search Results

Search found 22 results on 1 pages for 'cpickle'.

Page 1/1 | 1 

• cPickle ImportError: No module named multiarray

  - by Rafal

  Hello, I'm using cPickle to save my Database into file. The code looks like that: def Save_DataBase(): import cPickle from scipy import * from numpy import * a=Results.VersionName #filename='D:/results/'+a[a.find('/')+1:-a.find('/')-2]+Results.AssType[:3]+str(random.randint(0,100))+Results.Distribution+".lft" filename='D:/results/pppp.lft' plik=open(filename,'w') DataOutput=[[[DataBase.Arrays.Nodes,DataBase.Arrays.Links,DataBase.Arrays.Turns,DataBase.Arrays.Connectors,DataBase.Arrays.Zones], [DataBase.Nodes.Data,DataBase.Links.Data,DataBase.Turns.Data,DataBase.OrigConnectors.Data,DataBase.DestConnectors.Data,DataBase.Zones.Data], [DataBase.Nodes.DictionaryPy2Vis,DataBase.Links.DictionaryPy2Vis,DataBase.Turns.DictionaryPy2Vis,DataBase.OrigConnectors.DictionaryPy2Vis,DataBase.DestConnectors.DictionaryPy2Vis,DataBase.Zones.DictionaryPy2Vis], [DataBase.Nodes.DictionaryVis2Py,DataBase.Links.DictionaryVis2Py,DataBase.Turns.DictionaryVis2Py,DataBase.OrigConnectors.DictionaryVis2Py,DataBase.DestConnectors.DictionaryVis2Py,DataBase.Zones.DictionaryVis2Py], [DataBase.Paths.List]],[Results.VersionName,Results.noZones,Results.noNodes,Results.noLinks,Results.noTurns,Results.noTrips, Results.Times.VersionLoad,Results.Times.GetData,Results.Times.GetCoords,Results.Times.CrossTheTime,Results.Times.Plot_Cylinder, Results.AssType,Results.AssParam,Results.tStart,Results.tEnd,Results.Distribution,Results.tVector]] cPickle.dump(DataOutput, plik, protocol=0) plik.close()` And it works fine. Most of my Database rows are lists of a lists, vecor-like, or array-like data sets. But now when I input data, an error occurs: def Load_DataBase(): import cPickle from scipy import * from numpy import * filename='D:/results/pppp.lft' plik= open(filename, 'rb') """ first cPickle load approach """ A= cPickle.load(plik) """ fail """ """ Another approach - data format exact as in Output step above , also fails""" [[[DataBase.Arrays.Nodes,DataBase.Arrays.Links,DataBase.Arrays.Turns,DataBase.Arrays.Connectors,DataBase.Arrays.Zones], [DataBase.Nodes.Data,DataBase.Links.Data,DataBase.Turns.Data,DataBase.OrigConnectors.Data,DataBase.DestConnectors.Data,DataBase.Zones.Data], [DataBase.Nodes.DictionaryPy2Vis,DataBase.Links.DictionaryPy2Vis,DataBase.Turns.DictionaryPy2Vis,DataBase.OrigConnectors.DictionaryPy2Vis,DataBase.DestConnectors.DictionaryPy2Vis,DataBase.Zones.DictionaryPy2Vis], [DataBase.Nodes.DictionaryVis2Py,DataBase.Links.DictionaryVis2Py,DataBase.Turns.DictionaryVis2Py,DataBase.OrigConnectors.DictionaryVis2Py,DataBase.DestConnectors.DictionaryVis2Py,DataBase.Zones.DictionaryVis2Py], [DataBase.Paths.List]],[Results.VersionName,Results.noZones,Results.noNodes,Results.noLinks,Results.noTurns,Results.noTrips, Results.Times.VersionLoad,Results.Times.GetData,Results.Times.GetCoords,Results.Times.CrossTheTime,Results.Times.Plot_Cylinder, Results.AssType,Results.AssParam,Results.tStart,Results.tEnd,Results.Distribution,Results.tVector]]= cPickle.load(plik)` Error is (in both cases): A= cPickle.load(plik) ImportError: No module named multiarray Any Ideas? PS.

  Read the article

• How do I fix this unicode/cPickle error in Python?

  - by alex

  ids = cPickle.loads(gem.value) loads() argument 1 must be string, not unicode

  Read the article

• Python's cPickle deserialization from PHP?

  - by Ciantic

  Hi! I have to deserialize a dictionary in PHP that was serialized using cPickle in Python. In this specific case I probably could just regexp the wanted information, but is there a better way? Any extensions for PHP that would allow me to deserialize more natively the whole dictionary? Apparently it is serialized in Python like this: import cPickle as pickle data = { 'user_id' : 5 } pickled = pickle.dumps(data) print pickled Contents of such serialization cannot be pasted easily to here, because it contains binary data.

  Read the article

• load a pickle file from a zipfile

  - by eric.frederich

  For some reason I cannot get cPickle.load to work on the file-type object returned by ZipFile.open(). If I call read() on the file-type object returned by ZipFile.open() I can use cPickle.loads though. Example .... import zipfile import cPickle # the data we want to store some_data = {1: 'one', 2: 'two', 3: 'three'} # # create a zipped pickle file # zf = zipfile.ZipFile('zipped_pickle.zip', 'w', zipfile.ZIP_DEFLATED) zf.writestr('data.pkl', cPickle.dumps(some_data)) zf.close() # # cPickle.loads works # zf = zipfile.ZipFile('zipped_pickle.zip', 'r') sd1 = cPickle.loads(zf.open('data.pkl').read()) zf.close() # # cPickle.load doesn't work # zf = zipfile.ZipFile('zipped_pickle.zip', 'r') sd2 = cPickle.load(zf.open('data.pkl')) zf.close() Note: I don't want to zip just the pickle file but many files of other types. This is just an example.

  Read the article

• Accessing items from a dictionary using pickle efficiently in Python

  - by user248237

  I have a large dictionary mapping keys (which are strings) to objects. I pickled this large dictionary and at certain times I want to pull out only a handful of entries from it. The dictionary has usually thousands of entries total. When I load the dictionary using pickle, as follows: from cPickle import * # my dictionary from pickle, containing thousands of entries mydict = open(load('mypickle.pickle')) # accessing only handful of entries here for entry in relevant_entries: # find relevant entry value = mydict[entry] I notice that it can take up to 3-4 seconds to load the entire pickle, which I don't need, since I access only a tiny subset of the dictionary entries later on (shown above.) How can I make it so pickle only loads those entries that I have from the dictionary, to make this faster? Thanks.

  Read the article

• Django Admin interface with pickled set

  - by Rosarch

  I have a model that has a pickled set of strings. (It has to be pickled, because Django has no built in set field, right?) class Foo(models.Model): __bar = models.TextField(default=lambda: cPickle.dumps(set()), primary_key=True) def get_bar(self): return cPickle.loads(str(self.__bar)) def set_bar(self, values): self.__bar = cPickle.dumps(values) bar = property(get_bar, set_bar) I would like the set to be editable in the admin interface. Obviously the user won't be working with the pickled string directly. Also, the interface would need a widget for adding/removing strings from a set. What is the best way to go about doing this? I'm not super familiar with Django's admin system. Do I need to build a custom admin widget or something? Update: If I do need a custom widget, this looks helpful: http://www.fictitiousnonsense.com/archives/22

  Read the article

• Python - How to pickle yourself?

  - by Mark

  I want my class to implement Save and Load functions which simply do a pickle of the class. But apparently you cannot use 'self' in the fashion below. How can you do this? self = cPickle.load(f) cPickle.dump(self,f,2)

  Read the article

• Deterministic key serialization

  - by Mike Boers

  I'm writing a mapping class which uses SQLite as the storage backend. I am currently allowing only basestring keys but it would be nice if I could use a couple more types hopefully up to anything that is hashable (ie. same requirements as the builtin dict). To that end I would like to derive a deterministic serialization scheme. Ideally, I would like to know if any implementation/protocol combination of pickle is deterministic for hashable objects (e.g. can only use cPickle with protocol 0). I noticed that pickle and cPickle do not match: >>> import pickle >>> import cPickle >>> def dumps(x): ... print repr(pickle.dumps(x)) ... print repr(cPickle.dumps(x)) ... >>> dumps(1) 'I1\n.' 'I1\n.' >>> dumps('hello') "S'hello'\np0\n." "S'hello'\np1\n." >>> dumps((1, 2, 'hello')) "(I1\nI2\nS'hello'\np0\ntp1\n." "(I1\nI2\nS'hello'\np1\ntp2\n." Another option is to use repr to dump and ast.literal_eval to load. This would only be valid for builtin hashable types. I have written a function to determine if a given key would survive this process (it is rather conservative on the types it allows): def is_reprable_key(key): return type(key) in (int, str, unicode) or (type(key) == tuple and all( is_reprable_key(x) for x in key)) The question for this method is if repr itself is deterministic for the types that I have allowed here. I believe this would not survive the 2/3 version barrier due to the change in str/unicode literals. This also would not work for integers where 2**32 - 1 < x < 2**64 jumping between 32 and 64 bit platforms. Are there any other conditions (ie. do strings serialize differently under different conditions)? (If this all fails miserably then I can store the hash of the key along with the pickle of both the key and value, then iterate across rows that have a matching hash looking for one that unpickles to the expected key, but that really does complicate a few other things and I would rather not do it.) Any insights?

  Read the article

• [Python] How can I speed up unpickling large objects if I have plenty of RAM?

  - by conradlee

  It's taking me up to an hour to read a 1-gigabyte NetworkX graph data structure using cPickle (its 1-GB when stored on disk as a binary pickle file). Note that the file quickly loads into memory. In other words, if I run: import cPickle as pickle f = open("bigNetworkXGraph.pickle","rb") binary_data = f.read() # This part doesn't take long graph = pickle.loads(binary_data) # This takes ages How can I speed this last operation up? Note that I have tried pickling the data both in using both binary protocols (1 and 2), and it doesn't seem to make much difference which protocol I use. Also note that although I am using the "loads" (meaning "load string") function above, it is loading binary data, not ascii-data. I have 128gb of RAM on the system I'm using, so I'm hoping that somebody will tell me how to increase some read buffer buried in the pickle implementation.

  Read the article

• more efficient way to pickle a string

  - by gatoatigrado

  The pickle module seems to use string escape characters when pickling; this becomes inefficient e.g. on numpy arrays. Consider the following z = numpy.zeros(1000, numpy.uint8) len(z.dumps()) len(cPickle.dumps(z.dumps())) The lengths are 1133 characters and 4249 characters respectively. z.dumps() reveals something like "\x00\x00" (actual zeros in string), but pickle seems to be using the string's repr() function, yielding "'\x00\x00'" (zeros being ascii zeros). i.e. ("0" in z.dumps() == False) and ("0" in cPickle.dumps(z.dumps()) == True)

  Read the article

• default model field attribute in Django

  - by Rosarch

  I have a Django model: @staticmethod def getdefault(): print "getdefault called" return cPickle.dumps(set()) _applies_to = models.TextField(db_index=True, default=getdefault) For some reason, getdefault() is never called, even as I construct instances of this model and save them to the database. This seems to contradict the Django documentation: Field.default The default value for the field. This can be a value or a callable object. If callable it will be called every time a new object is created. Am I doing something wrong? Update: Originally, I had this, but then I switched to the above version to debug: _applies_to = models.TextField(db_index=True, default=cPickle.dumps(set())) I'm not sure why that wouldn't work.

  Read the article

• Optimizing Haskell code

  - by Masse

  I'm trying to learn Haskell and after an article in reddit about Markov text chains, I decided to implement Markov text generation first in Python and now in Haskell. However I noticed that my python implementation is way faster than the Haskell version, even Haskell is compiled to native code. I am wondering what I should do to make the Haskell code run faster and for now I believe it's so much slower because of using Data.Map instead of hashmaps, but I'm not sure I'll post the Python code and Haskell as well. With the same data, Python takes around 3 seconds and Haskell is closer to 16 seconds. It comes without saying that I'll take any constructive criticism :). import random import re import cPickle class Markov: def __init__(self, filenames): self.filenames = filenames self.cache = self.train(self.readfiles()) picklefd = open("dump", "w") cPickle.dump(self.cache, picklefd) picklefd.close() def train(self, text): splitted = re.findall(r"(\w+|[.!?',])", text) print "Total of %d splitted words" % (len(splitted)) cache = {} for i in xrange(len(splitted)-2): pair = (splitted[i], splitted[i+1]) followup = splitted[i+2] if pair in cache: if followup not in cache[pair]: cache[pair][followup] = 1 else: cache[pair][followup] += 1 else: cache[pair] = {followup: 1} return cache def readfiles(self): data = "" for filename in self.filenames: fd = open(filename) data += fd.read() fd.close() return data def concat(self, words): sentence = "" for word in words: if word in "'\",?!:;.": sentence = sentence[0:-1] + word + " " else: sentence += word + " " return sentence def pickword(self, words): temp = [(k, words[k]) for k in words] results = [] for (word, n) in temp: results.append(word) if n > 1: for i in xrange(n-1): results.append(word) return random.choice(results) def gentext(self, words): allwords = [k for k in self.cache] (first, second) = random.choice(filter(lambda (a,b): a.istitle(), [k for k in self.cache])) sentence = [first, second] while len(sentence) < words or sentence[-1] is not ".": current = (sentence[-2], sentence[-1]) if current in self.cache: followup = self.pickword(self.cache[current]) sentence.append(followup) else: print "Wasn't able to. Breaking" break print self.concat(sentence) Markov(["76.txt"]) -- module Markov ( train , fox ) where import Debug.Trace import qualified Data.Map as M import qualified System.Random as R import qualified Data.ByteString.Char8 as B type Database = M.Map (B.ByteString, B.ByteString) (M.Map B.ByteString Int) train :: [B.ByteString] -> Database train (x:y:[]) = M.empty train (x:y:z:xs) = let l = train (y:z:xs) in M.insertWith' (\new old -> M.insertWith' (+) z 1 old) (x, y) (M.singleton z 1) `seq` l main = do contents <- B.readFile "76.txt" print $ train $ B.words contents fox="The quick brown fox jumps over the brown fox who is slow jumps over the brown fox who is dead."

  Read the article

• Django: saving pickled object

  - by pistacchio

  Hi, i have a large dictionary I'd like to save. I have pickled it using cPickle.dumps and saved the result into a TextField. When trying to retrieve it (cPicle.loads) i get the following error: loads() argument 1 must be string, not unicode Does anybody have any experience in serializing python objects and storing them in a DB using Django? Thanks in advance.

  Read the article

• qwebview in pyside after packaged with pyinstaller goes wrong

  - by truease.com

  Here's my code import sys from PySide.QtCore import * from PySide.QtGui import * from PySide.QtWebKit import * from encodings import * from codecs import * class BrowserWindow( QWidget ): def __init__( self, parent=None ): QWidget.__init__( self, parent ) self.Setup() self.SetupEvent() def Setup( self ): self.setWindowTitle( u"Truease Speedy Browser" ) self.addr_input = QLineEdit() self.addr_go = QPushButton( "GO" ) self.addr_bar = QHBoxLayout() self.addr_bar.addWidget( self.addr_input ) self.addr_bar.addWidget( self.addr_go ) for attr in [ QWebSettings.AutoLoadImages, QWebSettings.JavascriptEnabled, QWebSettings.JavaEnabled, QWebSettings.PluginsEnabled, QWebSettings.JavascriptCanOpenWindows, QWebSettings.JavascriptCanAccessClipboard, QWebSettings.DeveloperExtrasEnabled, QWebSettings.SpatialNavigationEnabled, QWebSettings.OfflineStorageDatabaseEnabled, QWebSettings.OfflineWebApplicationCacheEnabled, QWebSettings.LocalStorageEnabled, QWebSettings.LocalStorageDatabaseEnabled, QWebSettings.LocalContentCanAccessRemoteUrls, QWebSettings.LocalContentCanAccessFileUrls, ]: QWebSettings.globalSettings().setAttribute( attr, True ) self.web_view = QWebView() self.web_view.load( "http://www.baidu.com" ) layout = QVBoxLayout() layout.addLayout( self.addr_bar ) layout.addWidget( self.web_view ) self.setLayout( layout ) def SetupEvent( self ): self.connect( self.addr_input, SIGNAL("editingFinished()"), self, SLOT("Load()"), ) self.connect( self.addr_go, SIGNAL("pressed()"), self, SLOT("Load()") ) self.connect( self.web_view, SIGNAL("urlChanged(const QUrl&)"), self, SLOT("SetURL()"), ) def Load( self, *args, **kwargs ): url = self.GetCleanedURL() if url != self.CurrentURL(): self.web_view.load( url ) def SetURL( self, *args, **kwargs ): self.addr_input.setText( self.CurrentURL() ) def GetCleanedURL( self ): url = self.addr_input.text().strip() if not url.startswith("http"): url = "http://" + url return url def CurrentURL( self ): url = self.web_view.url().toString() return url def Main(): app = QApplication( sys.argv ) widget = BrowserWindow() widget.show() return app.exec_() if __name__ == '__main__': sys.exit( Main() ) I works well when i using python browser.py. but it goes wrong after packaged with pyinstaller -w browser.py. it doesn't load images can only display correct text in utf-8 And this is the pyinstaller output: E:\true\wuk\app2>pyinstaller -w b.py 16 INFO: wrote E:\true\wuk\app2\b.spec 16 INFO: Testing for ability to set icons, version resources... 32 INFO: ... resource update available 32 INFO: UPX is not available. 46 INFO: Processing hook hook-os 141 INFO: Processing hook hook-time 157 INFO: Processing hook hook-cPickle 218 INFO: Processing hook hook-_sre 312 INFO: Processing hook hook-cStringIO 407 INFO: Processing hook hook-encodings 421 INFO: Processing hook hook-codecs 750 INFO: Processing hook hook-httplib 750 INFO: Processing hook hook-email 843 INFO: Processing hook hook-email.message 1046 WARNING: library python%s%s required via ctypes not found 1171 INFO: Extending PYTHONPATH with E:\true\wuk\app2 1171 INFO: checking Analysis 1171 INFO: building because b.py changed 1171 INFO: running Analysis out00-Analysis.toc 1171 INFO: Adding Microsoft.VC90.CRT to dependent assemblies of final executable 1171 INFO: Searching for assembly x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_x-ww ... 1171 INFO: Found manifest C:\WINDOWS\WinSxS\Manifests\x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_x-ww_d08d0375.manifest 1187 INFO: Searching for file msvcr90.dll 1187 INFO: Found file C:\WINDOWS\WinSxS\x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_x-ww_d08d0375\msvcr90.dll 1187 INFO: Searching for file msvcp90.dll 1187 INFO: Found file C:\WINDOWS\WinSxS\x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_x-ww_d08d0375\msvcp90.dll 1187 INFO: Searching for file msvcm90.dll 1187 INFO: Found file C:\WINDOWS\WinSxS\x86_Microsoft.VC90.CRT_1fc8b3b9a1e18e3b_9.0.21022.8_x-ww_d08d0375\msvcm90.dll 1266 INFO: Analyzing D:\Applications\Python\lib\site-packages\pyinstaller-2.1-py2.7.egg\PyInstaller\loader\_pyi_bootstrap.py 1266 INFO: Processing hook hook-os 1282 INFO: Processing hook hook-site 1296 INFO: Processing hook hook-encodings 1391 INFO: Processing hook hook-time 1407 INFO: Processing hook hook-cPickle 1468 INFO: Processing hook hook-_sre 1578 INFO: Processing hook hook-cStringIO 1671 INFO: Processing hook hook-codecs 2016 INFO: Processing hook hook-httplib 2016 INFO: Processing hook hook-email 2109 INFO: Processing hook hook-email.message 2312 WARNING: library python%s%s required via ctypes not found 2468 INFO: Processing hook hook-pydoc 2516 INFO: Analyzing D:\Applications\Python\lib\site-packages\pyinstaller-2.1-py2.7.egg\PyInstaller\loader\pyi_importers.py 2609 INFO: Analyzing D:\Applications\Python\lib\site-packages\pyinstaller-2.1-py2.7.egg\PyInstaller\loader\pyi_archive.py 2687 INFO: Analyzing D:\Applications\Python\lib\site-packages\pyinstaller-2.1-py2.7.egg\PyInstaller\loader\pyi_carchive.py 2782 INFO: Analyzing D:\Applications\Python\lib\site-packages\pyinstaller-2.1-py2.7.egg\PyInstaller\loader\pyi_os_path.py 2782 INFO: Analyzing b.py 2796 INFO: Processing hook hook-PySide 2875 INFO: Hidden import 'codecs' has been found otherwise 2875 INFO: Hidden import 'encodings' has been found otherwise 2875 INFO: Looking for run-time hooks 7766 INFO: Using Python library C:\WINDOWS\system32\python27.dll 7796 INFO: E:\true\wuk\app2\build\b\out00-Analysis.toc no change! 7796 INFO: checking PYZ 7812 INFO: checking PKG 7812 INFO: building because E:\true\wuk\app2\build\b\b.exe.manifest changed 7812 INFO: building PKG (CArchive) out00-PKG.pkg 7828 INFO: checking EXE 7843 INFO: rebuilding out00-EXE.toc because pkg is more recent 7843 INFO: building EXE from out00-EXE.toc 7843 INFO: Appending archive to EXE E:\true\wuk\app2\build\b\b.exe 7843 INFO: checking COLLECT 7843 INFO: building COLLECT out00-COLLECT.toc Use pyinstaller browser.py, and in the console window i got QFont::setPixelSize: Pixel size <= 0 (0) QSslSocket: cannot call unresolved function SSLv23_client_method QSslSocket: cannot call unresolved function SSL_CTX_new QSslSocket: cannot call unresolved function SSL_library_init QSslSocket: cannot call unresolved function ERR_get_error QSslSocket: cannot call unresolved function SSLv23_client_method QSslSocket: cannot call unresolved function SSL_CTX_new QSslSocket: cannot call unresolved function SSL_library_init QSslSocket: cannot call unresolved function ERR_get_error QSslSocket: cannot call unresolved function SSLv23_client_method QSslSocket: cannot call unresolved function SSL_CTX_new QSslSocket: cannot call unresolved function SSL_library_init QSslSocket: cannot call unresolved function ERR_get_error QSslSocket: cannot call unresolved function SSLv23_client_method QSslSocket: cannot call unresolved function SSL_CTX_new QSslSocket: cannot call unresolved function SSL_library_init QSslSocket: cannot call unresolved function ERR_get_error QFont::setPixelSize: Pixel size <= 0 (0)

  Read the article

• How to save big "database-like" class in python

  - by Rafal

  Hi there, I'm doing a project with reasonalby big DataBase. It's not a probper DB file, but a class with format as follows: DataBase.Nodes.Data=[[] for i in range(1,1000)] f.e. this DataBase is all together something like few thousands rows. Fisrt question - is the way I'm doing efficient, or is it better to use SQL, or any other "proper" DB, which I've never used actually. And the main question - I'd like to save my DataBase class with all record, and then re-open it with Python in another session. Is that possible, what tool should I use? cPickle - it seems to be only for strings, any other? In matlab there's very useful functionality named save workspace - it saves all Your variables to a file that You can open at another session - this would be vary useful in python!

  Read the article

• Best way to save complex Python data structures across program sessions (pickle, json, xml, database

  - by Malcolm

  Looking for advice on the best technique for saving complex Python data structures across program sessions. Here's a list of techniques I've come up with so far: pickle/cpickle json jsonpickle xml database (like SQLite) Pickle is the easiest and fastest technique, but my understanding is that there is no guarantee that pickle output will work across various versions of Python 2.x/3.x or across 32 and 64 bit implementations of Python. Json only works for simple data structures. Jsonpickle seems to correct this AND seems to be written to work across different versions of Python. Serializing to XML or to a database is possible, but represents extra effort since we would have to do the serialization ourselves manually. Thank you, Malcolm

  Read the article

• [Python] How do I read binary pickle data first, then unpickle it?

  - by conradlee

  I'm unpickling a NetworkX object that's about 1GB in size on disk. Although I saved it in the binary format (using protocol 2), it is taking a very long time to unpickle this file---at least half an hour. The system I'm running on has plenty of system memory (128 GB), so that's not the bottleneck. I've read here that pickling can be sped up by first reading the entire file into memory, and then unpickling it (that particular thread refers to python 3.0, which I'm not using, but the point should still be true in python 2.6). How do I first read the binary file, and then unpickle it? I have tried: import cPickle as pickle f = open("big_networkx_graph.pickle","rb") bin_data = f.read() graph_data = pickle.load(bin_data) But this returns: TypeError: argument must have 'read' and 'readline' attributes Any ideas?

  Read the article

• Any way to set or overwrite the __line__ and __file__ metadata?

  - by charles.merriam

  I'm writing some code that needs to change function signatures. Right now, I'm using Simionato's FunctionMaker class, which uses the (hacky) inspect module, and does a compile. Unfortunately, this still loses the line and file metadata. Does anyone know: If it is possible to overwrite these values in some odd way? If hacking up a class with a complex getattribute() to intercept the values and also try to make the class looks like a function is any more possible than a moose with a flying nun hat? Is there an alternative to the (hacky) inspect module? PEP 362 is dead dead dead? I know decorators and cPickle users fight with this. What other situations is the read only metadata in people's way? I appreciate any insights. Thank you.

  Read the article

• Get Python 2.7's 'json' to not throw an exception when it encounters random byte strings

  - by Chris Dutrow

  Trying to encode a a dict object into json using Python 2.7's json (ie: import json). The object has some byte strings in it that are "pickled" data using cPickle, so for json's purposes, they are basically random byte strings. I was using django.utils's simplejson and this worked fine. But I recently switched to Python 2.7 on google app engine and they don't seem to have simplejson available anymore. Now that I am using json, it throws an exception when it encounters bytes that aren't part of UTF-8. The error that I'm getting is: UnicodeDecodeError: 'utf8' codec can't decode byte 0x80 in position 0: invalid start byte It would be nice if it printed out a string of the character codes like the debugging might do, ie: \u0002]q\u0000U\u001201. But I really don't much care how it handles this data just as long as it doesn't throw an exception and continues serializing the information that it does recognize. How can I make this happen? Thanks!

  Read the article

• Return a list of imported Python modules used in a script?

  - by Jono Bacon

  Hi All, I am writing a program that categorizes a list of Python files by which modules they import. As such I need to scan the collection of .py files ad return a list of which modules they import. As an example, if one of the files I import has the following lines: import os import sys, gtk I would like it to return: ["os", "sys", "gtk"] I played with modulefinder and wrote: from modulefinder import ModuleFinder finder = ModuleFinder() finder.run_script('testscript.py') print 'Loaded modules:' for name, mod in finder.modules.iteritems(): print '%s ' % name, but this returns more than just the modules used in the script. As an example in a script which merely has: import os print os.getenv('USERNAME') The modules returned from the ModuleFinder script return: tokenize heapq __future__ copy_reg sre_compile _collections cStringIO _sre functools random cPickle __builtin__ subprocess cmd gc __main__ operator array select _heapq _threading_local abc _bisect posixpath _random os2emxpath tempfile errno pprint binascii token sre_constants re _abcoll collections ntpath threading opcode _struct _warnings math shlex fcntl genericpath stat string warnings UserDict inspect repr struct sys pwd imp getopt readline copy bdb types strop _functools keyword thread StringIO bisect pickle signal traceback difflib marshal linecache itertools dummy_thread posix doctest unittest time sre_parse os pdb dis ...whereas I just want it to return 'os', as that was the module used in the script. Can anyone help me achieve this?

  Read the article

• Save PyML.classifiers.multi.OneAgainstRest(SVM()) object?

  - by Michael Aaron Safyan

  I'm using PYML to construct a multiclass linear support vector machine (SVM). After training the SVM, I would like to be able to save the classifier, so that on subsequent runs I can use the classifier right away without retraining. Unfortunately, the .save() function is not implemented for that classifier, and attempting to pickle it (both with standard pickle and cPickle) yield the following error message: pickle.PicklingError: Can't pickle : it's not found as __builtin__.PySwigObject Does anyone know of a way around this or of an alternative library without this problem? Thanks. Edit/Update I am now training and attempting to save the classifier with the following code: mc = multi.OneAgainstRest(SVM()); mc.train(dataset_pyml,saveSpace=False); for i, classifier in enumerate(mc.classifiers): filename=os.path.join(prefix,labels[i]+".svm"); classifier.save(filename); Notice that I am now saving with the PyML save mechanism rather than with pickling, and that I have passed "saveSpace=False" to the training function. However, I am still gettting an error: ValueError: in order to save a dataset you need to train as: s.train(data, saveSpace = False) However, I am passing saveSpace=False... so, how do I save the classifier(s)? P.S. The project I am using this in is pyimgattr, in case you would like a complete testable example... the program is run with "./pyimgattr.py train"... that will get you this error. Also, a note on version information: [michaelsafyan@codemage /Volumes/Storage/classes/cse559/pyimgattr]$ python Python 2.6.1 (r261:67515, Feb 11 2010, 00:51:29) [GCC 4.2.1 (Apple Inc. build 5646)] on darwin Type "help", "copyright", "credits" or "license" for more information. import PyML print PyML.__version__ 0.7.0

  Read the article

• Return a list of important Python modules in a script?

  - by Jono Bacon

  Hi All, I am writing a program that categorizes a list of Python files by which modules they import. As such I need to scan the collection of .py files ad return a list of which modules they import. As an example, if one of the files I import has the following lines: import os import sys, gtk I would like it to return: ["os", "sys", "gtk"] I played with modulefinder and wrote: from modulefinder import ModuleFinder finder = ModuleFinder() finder.run_script('testscript.py') print 'Loaded modules:' for name, mod in finder.modules.iteritems(): print '%s ' % name, but this returns more than just the modules used in the script. As an example in a script which merely has: import os print os.getenv('USERNAME') The modules returned from the ModuleFinder script return: tokenize heapq __future__ copy_reg sre_compile _collections cStringIO _sre functools random cPickle __builtin__ subprocess cmd gc __main__ operator array select _heapq _threading_local abc _bisect posixpath _random os2emxpath tempfile errno pprint binascii token sre_constants re _abcoll collections ntpath threading opcode _struct _warnings math shlex fcntl genericpath stat string warnings UserDict inspect repr struct sys pwd imp getopt readline copy bdb types strop _functools keyword thread StringIO bisect pickle signal traceback difflib marshal linecache itertools dummy_thread posix doctest unittest time sre_parse os pdb dis ...whereas I just want it to return 'os', as that was the module used in the script. Can anyone help me achieve this?

  Read the article

1