Search Results

Search found 16680 results on 668 pages for 'python datetime'.

Page 387/668 | < Previous Page | 383 384 385 386 387 388 389 390 391 392 393 394  | Next Page >

  • Fastest way to generate delimited string from 1d numpy array

    - by Abiel
    I have a program which needs to turn many large one-dimensional numpy arrays of floats into delimited strings. I am finding this operation quite slow relative to the mathematical operations in my program and am wondering if there is a way to speed it up. For example, consider the following loop, which takes 100,000 random numbers in a numpy array and joins each array into a comma-delimited string. import numpy as np x = np.random.randn(100000) for i in range(100): ",".join(map(str, x)) This loop takes about 20 seconds to complete (total, not each cycle). In contrast, consider that 100 cycles of something like elementwise multiplication (x*x) would take than one 1/10 of a second to complete. Clearly the string join operation creates a large performance bottleneck; in my actual application it will dominate total runtime. This makes me wonder, is there a faster way than ",".join(map(str, x))? Since map() is where almost all the processing time occurs, this comes down to the question of whether there a faster to way convert a very large number of numbers to strings.

    Read the article

  • Why do I get rows of zeros in my 2D fft?

    - by Nicholas Pringle
    I am trying to replicate the results from a paper. "Two-dimensional Fourier Transform (2D-FT) in space and time along sections of constant latitude (east-west) and longitude (north-south) were used to characterize the spectrum of the simulated flux variability south of 40degS." - Lenton et al(2006) The figures published show "the log of the variance of the 2D-FT". I have tried to create an array consisting of the seasonal cycle of similar data as well as the noise. I have defined the noise as the original array minus the signal array. Here is the code that I used to plot the 2D-FT of the signal array averaged in latitude: import numpy as np from numpy import ma from matplotlib import pyplot as plt from Scientific.IO.NetCDF import NetCDFFile ### input directory indir = '/home/nicholas/data/' ### get the flux data which is in ### [time(5day ave for 10 years),latitude,longitude] nc = NetCDFFile(indir + 'CFLX_2000_2009.nc','r') cflux_southern_ocean = nc.variables['Cflx'][:,10:50,:] cflux_southern_ocean = ma.masked_values(cflux_southern_ocean,1e+20) # mask land nc.close() cflux = cflux_southern_ocean*1e08 # change units of data from mmol/m^2/s ### create an array that consists of the seasonal signal fro each pixel year_stack = np.split(cflux, 10, axis=0) year_stack = np.array(year_stack) signal_array = np.tile(np.mean(year_stack, axis=0), (10, 1, 1)) signal_array = ma.masked_where(signal_array > 1e20, signal_array) # need to mask ### average the array over latitude(or longitude) signal_time_lon = ma.mean(signal_array, axis=1) ### do a 2D Fourier Transform of the time/space image ft = np.fft.fft2(signal_time_lon) mgft = np.abs(ft) ps = mgft**2 log_ps = np.log(mgft) log_mgft= np.log(mgft) Every second row of the ft consists completely of zeros. Why is this? Would it be acceptable to add a randomly small number to the signal to avoid this. signal_time_lon = signal_time_lon + np.random.randint(0,9,size=(730, 182))*1e-05 EDIT: Adding images and clarify meaning The output of rfft2 still appears to be a complex array. Using fftshift shifts the edges of the image to the centre; I still have a power spectrum regardless. I expect that the reason that I get rows of zeros is that I have re-created the timeseries for each pixel. The ft[0, 0] pixel contains the mean of the signal. So the ft[1, 0] corresponds to a sinusoid with one cycle over the entire signal in the rows of the starting image. Here are is the starting image using following code: plt.pcolormesh(signal_time_lon); plt.colorbar(); plt.axis('tight') Here is result using following code: ft = np.fft.rfft2(signal_time_lon) mgft = np.abs(ft) ps = mgft**2 log_ps = np.log1p(mgft) plt.pcolormesh(log_ps); plt.colorbar(); plt.axis('tight') It may not be clear in the image but it is only every second row that contains completely zeros. Every tenth pixel (log_ps[10, 0]) is a high value. The other pixels (log_ps[2, 0], log_ps[4, 0] etc) have very low values.

    Read the article

  • Polynomial fitting with log log plot

    - by viral parekh
    I have a simple problem to fit a straight line on log-log scale. My code is, data=loadtxt(filename) xdata=data[:,0] ydata=data[:,1] polycoeffs = scipy.polyfit(xdata, ydata, 1) yfit = scipy.polyval(polycoeffs, xdata) pylab.plot(xdata, ydata, 'k.') pylab.plot(xdata, yfit, 'r-') Now I need to plot fit line on log scale so I just change x and y axis, ax.set_yscale('log') ax.set_xscale('log') then its not plotting correct fit line. So how can I change fit function (in log scale) so that it can plot fit line on log-log scale? Thanks -Viral

    Read the article

  • [Gdata] GetAuthSubToken returns None

    - by Matt
    Hey guys, I am a little lost on how to get the auth token. Here is the code I am using on the return from authorizing my app: client = gdata.service.GDataService() gdata.alt.appengine.run_on_appengine(client) sessionToken = gdata.auth.extract_auth_sub_token_from_url(self.request.uri) client.UpgradeToSessionToken(sessionToken) logging.info(client.GetAuthSubToken()) what gets logged is "None" so that does seem right :-( if I use this: temp = client.upgrade_to_session_token(sessionToken) logging.info(dump(temp)) I get this: {'scopes': ['http://www.google.com/calendar/feeds/'], 'auth_header': 'AuthSub token=CNKe7drpFRDzp8uVARjD-s-wAg'} so I can see that I am getting a AuthSub Token and I guess I could just parse that and grab the token but that doesn't seem like the way things should work. If I try to use AuthSubTokenInfo I get this: Traceback (most recent call last): File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/webapp/__init__.py", line 507, in __call__ handler.get(*groups) File "controllers/indexController.py", line 47, in get logging.info(client.AuthSubTokenInfo()) File "/Users/matthusby/Dropbox/appengine/projects/FBCal/gdata/service.py", line 938, in AuthSubTokenInfo token = self.token_store.find_token(scopes[0]) TypeError: 'NoneType' object is unsubscriptable so it looks like my token_store is not getting filled in correctly, is that something I should be doing? Also I am using gdata 2.0.9 Thanks Matt

    Read the article

  • Assignment to None

    - by Joel
    Hello, I have a function which returns 3 numbers, e.g.: def numbers(): return 1,2,3 usually I call this function to receive all three returned numbers e.g.: a,b,c=numbers() However, I have one case in which I only need the first returned number. I tried using: a, None None = numbers() But I receive "SyntaxError: assignment to None". I know, of course, that i can use the first option I mentioned and then not use "b" and "c", but only "a". However, this seems like a "waste" of two vars and feels like wrong programming. Any ideas? Thanks, Joek

    Read the article

  • Group Chat XMPP with Google App Engine

    - by David Shellabarger
    Google App Engine has a great XMPP service built in. One of the few limitations it has is that it doesn't support receiving messages from a group chat. That's the one thing I want to do with it. :( Can I run a 3rd party XMPP/Jabber server on App Engine that supports group chat? If so, which one?

    Read the article

  • PDB: exception when in console - full stack trace

    - by EoghanM
    When at the pdb console, entering a statement which causes an exception results in just a single line stack trace, e.g. (Pdb) someFunc() *** TypeError: __init__() takes exactly 2 arguments (1 given) However I'd like to figure out where exactly in someFunc the error originates. i.e. in this case, which class __init__ is attached to. Is there a way to get a full stack trace in Pdb?

    Read the article

  • Django templates check condition

    - by Hulk
    If there are are no values in the table how can should the code be to indicate no name found else show the drop down box in the below code {% for name in dict.names %} <option value="{{name.id}}" {% for selected_id in selected_name %}{% ifequal name.id selected_id %} {{ selected }} {% endifequal %} {% endfor %}>{{name.firstname}}</option>{% endfor %} </select> Thanks..

    Read the article

  • How do i use repoze.who?

    - by misterwebz
    I'm having some trouble understanding how repoze.who works. I've followed a tutorial i found by searching on google and here's what i already have: This is what i added in my middleware.py file from repoze.who.config import make_middleware_with_config as make_who_with_config app = make_who_with_config(app, global_conf, app_conf['who.config_file'], app_conf['who.log_file'], app_conf['who.log_level']) Here's the who.ini : http://pastebin.com/w5Tba2Fp Here's repoze_auth.py in /lib/auth/: from paste.httpexceptions import HTTPFound from iwant.model import User class UserModelPlugin(object): def authenticate(self, environ, identity): try: username = identity['login'] password = identity['password'] except KeyError: return None success = User.authenticate(username, password) return success def add_metadata(self, environ, identity): username = identity.get('repoze.who.userid') user = User.get(username) if user is not None: identity['user'] = user I've also checked the plugins in the repoze.who folder, but i failed to understand how it's supposed to be used. I'd appreciate it if someone would push me in the right direction.

    Read the article

  • Convert or strip out "illegal" Unicode characters

    - by Oli
    I've got a database in MSSQL that I'm porting to SQLite/Django. I'm using pymssql to connect to the database and save a text field to the local SQLite database. However for some characters, it explodes. I get complaints like this: UnicodeDecodeError: 'ascii' codec can't decode byte 0x97 in position 1916: ordinal not in range(128) Is there some way I can convert the chars to proper unicode versions? Or strip them out?

    Read the article

  • JSON serialization of Google App Engine models

    - by user111677
    I've been search for quite a while with no success. My project isn't using Django, is there a simple way to serialize App Engine models (google.appengine.ext.db.Model) into JSON or do I need to write my own serializer? My model class is fairly simple. For instance: class Photo(db.Model): filename = db.StringProperty() title = db.StringProperty() description = db.StringProperty(multiline=True) date_taken = db.DateTimeProperty() date_uploaded = db.DateTimeProperty(auto_now_add=True) album = db.ReferenceProperty(Album, collection_name='photo') Thanks in advance.

    Read the article

  • Good way to edit the previous defined class in ipython

    - by leo
    Hi, I am wondering a good way to follow if i would like to redefine the members of a previous defined class in ipython. say : I have defined a class intro like below, and later i want to redefine part of the function definition _print_api. Any way to do that without retyping it . class intro(object): def _print_api(self,obj): def _print(key): if key.startswith('_'): return '' value = getattr(obj,key) if not hasattr(value,im_func): doc = type(valuee).__name__ else: if value.__doc__ is None: doc = 'no docstring' else: doc = value.__doc__ return ' %s :%s' %(key,doc) res = [_print(element) for element in dir(obj)] return '\n'.join([element for element in res if element != '']) def __get__(self,instance,klass): if instance is not None: return self._print(instance) else: return self._print_api(klass)

    Read the article

  • Server authorization with MD5 and SQL.

    - by Charles
    I currently have a SQL database of passwords stored in MD5. The server needs to generate a unique key, then sends to the client. In the client, it will use the key as a salt then hash together with the password and send back to the server. The only problem is that the the SQL DB has the passwords in MD5 already. Therefore for this to work, I would have to MD5 the password client side, then MD5 it again with the salt. Am I doing this wrong, because it doesn't seem like a proper solution. Any information is appreciated.

    Read the article

  • How to get "paster request" to use config host value instead of localhost?

    - by mmartinez
    I'm trying to access my pylons application via cron job to send notifications to my users. The way I'm doing this is by running the application using something like: paster request myconfig.ini /maintenance/do In the actual controller I check for the "paste.command_request" to block public access. Everything works but the only problem is that within the notifications that I send to my users there is a link to their profile and the host is "localhost" which should instead be the domain name of the application. When the notifications are sent from within the served application (say, a user modifies their settings on the site) the notifications have the correct url. I am using mako to render my email tamplates and within the template I am using the "pylons.url" method with "qualified" set to "True". Am I missing something here? Thanks in advance.

    Read the article

  • How to run unittest for Django?

    - by photon
    I configured properties for my django project under pydev. I can run the django app under pydev or under console window. But I have problems to run unittest under pydev. I cannot run unittest for app under console window either. I guessed it's something related to run configurations of pydev, so I made several trials, but with no success. Once I got messages like this: ImportError: Could not import settings 'D:\django_projects\MyProject' (Is it on sys.path? Does it have syntax errors?): No module named D:\django_projects\MyProject ERROR: Module: MyUnittestFile could not be imported. Another time I got messages like this: ImportError: Could not import settings 'MyProject.settngs' (Is it on sys.path? Does it have syntax errors?): No module named settngs 'ERROR: Module: MyUnittestFile could not be imported. I use pydev 1.5.6 on eclipse and windows xp. Any ideas for this problem? Now I think it's not something related to pydev, thanks for Xavier Ho's suggestion.

    Read the article

  • Inside a decorator-class, access instance of the class which contains the decorated method

    - by ifischer
    I have the following decorator, which saves a configuration file after a method decorated with @saveconfig is called: class saveconfig(object): def __init__(self, f): self.f = f def __call__(self, *args): self.f(object, *args) # Here i want to access "cfg" defined in pbtools print "Saving configuration" I'm using this decorator inside the following class. After the method createkvm is called, the configuration object self.cfg should be saved inside the decorator: class pbtools() def __init__(self): self.configfile = open("pbt.properties", 'r+') # This variable should be available inside my decorator self.cfg = ConfigObj(infile = self.configfile) @saveconfig def createkvm(self): print "creating kvm" My problem is that i need to access the object variable self.cfg inside the decorator saveconfig. A first naive approach was to add a parameter to the decorator which holds the object, like @saveconfig(self), but this doesn't work. How can I access object variables of the method host inside the decorator? Do i have to define the decorator inside the same class to get access?

    Read the article

  • How to get debugging of an App Engine application working?

    - by Chris Lacy
    I've got 10+ years in C/C++, and it appears Visual Studio has spoilt me during that time. In Visual Studio, debbuging issimple: I just add a breakpoint to a line of code, and as soon as that code is executed, my breakpoint triggers, at which point I can view a callstack, local/member variables, etc. I'm trying to achieve this functionality under App Engine. I assume that is possible? All the searching I've done to this point has led me to using Pydev in Eclipse. As best I can tell, I am successfully launching my simple 'hello world' program in Debug mode. But the IDE doesn't even seem to have an option to set a breakpoint? I must be missing something. I've googled long and hard about this, but am having no luck. Most results trace back to the same old threads that don't deal directly with my issue. Can anyone shed some light on how you get basic debugging setup using Pydev/Eclipse with App Engine? Alternatively, if there's an easier way to debug App Engine than using Pydev/Eclipse, I'd love to hear about it. Thanks in advance.

    Read the article

  • Fixing color in scatter plots in matplotlib

    - by ajhall
    Hi guys, I'm going to have to come back and add some examples if you need them, which you might. But, here's the skinny- I'm plotting scatter plots of lab data for my research. I need to be able to visually compare the scatter plots from one plot to the next, so I want to fix the color range on the scatter plots and add in a colorbar to each plot (which will be the same in each figure). Essentially, I'm fixing all aspects of the axes and colorspace etc. so that the plots are directly comparable by eye. For the life of me, I can't seem to get my scatter() command to properly set the color limits in the colorspace (default)... i.e., I figure out my total data's min and total data's max, then apply them to vmin, vmax, for the subset of data, and the color still does not come out properly in both plots. This must come up here and there, I can't be the only one that wants to compare various subsets of data amongst plots... so, how do you fix the colors so that each data keeps it's color between plots and doesn't get remapped to a different color due to the change in max/min of the subset -v- the whole set? I greatly appreciate all your thoughts!!! A mountain-dew and fiery-hot cheetos to all! -Allen

    Read the article

  • Reliable and fast way to convert a zillion ODT files in PDF?

    - by Marco Mariani
    I need to pre-produce a million or two PDF files from a simple template (a few pages and tables) with embedded fonts. Usually, I would stay low level in a case like this, and compose everything with a library like ReportLab, but I joined late in the project. Currently, I have a template.odt and use markers in the content.xml files to fill with data from a DB. I can smoothly create the ODT files, they always look rigth. For the ODT to PDF conversion, I'm using openoffice in server mode (and PyODConverter w/ named pipe), but it's not very reliable: in a batch of documents, there is eventually a point after which all the processed files are converted into garbage (wrong fonts and letters sprawled all over the page). Problem is not predictably reproducible (does not depend on the data), happens in OOo 2.3 and 3.2, in Ubuntu, XP, Server 2003 and Windows 7. My Heisenbug detector is ticking. I tried to reduce the size of batches and restarting OOo after each one; still, a small percentage of the documents are messed up. Of course I'll write about this on the Ooo mailing lists, but in the meanwhile, I have a delivery and lost too much time already. Where do I go? Completely avoid the ODT format and go for another template system. Suggestions? Anything that takes a few seconds to run is way too slow. OOo takes around a second and it sums to 15 days of processing time. I had to write a program for clustering the jobs over several clients. Keep the format but go for another tool/program for the conversion. Which one? There are many apps in the shareware or commercial repositories for windows, but trying each one is a daunting task. Some are too slow, some cannot be run in batch without buying it first, some cannot work from command line, etc. Open source tools tend not to reinvent the wheel and often depend on openoffice. Converting to an intermediate .DOC format could help to avoid the OOo bug, but it would double the processing time and complicate a task that is already too hairy. Try to produce the PDFs twice and compare them, discarding the whole batch if there's something wrong. Although the documents look equal, I know of no way to compare the binary content. Restart OOo after processing each document. it would take a lot more time to produce them it would lower the percentage of the wrong files, and make it very hard to identify them. Go for ReportLab and recreate the pages programmatically. This is the approach I'm going to try in a few minutes. Learn to properly format bulleted lists Thanks a lot.

    Read the article

  • How to call Twiter's Streaming/Filter Feed with urllib2/httplib?

    - by Simon
    Update: I switched this back from answered as I tried the solution posed in cogent Nick's answer and switched to Google's urlfetch: logging.debug("starting urlfetch for http://%s%s" % (self.host, self.url)) result = urlfetch.fetch("http://%s%s" % (self.host, self.url), payload=self.body, method="POST", headers=self.headers, allow_truncated=True, deadline=5) logging.debug("finished urlfetch") but unfortunately finished urlfetch is never printed - I see the timeout happen in the logs (it returns 200 after 5 seconds), but execution doesn't seem tor return. Hi All- I'm attempting to play around with Twitter's Streaming (aka firehose) API with Google App Engine (I'm aware this probably isn't a great long term play as you can't keep the connection perpetually open with GAE), but so far I haven't had any luck getting my program to actually parse the results returned by Twitter. Some code: logging.debug("firing up urllib2") req = urllib2.Request(url="http://%s%s" % (self.host, self.url), data=self.body, headers=self.headers) logging.debug("called urlopen for %s %s, about to call urlopen" % (self.host, self.url)) fobj = urllib2.urlopen(req) logging.debug("called urlopen") When this executes, unfortunately, my debug output never shows the called urlopen line printed. I suspect what's happening is that Twitter keeps the connection open and urllib2 doesn't return because the server doesn't terminate the connection. Wireshark shows the request being sent properly and a response returned with results. I tried adding Connection: close to my request header, but that didn't yield a successful result. Any ideas on how to get this to work? thanks -Simon

    Read the article

  • Displaying Google Calendar event data on FullCalendar

    - by aurealus
    I am using Google Calendar as a storage engine for a calendar system I am building, however, I am using a single Google user account with multiple calendars, i.e. each user on my system has their own calendar within the one user account. I'm able to create a calendar per user just fine, but I would like to have FullCalendar retrieve the events for display purposes, without manually getting the magic cookie url from Google Calendar settings. I would like to be able to retrieve it programmatically or 'proxy' the feed via an authenticated call to get event data that I'm doing in Django. $('#calendar').fullCalendar({ events: $.fullCalendar.gcalFeed( "http://www.google.com/calendar_url/" <-- or /my/event/feed/url ) });

    Read the article

< Previous Page | 383 384 385 386 387 388 389 390 391 392 393 394  | Next Page >