Search Results

Search found 16680 results on 668 pages for 'python datetime'.

Page 431/668 | < Previous Page | 427 428 429 430 431 432 433 434 435 436 437 438  | Next Page >

  • 50 million+ Rows of Data - CSV or MySQL

    - by eWizardII
    Hello, I have a CSV file which is about 1GB big and contains about 50million rows of data, I am wondering is it better to keep it as a CSV file or store it as some form of a database. I don't know a great deal about MySQL to argue for why I should use it or another database framework over just keeping it as a CSV file. I am basically doing a Breadth-First Search with this dataset, so once I get the initial "seed" set the 50million I use this as the first values in my queue. Thanks,

    Read the article

  • prints line number in both txtfile and list????

    - by jad
    i have this code which prints the line number in infile but also the linenumber in words what do i do to only print the line number of the txt file next to the words??? d = {} counter = 0 wrongwords = [] for line in infile: infile = line.split() wrongwords.extend(infile) counter += 1 for word in infile: if word not in d: d[word] = [counter] if word in d: d[word].append(counter) for stuff in wrongwords: print(stuff, d[stuff]) the output is : hello [1, 2, 7, 9] # this is printing the linenumber of the txt file hello [1] # this is printing the linenumber of the list words hello [1] what i want is: hello [1, 2, 7, 9]

    Read the article

  • Is it possible to bulk load an NDB child Entity in GAE?

    - by hmacread
    At some point in the future I may need to bulk load migration data (i.e. from a CSV). Has anyone had exceptions raised doing the following? Also is there any change in behaviour if the ndb.put_multi() function is used? from google.appengine.ext import ndb while True: if not id: break id, name = read_csv_row(readline()) x = X(parent=ndb.Key('Y','static_id') x.id, x.name = id, name x.put() class X(ndb.Model): id = StringProperty() name = StringProperty() class Y(ndb.Model): pass def read_csv_row(line): """returns tuple"""

    Read the article

  • variable being weirdly deleted

    - by calccrypto
    im having a weird problem with one variable: its not being recognized but its still printing. i would post my code, but it is massive. the basic idea is: # pseudocode def function(stuff): <do stuff> # These are the only 2 conditions if tag == 3: pka = <a string> if tag == 4: pka = <a string> print pka # (1) print pka # (2) <do stuff not modifying pka> print pka # (3) if pka == 'RSA': <do stuff> elif pka == 'DSA': <do stuff> my code will error at (2). however, it will print out (1), (2), and (3), all of which are the same. is there any general explanation of why this is happening? if my code is really needed, i will post it, but otherwise, i would rather not due to its size update: now the code will error at the if statement after (3), saying UnboundLocalError: local variable 'pka' referenced before assignment even though (1),(2),(3) just printed

    Read the article

  • Easy way to keep counting up infinitely

    - by Andrew Alexander
    What's a good way to keep counting up infinitely? I'm trying to write a condition that will keep going until there's no value in a database, so it's going to iterate from 0, up to theoretically infinity (inside a try block, of course). How would I count upwards infinitely? Or should I use something else? I am looking for something similar to i++ in other languages, where it keeps iterating until failure.

    Read the article

  • How can I implement "real time" messaging on Google AppEngine?

    - by Freed
    I'm creating a web application on Google AppEngine where I want the user to be notified a quickly as possible after certain events occour. The problem is similar to say a chat server in that I need something happening on one connection (someone is writing a message in a chat room) to propagate to a number of other connections (other people in that chat room gets the message). To get speedy updates from the server to the client I'm planning on using long polling with XmlHttpRequest, hoping that AppEngine won't interfere other than possibly restriing the timeout. The real problem however is efficient notification between connections on AppEngine. Is there any support for this type of cross connection notification on AppEngine that does not involve busy-waiting? The only tools I can think of to do this at all is either using the data storage (slow) or memcache (unreliable), and none of them would let me avoid busy-waiting. Note: I know about XMPP support on AppEngine. It's related, but I want a browser based solution, sending messages to the users by XMPP is not an option.

    Read the article

  • Solution for distributing MANY simple network tasks?

    - by EmpireJones
    I would like to create some sort of a distributed setup for running a ton of small/simple REST web queries in a production environment. For each 5-10 related queries which are executed from a node, I will generate a very small amount of derived data, which will need to be stored in a standard relational database (such as PostgreSQL). What platforms are built for this type of problem set? The nature, data sizes, and quantities seem to contradict the mindset of Hadoop. There are also more grid based architectures such as Condor and Sun Grid Engine, which I have seen mentioned. I'm not sure if these platforms have any recovery from errors though (checking if a job succeeds). What I would really like is a FIFO type queue that I could add jobs to, with the end result of my database getting updated. Any suggestions on the best tool for the job?

    Read the article

  • How to retrieve items from a django queryset?

    - by sharataka
    I'm trying to get the video element in a queryset but am having trouble retrieving it. user_channel = Everything.objects.filter(profile = request.user, playlist = 'Channel') print user_channel[0] #returns the first result without error print user_channel[0]['video'] #returns error Models.py: class Everything(models.Model): profile = models.ForeignKey(User) playlist = models.CharField('Playlist', max_length = 2000, null=True, blank=True) platform = models.CharField('Platform', max_length = 2000, null=True, blank=True) video = models.CharField('VideoID', max_length = 2000, null=True, blank=True) video_title = models.CharField('Title of Video', max_length = 2000, null=True, blank=True) def __unicode__(self): return u'%s %s %s %s %s' % (self.profile, self.playlist, self.platform, self.video, self.video_title)

    Read the article

  • Can't iterate over nestled dict in django

    - by fredrik
    Hi, Im trying to iterate over a nestled dict list. The first level works fine. But the second level is treated like a string not dict. In my template I have this: {% for product in Products %} <li> <p>{{ product }}</p> {% for partType in product.parts %} <p>{{ partType }}</p> {% for part in partType %} <p>{{ part }}</p> {% endfor %} {% endfor %} </li> {% endfor %} It's the {{ part }} that just list 1 char at the time based on partType. And it seams that it's treated like a string. I can however via dot notation reach all dict but not with a for loop. The current output looks like this: Color C o l o r Style S ..... The Products object looks like this in the log: [{'product': <models.Products.Product object at 0x1076ac9d0>, 'parts': {u'Color': {'default': u'Red', 'optional': [u'Red', u'Blue']}, u'Style': {'default': u'Nice', 'optional': [u'Nice']}, u'Size': {'default': u'8', 'optional': [u'8', u'8.5']}}}] What I trying to do is to pair together a dict/list for a product from a number of different SQL queries. The web handler looks like this: typeData = Products.ProductPartTypes.all() productData = Products.Product.all() langCode = 'en' productList = [] for product in productData: typeDict = {} productDict = {} for type in typeData: typeDict[type.typeId] = { 'default' : '', 'optional' : [] } productDict['product'] = product productDict['parts'] = typeDict defaultPartsData = Products.ProductParts.gql('WHERE __key__ IN :key', key = product.defaultParts) optionalPartsData = Products.ProductParts.gql('WHERE __key__ IN :key', key = product.optionalParts) for defaultPart in defaultPartsData: label = Products.ProductPartLabels.gql('WHERE __key__ IN :key AND partLangCode = :langCode', key = defaultPart.partLabelList, langCode = langCode).get() productDict['parts'][defaultPart.type.typeId]['default'] = label.partLangLabel for optionalPart in optionalPartsData: label = Products.ProductPartLabels.gql('WHERE __key__ IN :key AND partLangCode = :langCode', key = optionalPart.partLabelList, langCode = langCode).get() productDict['parts'][optionalPart.type.typeId]['optional'].append(label.partLangLabel) productList.append(productDict) logging.info(productList) templateData = { 'Languages' : Settings.Languges.all().order('langCode'), 'ProductPartTypes' : typeData, 'Products' : productList } I've tried making the dict in a number of different ways. Like first making a list, then a dict, used tulpes anything I could think of. Any help is welcome! Bouns: If someone have an other approach to the SQL quires, that is more then welcome. I feel that it kinda stupid to run that amount of quires. What is happening that each product part has a different label base on langCode. ..fredrik

    Read the article

  • Django: automatically import MEDIA_URL in context

    - by pistacchio
    Hi, like exposed here, one can set a MEDIA_URL in settings.py (for example i'm pointing to Amazon S3) and serve the files in the view via {{ MEDIA_URL }}. Since MEDIA_URL is not automatically in the context, one have to manually add it to the context, so, for example, the following works: #views.py from django.shortcuts import render_to_response from django.template import RequestContext def test(request): return render_to_response('test.html', {}, context_instance=RequestContext(request)) This means that in each view.py file i have to add from django.template import RequestContext and in each response i have to explicitly specify context_instance=RequestContext(request). Is there a way to automatically (DRY) add MEDIA_URL to the default context? Thanks in advance.

    Read the article

  • Setting opacity on a PyGTK label

    - by snostorm
    Is there a way to make a PyGTK widget partly transparent, so that the widgets behind it can be seen through it? Specifically I'm trying to do this on a label, for typographic effect; I don't want to change the color instead, as it may not look right on all themes.

    Read the article

  • Matplotlib not showing up in Mac OSX

    - by Werner
    Hi, I am running Mac OSX 10.5.8. I installed matplotlib using macports. I get some examples from the matplotlib gallery like this one, without modification: http://matplotlib.sourceforge.net/examples/api/unicode_minus.html I run it, get no error, but the picture does not show up. In Linux Ubuntu I get it. Do you know what could be wrong here? Thanks

    Read the article

  • Efficient way to combine results of two database queries.

    - by ensnare
    I have two tables on different servers, and I'd like some help finding an efficient way to combine and match the datasets. Here's an example: From server 1, which holds our stories, I perform a query like: query = """SELECT author_id, title, text FROM stories ORDER BY timestamp_created DESC LIMIT 10 """ results = DB.getAll(query) for i in range(len(results)): #Build a string of author_ids, e.g. '1314,4134,2624,2342' But, I'd like to fetch some info about each author_id from server 2: query = """SELECT id, avatar_url FROM members WHERE id IN (%s) """ values = (uid_list) results = DB.getAll(query, values) Now I need some way to combine these two queries so I have a dict that has the story as well as avatar_url and member_id. If this data were on one server, it would be a simple join that would look like: SELECT * FROM members, stories WHERE members.id = stories.author_id But since we store the data on multiple servers, this is not possible. What is the most efficient way to do this? Thanks.

    Read the article

  • removing elements incrementally from a list

    - by Javier
    Dear all, I've a list of float numbers and I would like to delete incrementally a set of elements in a given range of indexes, sth. like: for j in range(beginIndex, endIndex+1): print ("remove [%d] => val: %g" % (j, myList[j])) del myList[j] However, since I'm iterating over the same list, the indexes (range) are not valid any more for the new list. Does anybody has some suggestions on how to delete the elements properly? Best wishes

    Read the article

< Previous Page | 427 428 429 430 431 432 433 434 435 436 437 438  | Next Page >