Search Results

Search found 34110 results on 1365 pages for 'gdata python client'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 492/1365 | < Previous Page | 488 489 490 491 492 493 494 495 496 497 498 499  | Next Page >

  • Scraping paginated items from a website using scrapy

    - by Mridang Agarwalla
    I'm using scrapy to scrape items from a site. I'm not being able to implement this scraping pattern. The site I'm trying to scrape is a forum and I scrape the site once a day. Each page has a table containing posts. New posts are added to the top of the table and as more and more posts are posted to the site, the older posts go further into the pages due to pagination. This is a very simple scenario and we will assume that the order of the posts never change. I would like to scrape this site and scrape all the "new" records until the last scraped post from yesterday is encountered. I have configured my spider to paginate endlessly and when it encounters yesterday's last scraped post, it should stop. How can implement this? (My Scrapy installation works with my Django installation using django-dynamic-scraper )

    Read the article

  • How can I detect whether an image is a PNG or APNG format?

    - by perlit
    APNG is backwards compatible with PNG. I opened up an apng and png file in a hex editor and the first few bytes look identical. So if a user uploads either of these formats, how do I detect what the format really is? I've seen this done on some sites that block apng. I'm guessing the ImageMagick library makes this easy, but what if I were to do the detect without the use of an image processing library (for learning purposes)? Can I look for specific bytes that tell me if the file is apng? Solutions in any language is welcome.

    Read the article

  • how to pass in dynamic data to decorators

    - by steve
    Hi, I am trying to write a base crud controller class that does the following: class BaseCrudController: model = "" field_validation = {} template_dir = "" @expose(self.template_dir) def new(self, *args, **kwargs) .... @validate(self.field_validation, error_handler=new) @expose() def post(self, *args, **kwargs): ... My intent is to have my controllers extend this base class, set the model, field_validation, and template locations, and am ready to go. Unfortunately, decorators (to my understanding), are interpreted when the function is defined. Hence it won't have access to instance's value. Is there a way to pass in dynamic data or values from the sub class? If not, I guess I could use override_template as a workaround to expose and set the template within the controller action. How would I go about validating the form within the controller action? Thanks, Steve

    Read the article

  • SQLAlchemy - select for update example

    - by Mark
    I'm looking for a complete example of using select for update in SQLAlchemy, but haven't found one googling. I need to lock a single row and update a column, the following code doesn't work (blocks forever): s = table.select(table.c.user=="test",for_update=True) u = table.update().where(table.c.user=="test") u.execute(email="foo") Do I need a commit? How do I do that? As far as I know you need to: begin transaction select ... for update update commit

    Read the article

  • Prepopulate drop-box according to another drop-box choice in Django Admin

    - by onorua
    I have models like this: class User(models.Model): Switch = models.ForeignKey(Switch, related_name='SwitchUsers') Port = models.ForeignKey(Port) class Switch(models.Model): Name = models.CharField(max_length=50) class Port(models.Model): PortNum = models.PositiveIntegerField() Switch = models.ForeignKey(Switch, related_name = "Ports") When I'm in Admin interface and choose Switch from Switches available, I would like to have Port prepopulated accordingly with Ports from the related Switch. As far as I understand I need to create some JS script to prepopulate it. Unfortunately I don't have this experience, and I would like to keep things simple as it possible and don't rewrite all Django admin interface. Just add this functionality for one Field. Could you please help me with my problem? Thank you.

    Read the article

  • How to replace empty string with zero in comma-separated string?

    - by dsaccount1
    "8,5,,1,4,7,,,,7,,1,9,3,6,,,8,6,3,9,,2,5,4,,,,,3,2,,,7,4,1,1,,4,,6,9,,5,,,,5,,,1,,6,3,,,6,5,,,,7,4,,1,7,6,,,,8,,5,,,7,1,,3,9," I'm doing a programming challenge where i need to parse this sequence into my sudoku script. Need to get the above sequence into 8,5,0,1,4,7,0,0,0,7,0,1,9,3,6,0,0,8......... I tried re but without success, help is appreciated, thanks.

    Read the article

  • strange chi-square result using scikit_learn with feature matrix

    - by user963386
    I am using scikit learn to calculate the basic chi-square statistics(sklearn.feature_selection.chi2(X, y)): def chi_square(feat,target): """ """ from sklearn.feature_selection import chi2 ch,pval = chi2(feat,target) return ch,pval chisq,p = chi_square(feat_mat,target_sc) print(chisq) print("**********************") print(p) I have 1500 samples,45 features,4 classes. The input is a feature matrix with 1500x45 and a target array with 1500 components. The feature matrix is not sparse. When I run the program and I print the arrray "chisq" with 45 components, I can see that the component 13 has a negative value and p = 1. How is it possible? Or what does it mean or what is the big mistake that I am doing? I am attaching the printouts of chisq and p: [ 9.17099260e-01 3.77439701e+00 5.35004211e+01 2.17843312e+03 4.27047184e+04 2.23204883e+01 6.49985540e-01 2.02132664e-01 1.57324454e-03 2.16322638e-01 1.85592258e+00 5.70455805e+00 1.34911126e-02 -1.71834753e+01 1.05112366e+00 3.07383691e-01 5.55694752e-02 7.52801686e-01 9.74807972e-01 9.30619466e-02 4.52669897e-02 1.08348058e-01 9.88146259e-03 2.26292358e-01 5.08579194e-02 4.46232554e-02 1.22740419e-02 6.84545170e-02 6.71339545e-03 1.33252061e-02 1.69296016e-02 3.81318236e-02 4.74945604e-02 1.59313146e-01 9.73037448e-03 9.95771327e-03 6.93777954e-02 3.87738690e-02 1.53693158e-01 9.24603716e-04 1.22473138e-01 2.73347277e-01 1.69060817e-02 1.10868365e-02 8.62029628e+00] ********************** [ 8.21299526e-01 2.86878266e-01 1.43400668e-11 0.00000000e+00 0.00000000e+00 5.59436980e-05 8.84899894e-01 9.77244281e-01 9.99983411e-01 9.74912223e-01 6.02841813e-01 1.26903019e-01 9.99584918e-01 1.00000000e+00 7.88884155e-01 9.58633878e-01 9.96573548e-01 8.60719653e-01 8.07347364e-01 9.92656816e-01 9.97473024e-01 9.90817144e-01 9.99739526e-01 9.73237195e-01 9.96995722e-01 9.97526259e-01 9.99639669e-01 9.95333185e-01 9.99853998e-01 9.99592531e-01 9.99417113e-01 9.98042114e-01 9.97286030e-01 9.83873717e-01 9.99745466e-01 9.99736512e-01 9.95239765e-01 9.97992843e-01 9.84693908e-01 9.99992525e-01 9.89010468e-01 9.64960636e-01 9.99418323e-01 9.99690553e-01 3.47893682e-02]

    Read the article

  • Django: Summing values

    - by Anry
    I have a two Model - Project and Cost. class Project(models.Model): title = models.CharField(max_length=150) url = models.URLField() manager = models.ForeignKey(User) class Cost(models.Model): project = models.ForeignKey(Project) cost = models.FloatField() date = models.DateField() I must return the sum of costs for each project. view.py: from mypm.costs.models import Project, Cost from django.shortcuts import render_to_response from django.db.models import Avg, Sum def index(request): #... return render_to_response('index.html',... How?

    Read the article

  • Loading url with cyrillic symbols

    - by Ockonal
    Hi guys, I have to load some url with cyrillic symbols. My script should work with this: http://wincode.org/%D0%BF%D1%80%D0%BE%D0%B3%D1%80%D0%B0%D0%BC%D0%BC%D0%B8%D1%80%D0%BE%D0%B2%D0%B0%D0%BD%D0%B8%D0%B5/ If I'll use this in browser it would replaced into normal symbols, but urllib code fails with 404 error. How to decode correctly this url? When I'm using that url directly in code, like address = 'that address', it works perfect. But I used parsing page for getting this url. I have a list of urls which contents cyrillic. Maybe they have uncorrect encoding? Here is more code: requestData = urllib2.Request( %SOME_ADDRESS%, None, {"User-Agent": user_agent}) requestHandler = pageHandler.open(requestData) pageData = requestHandler.read().decode('utf-8') soupHandler = BeautifulSoup(pageData) topicLinks = [] for postBlock in soupHandler.findAll('a', href=re.compile('%SOME_REGEXP%')): topicLinks.append(postBlock['href']) postAddress = choice(topicLinks) postRequestData = urllib2.Request(postAddress, None, {"User-Agent": user_agent}) postHandler = pageHandler.open(postRequestData) postData = postHandler.read() File "/usr/lib/python2.6/urllib2.py", line 518, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 404: Not Found

    Read the article

  • I am currently serving my static files in Django. How do I use Apache2 to do this?

    - by alex
    (r'^media/(?P<path>.*)$', 'django.views.static.serve',{'document_root': settings.MEDIA_ROOT}), As you can see, I have a directory called "media" under my Django project. I would like to delete this line in my urls.py and instead us Apache to serve my static files. What do I do to my Apache configs (which files do I change) in order to do this? By the way, I installed Apache2 like normal: sudo aptitude install apache2

    Read the article

  • SelfReferenceProperty vs. ListProperty Google App Engine

    - by John
    Hi All, I am experimenting with the Google App Engine and have a question. For the sake of simplicity, let's say my app is modeling a computer network (a fairly large corporate network with 10,000 nodes). I am trying to model my Node class as follows: class Node(db.Model): name = db.StringProperty() neighbors = db.SelfReferenceProperty() Let's suppose, for a minute, that I cannot use a ListProperty(). Based on my experiments to date, I can assign only a single entity to 'neighbors' - and I cannot use the "virtual" collection (node_set) to access the list of Node neighbors. So... my questions are: Does SelfReferenceProperty limit you to a single entity that you can reference? If I instead use a ListProperty, I believe I am limited to 5,000 keys, which I need to exceed. Thoughts? Thanks, John

    Read the article

  • the error "invalid literal for int() with base 10:" keeps coming up

    - by ratce003
    I'm trying to write a very simple program, I want to print out the sum of all the multiples of 3 and 5 below 100, but, an error keeps accuring, saying "invalid literal for int() with base 10:" my program is as follows: sum = "" sum_int = int(sum) for i in range(1, 101): if i % 5 == 0: sum += i elif i % 3 == 0: sum += i else: sum += "" print sum Any help would be much appreciated.

    Read the article

  • Is using os.path.abspath to validate an untrusted filename's location secure?

    - by mcmt
    I don't think I'm missing anything. Then again I'm kind of a newbie. def GET(self, filename): name = urllib.unquote(filename) full = path.abspath(path.join(STATIC_PATH, filename)) #Make sure request is not tricksy and tries to get out of #the directory, e.g. filename = "../.ssh/id_rsa". GET OUTTA HERE assert full[:len(STATIC_PATH)] == STATIC_PATH, "bad path" return open(full).read() Edit: I realize this will return the wrong HTTP error code if the file doesn't exist (at least under web.py). I will fix this.

    Read the article

  • Why wont numpy matrix let me print its rows?

    - by uberjumper
    Okay this is probably a really dumb question, however its really starting to hurt. I have a numpy matrix, and basically i print it out row by row. However i want to make each row be formatted and separated properly. >>> arr = numpy.matrix([[x for x in range(5)] for y in range(5)]) >>> arr matrix([[0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4], [0, 1, 2, 3, 4]]) Lets say i want to print the first row, and add a '|' between each element: >>> '|'.join(map(str, arr[0,])) '[[0 1 2 3 4]]' Err... >>> '|'.join(map(lambda x: str(x[0]), arr[0])) '[[0 1 2 3 4]]' I am really confused by this behavior why does it do this?

    Read the article

  • Twisted - how to create multi protocol process and send the data between the protocols

    - by SpankMe
    Hey, Im trying to write a program that would be listening for data (simple text messages) on some port (say tcp 6666) and then pass them to one or more different protocols - irc, xmpp and so on. I've tried many approaches and digged the Internet, but I cant find easy and working solution for such task. The code I am currently fighting with is here: http://pastebin.com/ri7caXih I would like to know how to from object like: ircf = ircFactory('asdfasdf', '#asdf666') get access to self protocol methods, because this: self.protocol.dupa1(msg) returns error about self not being passed to active protocol object. Or maybe there is other, better, easier and more kosher way to create single reactor with multiple protocols and have actions triggeres when a message arrives on any of them, and then pass that message to other protocols for handling/processing/sending? Any help will be highly appreciated!

    Read the article

  • SQLAlchemy - SQLite for testing and Postgresql for devlopment - How to port?

    - by StackUnderflow
    I want to use sqlite memory database for all my testing and Postgresql for my development/production server. But the SQL syntax is not same in both dbs. for ex: SQLite has autoincrement, and Postgresql has serial Is it easy to port the SQL script from sqlite to postgresql... what are your solutions? If you want me to use standard SQL, how should I go about generating primary key in both the databases?

    Read the article

  • How do I use Django to insert a Geometry Field into the database?

    - by alex
    class LocationLog(models.Model): user = models.ForeignKey(User) utm = models.GeometryField(spatial_index=True) This is my database model. I would like to insert a row. I want to insert a circle at point -55, 333. With a radius of 10. How can I put this circle into the geometry field? Of course, then I would want to check which circles overlap a given circle. (my select statement)

    Read the article

  • Find subset with K elements that are closest to eachother

    - by Nima
    Given an array of integers size N, how can you efficiently find a subset of size K with elements that are closest to each other? Let the closeness for a subset (x1,x2,x3,..xk) be defined as: 2 <= N <= 10^5 2 <= K <= N constraints: Array may contain duplicates and is not guaranteed to be sorted. My brute force solution is very slow for large N, and it doesn't check if there's more than 1 solution: N = input() K = input() assert 2 <= N <= 10**5 assert 2 <= K <= N a = [] for i in xrange(0, N): a.append(input()) a.sort() minimum = sys.maxint startindex = 0 for i in xrange(0,N-K+1): last = i + K tmp = 0 for j in xrange(i, last): for l in xrange(j+1, last): tmp += abs(a[j]-a[l]) if(tmp > minimum): break if(tmp < minimum): minimum = tmp startindex = i #end index = startindex + K? Examples: N = 7 K = 3 array = [10,100,300,200,1000,20,30] result = [10,20,30] N = 10 K = 4 array = [1,2,3,4,10,20,30,40,100,200] result = [1,2,3,4]

    Read the article

  • Project design / FS layout for large django projects

    - by rcreswick
    What is the best way to layout a large django project? The tutuorials provide simple instructions for setting up apps, models, and views, but there is less information about how apps and projects should be broken down, how much sharing is allowable/necessary between apps in a typical project (obviously that is largely dependent on the project) and how/where general templates should be kept. Does anyone have examples, suggestions, and explanations as to why a certain project layout is better than another? I am particularly interested in the incorporation of large numbers of unit tests (2-5x the size of the actual code base) and string externalization / templates.

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 488 489 490 491 492 493 494 495 496 497 498 499  | Next Page >