Search Results

Search found 144 results on 6 pages for 'urlopen'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 1/6 | 1 2 3 4 5 6  | Next Page >

  • Urllib's urlopen broken on some sites (StackApps api)

    - by Edan Maor
    I'm using urllib2's urlopen function to try and get a JSON result from the StackOverflow api. The code I'm using: >>> import urllib2 >>> conn = urllib2.urlopen("http://api.stackoverflow.com/0.8/users/") >>> conn.readline() The result I'm getting: '\x1f\x8b\x08\x00\x00\x00\x00\x00\x04\x00\xed\xbd\x07`\x1cI\x96%&/m\xca{\x7fJ\... I'm fairly new to urllib, but this doesn't seem like the result I should be getting. I've tried it in other places and I get what I expect (the same as visiting the address with a browser gives me: a JSON object). Using urlopen on other sites (e.g. "http://google.com") works fine, and gives me actual html. I've also tried using urllib and it gives the same result. I'm pretty stuck, not even knowing where to look to solve this problem. Any ideas?

    Read the article

  • Urllib's urlopen breaking on some sites (e.g. StackApps api)

    - by Edan Maor
    I'm using urllib2's urlopen function to try and get a JSON result from the StackOverflow api. The code I'm using: >>> import urllib2 >>> conn = urllib2.urlopen("http://api.stackoverflow.com/0.8/users/") >>> conn.readline() The result I'm getting: '\x1f\x8b\x08\x00\x00\x00\x00\x00\x04\x00\xed\xbd\x07`\x1cI\x96%&/m\xca{\x7fJ\... I'm fairly new to urllib, but this doesn't seem like the result I should be getting. I've tried it in other places and I get what I expect (the same as visiting the address with a browser gives me: a JSON object). Using urlopen on other sites (e.g. "http://google.com") works fine, and gives me actual html. I've also tried using urllib and it gives the same result. I'm pretty stuck, not even knowing where to look to solve this problem. Any ideas?

    Read the article

  • Urllib's urlopen breaking on some sites (e.g. StackApps api): returns garbage results

    - by Edan Maor
    I'm using urllib2's urlopen function to try and get a JSON result from the StackOverflow api. The code I'm using: >>> import urllib2 >>> conn = urllib2.urlopen("http://api.stackoverflow.com/0.8/users/") >>> conn.readline() The result I'm getting: '\x1f\x8b\x08\x00\x00\x00\x00\x00\x04\x00\xed\xbd\x07`\x1cI\x96%&/m\xca{\x7fJ\... I'm fairly new to urllib, but this doesn't seem like the result I should be getting. I've tried it in other places and I get what I expect (the same as visiting the address with a browser gives me: a JSON object). Using urlopen on other sites (e.g. "http://google.com") works fine, and gives me actual html. I've also tried using urllib and it gives the same result. I'm pretty stuck, not even knowing where to look to solve this problem. Any ideas?

    Read the article

  • Urllib's urlopen broken on some sites (e.g. StackApps api)

    - by Edan Maor
    I'm using urllib2's urlopen function to try and get a JSON result from the StackOverflow api. The code I'm using: >>> import urllib2 >>> conn = urllib2.urlopen("http://api.stackoverflow.com/0.8/users/") >>> conn.readline() The result I'm getting: '\x1f\x8b\x08\x00\x00\x00\x00\x00\x04\x00\xed\xbd\x07`\x1cI\x96%&/m\xca{\x7fJ\... I'm fairly new to urllib, but this doesn't seem like the result I should be getting. I've tried it in other places and I get what I expect (the same as visiting the address with a browser gives me: a JSON object). Using urlopen on other sites (e.g. "http://google.com") works fine, and gives me actual html. I've also tried using urllib and it gives the same result. I'm pretty stuck, not even knowing where to look to solve this problem. Any ideas?

    Read the article

  • Does urllib2.urlopen() actually fetch the page?

    - by beagleguy
    hi all, I was condering when I use urllib2.urlopen() does it just to header reads or does it actually bring back the entire webpage? IE does the HTML page actually get fetch on the urlopen call or the read() call? handle = urllib2.urlopen(url) html = handle.read() The reason I ask is for this workflow... I have a list of urls (some of them with short url services) I only want to read the webpage if I haven't seen that url before I need to call urlopen() and use geturl() to get the final page that link goes to (after the 302 redirects) so I know if I've crawled it yet or not. I don't want to incur the overhead of having to grab the html if I've already parsed that page. thanks!

    Read the article

  • Problem opening Solr *.jsp pages with urllib2.urlopen.

    - by nestling
    I'm trying to open a page at http://localhost:8983/solr/admin/stats.jsp but urllib2.urlopen returns a blank string. It works fine for solr/ and solr/admin, but for all the pages above /solr/admin/ I get nothing but a blank string. 76]: t = urllib2.urlopen('http://localhost:8983/solr/admin/stats.jsp') 77]: s = t.read() 78]: s 78]: 79]: type(s) 79]: <type 'str'> 80]: urllib2.urlopen('http://localhost:8983/solr/admin/registry.jsp').read() 80]: In [84]: urllib2.urlopen('http://localhost:8983/solr/admin/schema.jsp').read() Out[84]: I know this isn't a problem with urllib2, but beyond that I am at a loss. I wish solr (or jetty) had an easy to get to log file, so that perhaps it could tell me its side of the story.

    Read the article

  • Python urllib.urlopen IOError

    - by Michael
    So I have the following lines of code in a function sock = urllib.urlopen(url) html = sock.read() sock.close() and they work fine when I call the function by hand. However, when I call the function in a loop (using the same urls as earlier) I get the following error: > Traceback (most recent call last): File "./headlines.py", line 256, in <module> main(argv[1:]) File "./headlines.py", line 37, in main write_articles(headline, output_folder + "articles_" + term +"/") File "./headlines.py", line 232, in write_articles print get_blogs(headline, 5) File "/Users/michaelnussbaum08/Documents/College/Sophmore_Year/Quarter_2/Innovation/Headlines/_code/get_content.py", line 41, in get_blogs sock = urllib.urlopen(url) File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib.py", line 87, in urlopen return opener.open(url) File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib.py", line 203, in open return getattr(self, name)(url) File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib.py", line 314, in open_http if not host: raise IOError, ('http error', 'no host given') IOError: [Errno http error] no host given Any ideas?

    Read the article

  • urlopen error [errno 111] connection refused

    - by Ui-Gyun Jeong
    I am doing python exercise with a book 'headfirst python' and making android app by using python and sl4a my code is import android import json import time from urllib import urlencode from urllib2 import urlopen hello_msg = "Welcome to Coach Kelly's Timing App" list_title = 'Here is your list of athletes:' quit_msg = "Quitting Coach Kelly's App." web_server = 'http://127.0.0.1:8080' get_names_cgi = '/cgi-bin/generate_name.py' def send_to_server(url, post_data=None): if post_data: page = urlopen(url, urlencode(post_data)) else: page = urlopen(url) return(page.read().decode("utf8")) app = android.Android() def status_update(msg, how_long=2): app.makeToast(msg) time.sleep(how_long) status_update(hello_msg) athlete_names = sorted(json.loads(send_to_server(web_server + get_names_cgi))) app.dialogCreateAlert(list_title) app.dialogSetSingleChoiceItems(athlete_names) app.dialogSetPositiveButtonText('Select') app.dialogSetNegativeButtonText('Quit') app.dialogShow() resp = app.dialogGetResponse().result status_update(quit_msg) this is my code and the result is what is the problem??? I can not figure out what the problem is...

    Read the article

  • Application closes on Nokia E71 when using urllib.urlopen

    - by sammr
    Hello, Im running the following code on my Nokia E71. But after the text input, the program closes abruptly. I have a GPRS connection on my phone,but i still seem to be having some problem with urllib.urlopen The code is as follows : import appuifw,urllib amountInDollars = appuifw.query(u"Enter amount in Dollars","text") data=urllib.urlopen("http://www.google.com").read() appuifw.note(u"Hey","info") Any way to fix this problem ? Thank You

    Read the article

  • Use Twisted's getPage as urlopen?

    - by RadiantHex
    Hi folks, I would like to use Twisted non-blocking getPage method within a webapp, but it feels quite complicated to use such function compared to urlopen. This is an example of what I'm trying to achive: def web_request(request): response = urllib.urlopen('http://www.example.org') return HttpResponse(len(response.read())) Is it so hard to have something similar with getPage?

    Read the article

  • With regards to urllib AttributeError: 'module' object has no attribute 'urlopen'

    - by Matt
    import re import string import shutil import os import os.path import time import datetime import math import urllib from array import array import random filehandle = urllib.urlopen('http://www.google.com/') #open webpage s = filehandle.read() #read print s #display #what i plan to do with it once i get the first part working #results = re.findall('[<td style="font-weight:bold;" nowrap>$][0-9][0-9][0-9][.][0-9][0-9][</td></tr></tfoot></table>]',s) #earnings = '$ ' #for money in results: #earnings = earnings + money[1]+money[2]+money[3]+'.'+money[5]+money[6] #print earnings #raw_input() this is the code that i have so far. now i have looked at all the other forums that give solutions such as the name of the script, which is parse_Money.py, and i have tried doing it with urllib.request.urlopen AND i have tried running it on python 2.5, 2.6, and 2.7. If anybody has any suggestions it would be really welcome, thanks everyone!! --Matt ---EDIT--- I also tried this code and it worked, so im thinking its some kind of syntax error, so if anybody with a sharp eye can point it out, i would be very appreciative. import shutil import os import os.path import time import datetime import math import urllib from array import array import random b = 3 #find URL URL = raw_input('Type the URL you would like to read from[Example: http://www.google.com/] :') while b == 3: #get file name file1 = raw_input('Enter a file name for the downloaded code:') filepath = file1 + '.txt' if os.path.isfile(filepath): print 'File already exists' b = 3 else: print 'Filename accepted' b = 4 file_path = filepath #open file FileWrite = open(file_path, 'a') #acces URL filehandle = urllib.urlopen(URL) #display souce code for lines in filehandle.readlines(): FileWrite.write(lines) print lines print 'The above has been saved in both a text and html file' #close files filehandle.close() FileWrite.close()

    Read the article

  • Python urllib.urlopen() call doesn't work with a URL that a browser accepts

    - by Charles Anderson
    If I point Firefox at http://bitbucket.org/tortoisehg/stable/wiki/Home/ReleaseNotes, I get a page of HTML. But if I try this in Python: import urllib site = 'http://bitbucket.org/tortoisehg/stable/wiki/Home/ReleaseNotes' req = urllib.urlopen(site) text = req.read() I get the following: 500 Internal Server Error The server encountered an internal error or misconfiguration and was unable to complete your request. What am I doing wrong?

    Read the article

  • Python urllib2 multiple try statement on urlopen()

    - by Kura
    So, simply I want to be able to run a for across a list of URLs, if one fails then I want to continue on to try the next. I've tried using the following but sadly it throws and exception if the first URL doesn't work. servers = ('http://www.google.com', 'http://www.stackoverflow.com') for server in servers: try: u = urllib2.urlopen(server) except urllib2.URLError: continue else: break else: raise Any ideas? Thanks in advance.

    Read the article

  • Why can't I download a whole image file with urllib2.urlopen()

    - by John Gann
    When I run the following code, it only seems to be downloading the first little bit of the file and then exiting. Occassionally, I will get a 10054 error, but usually it just exits without getting the whole file. My internet connection is crappy wireless, and I often get broken downloads on larger files in firefox, but my browser has no problem getting a 200k image file. I'm new to python, and programming in general, so I'm wondering what nuance I'm missing. import urllib2 xkcdpic=urllib2.urlopen("http://imgs.xkcd.com/comics/literally.png") xkcdpicfile=open("C:\\Documents and Settings\\John Gann\\Desktop\\xkcd.png","w") while 1: chunk=xkcdpic.read(4028) if chunk: print chunk xkcdpicfile.write(chunk) else: break

    Read the article

  • Saving animated GIFs using urllib.urlopen (image saved does not animate)

    - by wenbert
    I have Apache2 + Django + X-sendfile. My problem is that when I upload an animated GIF, it won't "animate" when I output through the browser. Here is my code to display the image located outside the public accessible directory. def raw(request,uuid): target = str(uuid).split('.')[:-1][0] image = Uploads.objects.get(uuid=target) path = image.path filepath = os.path.join(path,"%s.%s" % (image.uuid,image.ext)) response = HttpResponse(mimetype=mimetypes.guess_type(filepath)) response['Content-Disposition']='filename="%s"'\ %smart_str(image.filename) response["X-Sendfile"] = filepath response['Content-length'] = os.stat(filepath).st_size return response UPDATE It turns out that it works. My problem is when I try to upload an image via URL. It probably doesn't save the entire GIF? def handle_url_file(request): """ Open a file from a URL. Split the file to get the filename and extension. Generate a random uuid using rand1() Then save the file. Return the UUID when successful. """ try: file = urllib.urlopen(request.POST['url']) randname = rand1(settings.RANDOM_ID_LENGTH) newfilename = request.POST['url'].split('/')[-1] ext = str(newfilename.split('.')[-1]).lower() im = cStringIO.StringIO(file.read()) # constructs a StringIO holding the image img = Image.open(im) filehash = checkhash(im) image = Uploads.objects.get(filehash=filehash) uuid = image.uuid return "%s" % (uuid) except Uploads.DoesNotExist: img.save(os.path.join(settings.UPLOAD_DIRECTORY,(("%s.%s")%(randname,ext)))) del img filesize = os.stat(os.path.join(settings.UPLOAD_DIRECTORY,(("%s.%s")%(randname,ext)))).st_size upload = Uploads( ip = request.META['REMOTE_ADDR'], filename = newfilename, uuid = randname, ext = ext, path = settings.UPLOAD_DIRECTORY, views = 1, bandwidth = filesize, source = request.POST['url'], size = filesize, filehash = filehash, ) upload.save() #return uuid return "%s" % (upload.uuid) except IOError, e: raise e Any ideas? Thanks! Wenbert

    Read the article

  • VB.Net HTTPWebRequest Speed is slow comparing Python URLOpen

    - by regexhacks
    Hi I am coding a web-crawler which will crawl the websites and selectively parse different sections of a web site. I am a .Net developer so the choice was obvious that I did it in .Net but the speed was very slow which included downloading and parsing of HTMLPages Then I tried to just download the contents first using .Net and then same domains using python but the python was very impressive in downloading data. I have achieved downloading using python but the later part is not that easy to code in python, which obviously i don't want to do. The same batch of domain which took 100 seconds in Python was taking 20 minutes in .Net based crawler I tried http://www.eqlit.com/ to download and in took 8 seconds in Python and same was taking 100 Seconds in .Net crawler Does anyone anyone have any idea why this is slow in .Net but fast in python?

    Read the article

  • Why can't I get Python's urlopen() method to work?

    - by froadie
    Why isn't this simple Python code working? import urllib file = urllib.urlopen('http://www.google.com') print file.read() This is the error that I get: Traceback (most recent call last): File "C:\workspace\GarchUpdate\src\Practice.py", line 26, in <module> file = urllib.urlopen('http://www.google.com') File "C:\Python26\lib\urllib.py", line 87, in urlopen return opener.open(url) File "C:\Python26\lib\urllib.py", line 206, in open return getattr(self, name)(url) File "C:\Python26\lib\urllib.py", line 345, in open_http h.endheaders() File "C:\Python26\lib\httplib.py", line 892, in endheaders self._send_output() File "C:\Python26\lib\httplib.py", line 764, in _send_output self.send(msg) File "C:\Python26\lib\httplib.py", line 723, in send self.connect() File "C:\Python26\lib\httplib.py", line 704, in connect self.timeout) File "C:\Python26\lib\socket.py", line 514, in create_connection raise error, msg IOError: [Errno socket error] [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond I've tried it with several different pages but I can never get the urlopen method to execute correctly.

    Read the article

  • Python form POST using urllib2 (also question on saving/using cookies)

    - by morpheous
    I am trying to write a function to post form data and save returned cookie info in a file so that the next time the page is visited, the cookie information is sent to the server (i.e. normal browser behavior). I wrote this relatively easily in C++ using curlib, but have spent almost an entire day trying to write this in Python, using urllib2 - and still no success. This is what I have so far: import urllib, urllib2 import logging # the path and filename to save your cookies in COOKIEFILE = 'cookies.lwp' cj = None ClientCookie = None cookielib = None logger = logging.getLogger(__name__) # Let's see if cookielib is available try: import cookielib except ImportError: logger.debug('importing cookielib failed. Trying ClientCookie') try: import ClientCookie except ImportError: logger.debug('ClientCookie isn\'t available either') urlopen = urllib2.urlopen Request = urllib2.Request else: logger.debug('imported ClientCookie succesfully') urlopen = ClientCookie.urlopen Request = ClientCookie.Request cj = ClientCookie.LWPCookieJar() else: logger.debug('Successfully imported cookielib') urlopen = urllib2.urlopen Request = urllib2.Request # This is a subclass of FileCookieJar # that has useful load and save methods cj = cookielib.LWPCookieJar() login_params = {'name': 'anon', 'password': 'pass' } def login(theurl, login_params): init_cookies(); data = urllib.urlencode(login_params) txheaders = {'User-agent' : 'Mozilla/4.0 (compatible; MSIE 5.5; Windows NT)'} try: # create a request object req = Request(theurl, data, txheaders) # and open it to return a handle on the url handle = urlopen(req) except IOError, e: log.debug('Failed to open "%s".' % theurl) if hasattr(e, 'code'): log.debug('Failed with error code - %s.' % e.code) elif hasattr(e, 'reason'): log.debug("The error object has the following 'reason' attribute :"+e.reason) sys.exit() else: if cj is None: log.debug('We don\'t have a cookie library available - sorry.') else: print 'These are the cookies we have received so far :' for index, cookie in enumerate(cj): print index, ' : ', cookie # save the cookies again cj.save(COOKIEFILE) #return the data return handle.read() # FIXME: I need to fix this so that it takes into account any cookie data we may have stored def get_page(*args, **query): if len(args) != 1: raise ValueError( "post_page() takes exactly 1 argument (%d given)" % len(args) ) url = args[0] query = urllib.urlencode(list(query.iteritems())) if not url.endswith('/') and query: url += '/' if query: url += "?" + query resource = urllib.urlopen(url) logger.debug('GET url "%s" => "%s", code %d' % (url, resource.url, resource.code)) return resource.read() When I attempt to log in, I pass the correct username and pwd,. yet the login fails, and no cookie data is saved. My two questions are: can anyone see whats wrong with the login() function, and how may I fix it? how may I modify the get_page() function to make use of any cookie info I have saved ?

    Read the article

  • My python auto-login script is broken.

    - by user310392
    A long time ago, I wrote a little python script to automatically log me on to the wireless network at my office. Here is the code: #!/opt/local/bin/python from urllib2 import urlopen from ClientForm import ParseResponse try: if "Logged on as" in urlopen("https://MYWIRELESS.com/logon").read(): print "Already logged on." else: forms = ParseResponse(urlopen("https://MYWIRELESS.com/logon"), backwards_compat=False) form = forms[0] form["username"], form["password"] = "ME", "MYPASSWD" urlopen(form.click()) print "Logged on. (probably :-)"; except IOError, e: print "Couldn't connect to wireless login page:\n", e I changed computers recently, and it stopped working. Now, I get the error: File "login.txt", line 4, in <module> from ClientForm import ParseResponse ImportError: No module named ClientForm which makes it look like I don't have some package (ClientForm) installed, so I installed it (sudo port install py-clientform), but I still get the same error. Does anyone have an idea what I'm doing wrong?

    Read the article

  • Check if the internet cannot be accessed in Python

    - by Sridhar Ratnakumar
    I have an app that makes a HTTP GET request to a particular URL on the internet. But when the network is down (say, no public wifi - or my ISP is down, or some such thing), I get the following traceback at urllib.urlopen: 70, in get u = urllib2.urlopen(req) File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 126, in urlopen return _opener.open(url, data, timeout) File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 391, in open response = self._open(req, data) File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 409, in _open '_open', req) File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 369, in _call_chain result = func(*args) File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 1161, in http_open return self.do_open(httplib.HTTPConnection, req) File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 1136, in do_open raise URLError(err) URLError: <urlopen error [Errno 8] nodename nor servname provided, or not known> I want to print a friendly error to the user telling him that his network maybe down instead of this unfriendly "nodename nor servname provided" error message. Sure I can catch URLError, but that would catch every url error, not just the one related to network downtime. I am not a purist, so even an error message like "The server example.com cannot be reached; either the server is indeed having problems or your network connection is down" would be nice. How do I go about selectively catching such errors? (For a start, if DNS resolution fails at urllib.urlopen, that can be reasonably assumed as network inaccessibility? If so, how do I "catch" it in the except block?)

    Read the article

  • How to call Twiter's Streaming/Filter Feed with urllib2/httplib?

    - by Simon
    Update: I switched this back from answered as I tried the solution posed in cogent Nick's answer and switched to Google's urlfetch: logging.debug("starting urlfetch for http://%s%s" % (self.host, self.url)) result = urlfetch.fetch("http://%s%s" % (self.host, self.url), payload=self.body, method="POST", headers=self.headers, allow_truncated=True, deadline=5) logging.debug("finished urlfetch") but unfortunately finished urlfetch is never printed - I see the timeout happen in the logs (it returns 200 after 5 seconds), but execution doesn't seem tor return. Hi All- I'm attempting to play around with Twitter's Streaming (aka firehose) API with Google App Engine (I'm aware this probably isn't a great long term play as you can't keep the connection perpetually open with GAE), but so far I haven't had any luck getting my program to actually parse the results returned by Twitter. Some code: logging.debug("firing up urllib2") req = urllib2.Request(url="http://%s%s" % (self.host, self.url), data=self.body, headers=self.headers) logging.debug("called urlopen for %s %s, about to call urlopen" % (self.host, self.url)) fobj = urllib2.urlopen(req) logging.debug("called urlopen") When this executes, unfortunately, my debug output never shows the called urlopen line printed. I suspect what's happening is that Twitter keeps the connection open and urllib2 doesn't return because the server doesn't terminate the connection. Wireshark shows the request being sent properly and a response returned with results. I tried adding Connection: close to my request header, but that didn't yield a successful result. Any ideas on how to get this to work? thanks -Simon

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

1 2 3 4 5 6  | Next Page >