Search Results

Search found 144 results on 6 pages for 'urlopen'.

Page 2/6 | < Previous Page | 1 2 3 4 5 6  | Next Page >

• In Python, urllib2 giving error

  - by pythBegin

  I tried running this, >>> urllib2.urlopen('http://tycho.usno.navy.mil/cgi-bin/timer.pl') But it is giving error like this, can anyone tell me a solution ? Traceback (most recent call last): File "<pyshell#11>", line 1, in <module> urllib2.urlopen('http://tycho.usno.navy.mil/cgi-bin/timer.pl') File "C:\Python26\lib\urllib2.py", line 126, in urlopen return _opener.open(url, data, timeout) File "C:\Python26\lib\urllib2.py", line 391, in open response = self._open(req, data) File "C:\Python26\lib\urllib2.py", line 409, in _open '_open', req) File "C:\Python26\lib\urllib2.py", line 369, in _call_chain result = func(*args) File "C:\Python26\lib\urllib2.py", line 1161, in http_open return self.do_open(httplib.HTTPConnection, req) File "C:\Python26\lib\urllib2.py", line 1136, in do_open raise URLError(err) URLError: <urlopen error [Errno 11001] getaddrinfo failed>

  Read the article

• cant download youtube video

  - by dsaccount1

  I'm having trouble retrieving the youtube video automatically, heres the code. The problem is the last part. download = urllib.request.urlopen(download_url).read() # Youtube video download script # 10n1z3d[at]w[dot]cn import urllib.request import sys print("\n--------------------------") print (" Youtube Video Downloader") print ("--------------------------\n") try: video_url = sys.argv[1] except: video_url = input('[+] Enter video URL: ') print("[+] Connecting...") try: if(video_url.endswith('&feature=related')): video_id = video_url.split('www.youtube.com/watch?v=')[1].split('&feature=related')[0] elif(video_url.endswith('&feature=dir')): video_id = video_url.split('www.youtube.com/watch?v=')[1].split('&feature=dir')[0] elif(video_url.endswith('&feature=fvst')): video_id = video_url.split('www.youtube.com/watch?v=')[1].split('&feature=fvst')[0] elif(video_url.endswith('&feature=channel_page')): video_id = video_url.split('www.youtube.com/watch?v=')[1].split('&feature=channel_page')[0] else: video_id = video_url.split('www.youtube.com/watch?v=')[1] except: print("[-] Invalid URL.") exit(1) print("[+] Parsing token...") try: url = str(urllib.request.urlopen('http://www.youtube.com/get_video_info?&video_id=' + video_id).read()) token_value = url.split('video_id='+video_id+'&token=')[1].split('&thumbnail_url')[0] download_url = "http://www.youtube.com/get_video?video_id=" + video_id + "&t=" + token_value + "&fmt=18" except: url = str(urllib.request.urlopen('www.youtube.com/watch?v=' + video_id)) exit(1) v_url=str(urllib.request.urlopen('http://'+video_url).read()) video_title = v_url.split('"rv.2.title": "')[1].split('", "rv.4.rating"')[0] if '&quot;' in video_title: video_title = video_title.replace('&quot;','"') elif '&amp;' in video_title: video_title = video_title.replace('&amp;','&') print("[+] Downloading " + '"' + video_title + '"...') try: print(download_url) file = open(video_title + '.mp4', 'wb') download = urllib.request.urlopen(download_url).read() print(download) for line in download: file.write(line) file.close() except: print("[-] Error downloading. Quitting.") exit(1) print("\n[+] Done. The video is saved to the current working directory(cwd).\n")

  Read the article

• Permission denied error when trying to install with wubi

  - by badri

  I run into problem when tried to install Ubuntu 11.04 on Windows 7 using wubi installer . It downloads the image for sometime and ends up with the error that says Permission denied: for more details see the log file In the log it seems to be like DownloadError: Problem connecting to tracker - urlopen error (10060, 'Operation timed out') but my network is good and I checked it. Tried using wubi several times, but ends up with same problem. Log content: 10-08 16:56 DEBUG TaskList: ### Finished get_metalink 10-08 16:56 DEBUG TaskList: New task download 10-08 16:56 DEBUG TaskList: ### Running download... 10-08 16:56 DEBUG btdownloader: downloading http://releases.ubuntu.com/11.04/ubuntu-11.04-desktop-amd64.iso.torrent > C:\ubuntu\install\ubuntu-11.04-desktop-amd64.iso 10-08 18:02 ERROR TaskList: Traceback (most recent call last): File "\lib\bittorrent\RawServer.py", line 229, in listen_forever File "\lib\wubi\backends\common\btdownloader.py", line 70, in error_callback DownloadError: Traceback (most recent call last): File "\lib\bittorrent\RawServer.py", line 221, in listen_forever File "\lib\bittorrent\Rerequester.py", line 96, in fail File "\lib\wubi\backends\common\btdownloader.py", line 70, in error_callback DownloadError: Problem connecting to tracker - urlopen error (10060, 'Operation timed out') Traceback (most recent call last): File "\lib\wubi\backends\common\tasklist.py", line 197, in __call__ File "\lib\wubi\backends\common\btdownloader.py", line 79, in download File "\lib\bittorrent\download.py", line 303, in download File "\lib\bittorrent\RawServer.py", line 256, in listen_forever File "\lib\wubi\backends\common\btdownloader.py", line 70, in error_callback DownloadError: Traceback (most recent call last): File "\lib\bittorrent\RawServer.py", line 229, in listen_forever File "\lib\wubi\backends\common\btdownloader.py", line 70, in error_callback DownloadError: Traceback (most recent call last): File "\lib\bittorrent\RawServer.py", line 221, in listen_forever File "\lib\bittorrent\Rerequester.py", line 96, in fail File "\lib\wubi\backends\common\btdownloader.py", line 70, in error_callback DownloadError: Problem connecting to tracker - urlopen error (10060, 'Operation timed out') 10-08 18:02 ERROR TaskList: Non fatal error Traceback (most recent call last): File "\lib\bittorrent\RawServer.py", line 229, in listen_forever File "\lib\wubi\backends\common\btdownloader.py", line 70, in error_callback DownloadError: Traceback (most recent call last): File "\lib\bittorrent\RawServer.py", line 221, in listen_forever File "\lib\bittorrent\Rerequester.py", line 96, in fail File "\lib\wubi\backends\common\btdownloader.py", line 70, in error_callback DownloadError: Problem connecting to tracker - urlopen error (10060, 'Operation timed out') in task download 10-08 18:02 DEBUG TaskList: ### Finished download 10-08 18:02 ERROR TaskList: [Errno 13] Permission denied: u'C:\\ubuntu\\install\\ubuntu-11.04-desktop-amd64.iso' Traceback (most recent call last): File "\lib\wubi\backends\common\tasklist.py", line 197, in __call__ File "\lib\wubi\backends\common\backend.py", line 492, in get_iso File "\lib\wubi\backends\common\backend.py", line 347, in download_iso OSError: [Errno 13] Permission denied: u'C:\\ubuntu\\install\\ubuntu-11.04-desktop-amd64.iso' 10-08 18:02 DEBUG TaskList: # Cancelling tasklist 10-08 18:02 DEBUG TaskList: # Finished tasklist 10-08 18:02 ERROR root: [Errno 13] Permission denied: u'C:\\ubuntu\\install\\ubuntu-11.04-desktop-amd64.iso' Traceback (most recent call last): File "\lib\wubi\application.py", line 57, in run File "\lib\wubi\application.py", line 131, in select_task File "\lib\wubi\application.py", line 157, in run_installer File "\lib\wubi\backends\common\tasklist.py", line 197, in __call__ File "\lib\wubi\backends\common\backend.py", line 492, in get_iso File "\lib\wubi\backends\common\backend.py", line 347, in download_iso OSError: [Errno 13] Permission denied: u'C:\\ubuntu\\install\\ubuntu-11.04-desktop-amd64.iso'

  Read the article

• Python HTTPS requests (urllib2) fails on Ubuntu 12.04 without proxy

  - by Pablo

  I have an little app I wrote in Python and it used to work... until yesterday, when it suddenly started giving me an error in a HTTPS connection. I don't remember if there was an update, but both Python 2.7.3rc2 and Python 3.2 are failing just the same. I googled it and found out that this happens when people are behind a proxy, but I'm not (and nothing have changed in my network since the last time it worked). My syster's computer running windows and Python 2.7.2 has no problems (in the same network). response = urllib2.urlopen(url).read() File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen return _opener.open(url, data, timeout) File "/usr/lib/python2.7/urllib2.py", line 400, in open response = self._open(req, data) File "/usr/lib/python2.7/urllib2.py", line 418, in _open '_open', req) File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain result = func(*args) File "/usr/lib/python2.7/urllib2.py", line 1215, in https_open return self.do_open(httplib.HTTPSConnection, req) File "/usr/lib/python2.7/urllib2.py", line 1177, in do_open raise URLError(err) urllib2.URLError: <urlopen error [Errno 8] _ssl.c:504: EOF occurred in violation of protocol> What's wrong? Any help is appreciated. PS.: Older python versions don't work either, not in my system and not in a live session from USB, but DO work in a Ubuntu 11.10 live session.

  Read the article

• Problem with python urllib

  - by mudder

  I'm getting an error when ever I try to pull down a web page with urllib.urlopen. I've disabled windows firewall and my AV so its not that. I can access the pages in my browser. I even reinstalled python to rule out it being a broken urllib. Any help would be greatly appreciated. >>> import urllib >>> h = urllib.urlopen("http://www.google.com").read() Traceback (most recent call last): File "<pyshell#1>", line 1, in <module> h = urllib.urlopen("http://www.google.com").read() File "C:\Python26\lib\urllib.py", line 86, in urlopen return opener.open(url) File "C:\Python26\lib\urllib.py", line 205, in open return getattr(self, name)(url) File "C:\Python26\lib\urllib.py", line 344, in open_http h.endheaders() File "C:\Python26\lib\httplib.py", line 904, in endheaders self._send_output() File "C:\Python26\lib\httplib.py", line 776, in _send_output self.send(msg) File "C:\Python26\lib\httplib.py", line 735, in send self.connect() File "C:\Python26\lib\httplib.py", line 716, in connect self.timeout) File "C:\Python26\lib\socket.py", line 514, in create_connection raise error, msg IOError: [Errno socket error] [Errno 10061] No connection could be made because the target machine actively refused it >>>

  Read the article

• Repeated host lookups failing in urllib2

  - by reve_etrange

  I have code which issues many HTTP GET requests using Python's urllib2, in several threads, writing the responses into files (one per thread). During execution, it looks like many of the host lookups fail (causing a name or service unknown error, see appended error log for an example). Is this due to a flaky DNS service? Is it bad practice to rely on DNS caching, if the host name isn't changing? I.e. should a single lookup's result be passed into the urlopen? Exception in thread Thread-16: Traceback (most recent call last): File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner self.run() File "/home/da/local/bin/ThreadedDownloader.py", line 61, in run page = urllib2.urlopen(url) # get the page File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen return _opener.open(url, data, timeout) File "/usr/lib/python2.6/urllib2.py", line 391, in open response = self._open(req, data) File "/usr/lib/python2.6/urllib2.py", line 409, in _open '_open', req) File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain result = func(*args) File "/usr/lib/python2.6/urllib2.py", line 1170, in http_open return self.do_open(httplib.HTTPConnection, req) File "/usr/lib/python2.6/urllib2.py", line 1145, in do_open raise URLError(err) URLError: <urlopen error [Errno -2] Name or service not known>

  Read the article

• Facebook publish HTTP Error 400 : bad request

  - by Abhishek

  Hey I am trying to publish a score to Facebook through python's urllib2 library. import urllib2,urllib url = "https://graph.facebook.com/USER_ID/scores" data = {} data['score']=SCORE data['access_token']='APP_ACCESS_TOKEN' data_encode = urllib.urlencode(data) request = urllib2.Request(url, data_encode) response = urllib2.urlopen(request) responseAsString = response.read() I am getting this error: response = urllib2.urlopen(request) File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 124, in urlopen return _opener.open(url, data, timeout) File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 389, in open response = meth(req, response) File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 502, in http_response 'http', request, response, code, msg, hdrs) File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 427, in error return self._call_chain(*args) File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 361, in _call_chain result = func(*args) File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/urllib2.py", line 510, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) urllib2.HTTPError: HTTP Error 400: Bad Request Not sure if this is relating to Facebook's Open Graph or improper urllib2 API use.

  Read the article

• Difference between URLLIB2 call in IDLE and from Django?

  - by danspants

  The following piece of code works as expected when running in a local install of django apache 2.2 fx = urllib2.Request(f); fx.add_header('User-Agent','Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.36 Safari/525.19'); url_opened = urllib2.urlopen(fx); However when I enter that code into IDLE on the same machine I get the following error: url_opened = urllib2.urlopen(fx); File "C:\Python25\lib\urllib2.py", line 124, in urlopen return _opener.open(url, data) File "C:\Python25\lib\urllib2.py", line 387, in open response = meth(req, response) File "C:\Python25\lib\urllib2.py", line 498, in http_response 'http', request, response, code, msg, hdrs) File "C:\Python25\lib\urllib2.py", line 425, in error return self._call_chain(*args) File "C:\Python25\lib\urllib2.py", line 360, in _call_chain result = func(*args) File "C:\Python25\lib\urllib2.py", line 506, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp) HTTPError: HTTP Error 407: Proxy Authentication Required Any ideas?

  Read the article

• python socket problem

  - by uytg

  I write this python code: import socks import socket socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, "64.83.219.7", 58279) socket.socket = socks.socksocket socket.setdefaulttimeout(19) import urllib2 print urllib2.urlopen('http://www.google.com').read() but when I execute it, I get this error: urllib2.URLError: <urlopen error timed out> What am I doing wrong?

  Read the article

• Handling urllib2's timeout? - Python

  - by RadiantHex

  Hi folks, I'm using the timeout parameter within the urllib2's urlopen. urllib2.urlopen('http://www.example.org', timeout=1) How do I tell Python that if the timeout expires a custom error should be raised? Any ideas?

  Read the article

• use proxy in python to fetch a webpage

  - by carmao

  I am trying to write a function in Python to use a public anonymous proxy and fetch a webpage, but I got a rather strange error. The code (I have Python 2.4): import urllib2 def get_source_html_proxy(url, pip, timeout): # timeout in seconds (maximum number of seconds willing for the code to wait in # case there is a proxy that is not working, then it gives up) proxy_handler = urllib2.ProxyHandler({'http': pip}) opener = urllib2.build_opener(proxy_handler) opener.addheaders = [('User-agent', 'Mozilla/5.0')] urllib2.install_opener(opener) req=urllib2.Request(url) sock=urllib2.urlopen(req) timp=0 # a counter that is going to measure the time until the result (webpage) is # returned while 1: data = sock.read(1024) timp=timp+1 if len(data) < 1024: break timpLimita=50000000 * timeout if timp==timpLimita: # 5 millions is about 1 second break if timp==timpLimita: print IPul + ": Connection is working, but the webpage is fetched in more than 50 seconds. This proxy returns the following IP: " + str(data) return str(data) else: print "This proxy " + IPul + "= good proxy. " + "It returns the following IP: " + str(data) return str(data) # Now, I call the function to test it for one single proxy (IP:port) that does not support user and password (a public high anonymity proxy) #(I put a proxy that I know is working - slow, but is working) rez=get_source_html_proxy("http://www.whatismyip.com/automation/n09230945.asp", "93.84.221.248:3128", 50) print rez The error: Traceback (most recent call last): File "./public_html/cgi-bin/teste5.py", line 43, in ? rez=get_source_html_proxy("http://www.whatismyip.com/automation/n09230945.asp", "93.84.221.248:3128", 50) File "./public_html/cgi-bin/teste5.py", line 18, in get_source_html_proxy sock=urllib2.urlopen(req) File "/usr/lib64/python2.4/urllib2.py", line 130, in urlopen return _opener.open(url, data) File "/usr/lib64/python2.4/urllib2.py", line 358, in open response = self._open(req, data) File "/usr/lib64/python2.4/urllib2.py", line 376, in _open '_open', req) File "/usr/lib64/python2.4/urllib2.py", line 337, in _call_chain result = func(*args) File "/usr/lib64/python2.4/urllib2.py", line 573, in lambda r, proxy=url, type=type, meth=self.proxy_open: \ File "/usr/lib64/python2.4/urllib2.py", line 580, in proxy_open if '@' in host: TypeError: iterable argument required I do not know why the character "@" is an issue (I have no such in my code. Should I have?) Thanks in advance for your valuable help.

  Read the article

• How do I compile a Wikipedia lens and install?

  - by user49523

  I read a tutorial about how to compile and install a Wikipedia lens, but it didn't work. The tutorial sounds easy - i just copied and pasted to the file that was suppose to edit. I have tried some times and here are 2 edits edit 1: import logging import optparse import gettext from gettext import gettext as _ gettext.textdomain('wikipedia') from singlet.lens import SingleScopeLens, IconViewCategory, ListViewCategory from wikipedia import wikipediaconfig import urllib2 import simplejson class WikipediaLens(SingleScopeLens): wiki = "http://en.wikipedia.org" def wikipedia_query(self,search): try: search = search.replace(" ", "|") url = ("%s/w/api.php?action=opensearch&limit=25&format=json&search=%s" % (self.wiki, search)) results = simplejson.loads(urllib2.urlopen(url).read()) print "Searching Wikipedia" return results[1] except (IOError, KeyError, urllib2.URLError, urllib2.HTTPError, simplejson.JSONDecodeError): print "Error : Unable to search Wikipedia" return [] class Meta: name = 'Wikipedia' description = 'Wikipedia Lens' search_hint = 'Search Wikipedia' icon = 'wikipedia.svg' search_on_blank=True # TODO: Add your categories articles_category = ListViewCategory("Articles", "dialog-information-symbolic") def search(self, search, results): for article in self.wikipedia_query(search): results.append("%s/wiki/%s" % (self.wiki, article), "http://upload.wikimedia.org/wikipedia/commons/6/63/Wikipedia-logo.png", self.articles_category, "text/html", article, "Wikipedia Article", "%s/wiki/%s" % (self.wiki, article)) pass edit 2: import urllib2 import simplejson import logging import optparse import gettext from gettext import gettext as _ gettext.textdomain('wikipediaa') from singlet.lens import SingleScopeLens, IconViewCategory, ListViewCategory from wikipediaa import wikipediaaconfig class WikipediaaLens(SingleScopeLens): wiki = "http://en.wikipedia.org" def wikipedia_query(self,search): try: search = search.replace(" ", "|") url = ("%s/w/api.php?action=opensearch&limit=25&format=json&search=%s" % (self.wiki, search)) results = simplejson.loads(urllib2.urlopen(url).read()) print "Searching Wikipedia" return results[1] except (IOError, KeyError, urllib2.URLError, urllib2.HTTPError, simplejson.JSONDecodeError): print "Error : Unable to search Wikipedia" return [] def search(self, search, results): for article in self.wikipedia_query(search): results.append("%s/wiki/%s" % (self.wiki, article), "http://upload.wikimedia.org/wikipedia/commons/6/63/Wikipedia-logo.png", self.articles_category, "text/html", article, "Wikipedia Article", "%s/wiki/%s" % (self.wiki, article)) pass class Meta: name = 'Wikipedia' description = 'Wikipedia Lens' search_hint = 'Search Wikipedia' icon = 'wikipedia.svg' search_on_blank=True # TODO: Add your categories articles_category = ListViewCategory("Articles", "dialog-information-symbolic") def search(self, search, results): # TODO: Add your search results results.append('https://wiki.ubuntu.com/Unity/Lenses/Singlet', 'ubuntu-logo', self.example_category, "text/html", 'Learn More', 'Find out how to write your Unity Lens', 'https://wiki.ubuntu.com/Unity/Lenses/Singlet') pass so .. what can i change in the edit ? (if anybody give me the entire edit file edited i will appreciate)

  Read the article

• python urllib post question

  - by paul

  hello ALL im making some simple python post script but it not working well. there is 2 part to have to login. first login is using 'http://mybuddy.buddybuddy.co.kr/userinfo/UserInfo.asp' this one. and second login is using 'http://user.buddybuddy.co.kr/usercheck/UserCheckPWExec.asp' i can login first login page, but i couldn't login second page website. and return some error 'illegal access' such like . i heard this is related with some cooke but i don't know how to implement to resolve this problem. if anyone can help me much appreciated!! Thanks! import re,sys,os,mechanize,urllib,time import datetime,socket params = urllib.urlencode({'ID':'ph896011', 'PWD':'pk1089' }) rq = mechanize.Request("http://mybuddy.buddybuddy.co.kr/userinfo/UserInfo.asp", params) rs = mechanize.urlopen(rq) data = rs.read() logged_fail = r';history.back();</script>' in data if not logged_fail: print 'login success' try: params = urllib.urlencode({'PASSWORD':'pk1089'}) rq = mechanize.Request("http://user.buddybuddy.co.kr/usercheck/UserCheckPWExec.asp", params ) rs = mechanize.urlopen(rq) data = rs.read() print data except: print 'error'

  Read the article

• Able to ping but cannot browse after several hours running of my python program

  - by Shane

  It's a GUI program I wrote in python checking website/server status running on my XP SP3, multi threads are used to check different site/server. After several hours running, the program starts to get urlopen error timed out all the time, and this always happens right after a POST request from a server(not a certain one, might be A or B or C), and it's also not the first POST request causing the problem, normally after several hours running and it happens to make a POST request at an unknown moment, all you get from then on is urlopen error timed out. I'm still able to ping but cannot browse any site, once the program closed everything's fine. It's definitely the program causing this problem, well I just don't know how to debug/check what the problem is, also don't know if it's from OS side or my program wasting too many resources/connections(are you still able to ping when too many connections used?), would anybody please help me out?

  Read the article

• stopping a cherrypy server over http

  - by d.c

  I have a cherrypy app that I'm controlling over http with a wxpython ui. I want to kill the server when the ui closes, but I don't know how to do that. Right now I'm just doing a sys.exit() on the window close event but thats resulting in Traceback (most recent call last): File "ui.py", line 67, in exitevent urllib.urlopen("http://"+server+"/?sigkill=1") File "c:\python26\lib\urllib.py", line 87, in urlopen return opener.open(url) File "c:\python26\lib\urllib.py", line 206, in open return getattr(self, name)(url) File "c:\python26\lib\urllib.py", line 354, in open_http 'got a bad status line', None) IOError: ('http protocol error', 0, 'got a bad status line', None) is that because I'm not stopping cherrypy properly?

  Read the article

• Error while trying to parse a website url using python . how to debug it ?

  - by mekasperasky

  #!/usr/bin/python import json import urllib from BeautifulSoup import BeautifulSoup from BeautifulSoup import BeautifulStoneSoup import BeautifulSoup def showsome(searchfor): query = urllib.urlencode({'q': searchfor}) url = 'http://ajax.googleapis.com/ajax/services/search/web?v=1.0&%s' % query search_response = urllib.urlopen(url) search_results = search_response.read() results = json.loads(search_results) data = results['responseData'] print 'Total results: %s' % data['cursor']['estimatedResultCount'] hits = data['results'] print 'Top %d hits:' % len(hits) for h in hits: print ' ', h['url'] resp = urllib.urlopen(h['url']) res = resp.read() soup = BeautifulSoup(res) print soup.prettify() print 'For more results, see %s' % data['cursor']['moreResultsUrl'] showsome('sachin') What is the wrong in this code ? Note all the 4 links that I am getting out of the search , I am feeding it back to extract the contents out of it , and then use BeautifulSoup to parse it . How should I go about it ?

  Read the article

• Python: Is there a way to get HTML that was dynamically created by Javascript?

  - by Joschua

  As far as I can tell, this is the case for LyricWikia. The lyrics (example) can be accessed from the browser, but can't be found in the source code (can be opened with CTRL + U in most browsers) or reading the contents of the site with Python: from urllib.request import urlopen URL = 'http://lyrics.wikia.com/Billy_Joel:Piano_Man' r = urlopen(URL).read().decode('utf-8') And the test: >>> 'Now John at the bar is a friend of mine' in r False >>> 'John' in r False But when you select and look at the source code of the box in which the lyrics are displayed, you can see that there is: <div class="lyricbox">[...]</div> Is there a way to get the contents of that div-element with Python?

  Read the article

• how to send data to server using python

  - by Apache

  hi experts, how data can be send to the server, for example i retrieve MAC address, so i want send to the server ( i.e 211.21.24.43:8080/data?mac=00-0C-F1-56-98-AD i found snippet from internet as below from urllib2 import Request, urlopen from binascii import b2a_base64 def b64open(url, postdata): req = Request(url, b2a_base64(postdata), headers={'Content-Transfer-Encoding': 'base64'}) return urlopen(req) conn = b64open("http://211.21.24.43:8080/data","mac=00-0C-F1-56-98-AD") but when run, File "send2.py", line 8 SyntaxError: Non-ASCII character '\xc3' in file send2.py on line 8, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details can anyone help me how send data to the server thanks in advance

  Read the article

• Regular expressions in python unicode

  - by Remy

  I need to remove all the html tags from a given webpage data. I tried this using regular expressions: import urllib2 import re page = urllib2.urlopen("http://www.frugalrules.com") from bs4 import BeautifulSoup, NavigableString, Comment soup = BeautifulSoup(page) link = soup.find('link', type='application/rss+xml') print link['href'] rss = urllib2.urlopen(link['href']).read() souprss = BeautifulSoup(rss) description_tag = souprss.find_all('description') content_tag = souprss.find_all('content:encoded') print re.sub('<[^>]*>', '', content_tag) But the syntax of the re.sub is: re.sub(pattern, repl, string, count=0) So, I modified the code as (instead of the print statement above): for row in content_tag: print re.sub(ur"<[^>]*>",'',row,re.UNICODE But it gives the following error: Traceback (most recent call last): File "C:\beautifulsoup4-4.3.2\collocation.py", line 20, in <module> print re.sub(ur"<[^>]*>",'',row,re.UNICODE) File "C:\Python27\lib\re.py", line 151, in sub return _compile(pattern, flags).sub(repl, string, count) TypeError: expected string or buffer What am I doing wrong?

  Read the article

• php method for key retrieval

  - by parthav

  Is there any equivalent php method to 'urllib.urlopen()' for retrieving the public key from any public key server

  Read the article

• Generating PDF results in single page only?

  - by A T

  Generating a PDF from an email (Zurb Ink templated); but am always presented with a single page PDF. Runnable test-case: from weasyprint import HTML, CSS from urllib2 import urlopen if __name__ == '__main__': html = urlopen('http://zurb.com/ink/downloads/templates/basic.html').read() html = html.replace('<p class=\"lead\">', '{0}<p class=\"lead\">'.format( '<p class=\"lead\">{0}</p>'.format("foobar " * 50) * 50)) HTML(string=html).write_pdf('foo.pdf', stylesheets=[ CSS(string='@page { size: A4; margin: 2cm };' '* { float: none !important; };' '@media print { nav { display: none; } }') ]) How do I get a multi-page PDF?

  Read the article

• Error 400 in urllib2 when using cookies

  - by Hempage

  I've had some success using a different method to load cookies, but now I'm wanting to use the cookielib.MozillaCookieJar method to open a cookies.txt file. Here's the snippet of code that does this. cookieJar=MozillaCookieJar() cookieJar.load(argv[2]) After creating an HTTPCookieProcessor opener, and installing it, whenever I use urlopen, I get an HTTP 400 error. However, if I don't use a CookieJar, the urlopen method succeeds (though the response doesn't contain the data I need). I'm not sure whether the cookies.txt file is malformed, or whether there's something else going on.

  Read the article

• initrd.lz is corrupted error occured while installing 11.10

  - by zubendra

  C:\ubuntu\install\boot\initrd.lz is corrupted. Error pop-up comes up every time i am trying to install ubuntu-11.10-desktop-i386 using wubi. error comes when the installation process is almost completed. can anyone suggest a solution for this problem. Its occurring regularly. 03-19 18:01 DEBUG TaskList: ## Running copy_installation_files... 03-19 18:01 DEBUG WindowsBackend: Copying C:\DOCUME~1\HP_OWN~1.YOU\LOCALS~1\Temp\pyl59.tmp\data\custom-installation -> C:\ubuntu\install\custom-installation 03-19 18:01 DEBUG WindowsBackend: Copying C:\DOCUME~1\HP_OWN~1.YOU\LOCALS~1\Temp\pyl59.tmp\winboot -> C:\ubuntu\winboot 03-19 18:01 DEBUG WindowsBackend: Copying C:\DOCUME~1\HP_OWN~1.YOU\LOCALS~1\Temp\pyl59.tmp\data\images\Ubuntu.ico -> C:\ubuntu\Ubuntu.ico 03-19 18:01 DEBUG TaskList: ## Finished copy_installation_files 03-19 18:01 DEBUG TaskList: ## Running get_iso... 03-19 18:01 DEBUG CommonBackend: Trying to use pre-specified ISO X:\ubuntu-11.10-desktop-i386.iso 03-19 18:01 DEBUG TaskList: New task is_valid_iso 03-19 18:01 DEBUG TaskList: ### Running is_valid_iso... 03-19 18:01 DEBUG Distro: checking Ubuntu ISO X:\ubuntu-11.10-desktop-i386.iso 03-19 18:01 INFO Distro: Found a valid iso for Ubuntu: X:\ubuntu-11.10-desktop-i386.iso 03-19 18:01 DEBUG TaskList: ### Finished is_valid_iso 03-19 18:01 DEBUG TaskList: New task check_iso 03-19 18:01 DEBUG TaskList: ### Running check_iso... 03-19 18:01 DEBUG CommonBackend: Checking X:\ubuntu-11.10-desktop-i386.iso 03-19 18:01 DEBUG Distro: checking Ubuntu ISO X:\ubuntu-11.10-desktop-i386.iso 03-19 18:01 INFO Distro: Found a valid iso for Ubuntu: X:\ubuntu-11.10-desktop-i386.iso 03-19 18:01 DEBUG CommonBackend: Using distro Ubuntu i386 instead of Ubuntu amd64 03-19 18:01 DEBUG TaskList: New task get_metalink 03-19 18:01 DEBUG TaskList: #### Running get_metalink... 03-19 18:01 DEBUG downloader: downloading http://releases.ubuntu.com/11.10/ubuntu-11.10-desktop-i386.metalink > C:\ubuntu\install 03-19 18:01 ERROR CommonBackend: Cannot download metalink file http://releases.ubuntu.com/11.10/ubuntu-11.10-desktop-i386.metalink err=[Errno 4] IOError: <urlopen error (7, 'getaddrinfo failed')> 03-19 18:01 DEBUG downloader: downloading http://cdimage.ubuntu.com/daily-live/current/oneiric-desktop-i386.metalink > C:\ubuntu\install 03-19 18:01 ERROR CommonBackend: Cannot download metalink file2 http://cdimage.ubuntu.com/daily-live/current/oneiric-desktop-i386.metalink err=[Errno 4] IOError: <urlopen error (7, 'getaddrinfo failed')> 03-19 18:01 DEBUG TaskList: #### Finished get_metalink 03-19 18:01 ERROR CommonBackend: ERROR: the metalink file is not available, cannot check the md5 for X:\ubuntu-11.10-desktop-i386.iso, ignoring 03-19 18:01 DEBUG TaskList: ### Finished check_iso 03-19 18:01 DEBUG TaskList: New task copy_file 03-19 18:01 DEBUG CommonBackend: Copying X:\ubuntu-11.10-desktop-i386.iso > C:\ubuntu\install\installation.iso 03-19 18:01 DEBUG TaskList: ### Running copy_file... 03-19 18:01 DEBUG TaskList: ### Finished copy_file 03-19 18:01 DEBUG TaskList: ## Finished get_iso 03-19 18:01 DEBUG TaskList: ## Running extract_kernel... 03-19 18:01 DEBUG CommonBackend: Extracting files from ISO C:\ubuntu\install\installation.iso 03-19 18:01 DEBUG WindowsBackend: extracting md5sum.txt from C:\ubuntu\install\installation.iso 03-19 18:01 DEBUG WindowsBackend: extracting casper\vmlinuz from C:\ubuntu\install\installation.iso 03-19 18:01 DEBUG WindowsBackend: extracting casper\initrd.lz from C:\ubuntu\install\installation.iso 03-19 18:01 DEBUG CommonBackend: Checking kernel, initrd and md5sums 03-19 18:01 DEBUG CommonBackend: checking C:\ubuntu\install\boot\vmlinuz 03-19 18:01 DEBUG CommonBackend: C:\ubuntu\install\boot\vmlinuz md5 = fde150f5c6fd2de66ed7876efbfcc4c7 == fde150f5c6fd2de66ed7876efbfcc4c7 03-19 18:01 DEBUG CommonBackend: checking C:\ubuntu\install\boot\initrd.lz 03-19 18:01 DEBUG CommonBackend: C:\ubuntu\install\boot\initrd.lz md5 = 8900200c764438c1b124dff5ae92c763 != d6baee1e11f1d6de6eba6bd43dbde352 03-19 18:01 ERROR TaskList: File C:\ubuntu\install\boot\initrd.lz is corrupted Traceback (most recent call last): File "\lib\wubi\backends\common\tasklist.py", line 197, in __call__ File "\lib\wubi\backends\common\backend.py", line 623, in extract_kernel Exception: File C:\ubuntu\install\boot\initrd.lz is corrupted 03-19 18:01 DEBUG TaskList: # Cancelling tasklist 03-19 18:01 ERROR root: File C:\ubuntu\install\boot\initrd.lz is corrupted Traceback (most recent call last): File "\lib\wubi\application.py", line 58, in run File "\lib\wubi\application.py", line 132, in select_task File "\lib\wubi\application.py", line 158, in run_installer File "\lib\wubi\backends\common\tasklist.py", line 197, in __call__ File "\lib\wubi\backends\common\backend.py", line 623, in extract_kernel Exception: File C:\ubuntu\install\boot\initrd.lz is corrupted 03-19 18:01 DEBUG TaskList: # Finished tasklist

  Read the article

• Spam proof hit counter in Django

  - by Jim Robert

  I already looked at the most popular Django hit counter solutions and none of them seem to solve the issue of spamming the refresh button. Do I really have to log the IP of every visitor to keep them from artificially boosting page view counts by spamming the refresh button (or writing a quick and dirty script to do it for them)? More information So right now you can inflate your view count with the following few lines of Python code. Which is so little that you don't even really need to write a script, you could just type it into an interactive session: from urllib import urlopen num_of_times_to_hit_page = 100 url_of_the_page = "http://example.com" for x in range(num_of_times_to_hit_page): urlopen(url_of_the_page) Solution I'll probably use To me, it's a pretty rough situation when you need to do a bunch of writes to the database on EVERY page view, but I guess it can't be helped. I'm going to implement IP logging due to several users artificially inflating their view count. It's not that they're bad people or even bad users. See the answer about solving the problem with caching... I'm going to pursue that route first. Will update with results. For what it's worth, it seems Stack Overflow is using cookies (I can't increment my own view count, but it increased when I visited the site in another browser.) I think that the benefit is just too much, and this sort of 'cheating' is just too easy right now. Thanks for the help everyone!

  Read the article

• Python interface to PayPal - urllib.urlencode non-ASCII characters failing

  - by krys

  I am trying to implement PayPal IPN functionality. The basic protocol is as such: The client is redirected from my site to PayPal's site to complete payment. He logs into his account, authorizes payment. PayPal calls a page on my server passing in details as POST. Details include a person's name, address, and payment info etc. I need to call a URL on PayPal's site internally from my processing page passing back all the params that were passed in abovem and an additional one called 'cmd' with a value of '_notify-validate'. When I try to urllib.urlencode the params which PayPal has sent to me, I get a: While calling send_response_to_paypal. Traceback (most recent call last): File "<snip>/account/paypal/views.py", line 108, in process_paypal_ipn verify_result = send_response_to_paypal(params) File "<snip>/account/paypal/views.py", line 41, in send_response_to_paypal params = urllib.urlencode(params) File "/usr/local/lib/python2.6/urllib.py", line 1261, in urlencode v = quote_plus(str(v)) UnicodeEncodeError: 'ascii' codec can't encode character u'\ufffd' in position 9: ordinal not in range(128) I understand that urlencode does ASCII encoding, and in certain cases, a user's contact info can contain non-ASCII characters. This is understandable. My question is, how do I encode non-ASCII characters for POSTing to a URL using urllib2.urlopen(req) (or other method) Details: I read the params in PayPal's original request as follows (the GET is for testing): def read_ipn_params(request): if request.POST: params= request.POST.copy() if "ipn_auth" in request.GET: params["ipn_auth"]=request.GET["ipn_auth"] return params else: return request.GET.copy() The code I use for sending back the request to PayPal from the processing page is: def send_response_to_paypal(params): params['cmd']='_notify-validate' params = urllib.urlencode(params) req = urllib2.Request(PAYPAL_API_WEBSITE, params) req.add_header("Content-type", "application/x-www-form-urlencoded") response = urllib2.urlopen(req) status = response.read() if not status == "VERIFIED": logging.warn("PayPal cannot verify IPN responses: " + status) return False return True Obviously, the problem only arises if someone's name or address or other field used for the PayPal payment does not fall into the ASCII range.

  Read the article

< Previous Page | 1 2 3 4 5 6  | Next Page >