Search Results

Search found 101 results on 5 pages for 'mechanize'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 2/5 | < Previous Page | 1 2 3 4 5  | Next Page >

  • Get string value from http response with Mechanize

    - by Gearóid
    Hi, I'm currently integrating facebook into my current app and I've succeeded in retrieving the access_token using the following code: url="#{url}?#{client_id}&#{client_secret}&#{code}&#{redirect_uri}&type=client_cred" agent = Mechanize.new page = agent.get(url) The page object above has a body which contains text something along the lines of access_token=XXXXX I just want to pull out the access_token value. I can get the entire string simply by writing: page.body But I was wondering is there a way to get the access_token value without resorting to regular expressions etc? Thanks.

    Read the article

  • Python mechanize - two buttons of type 'submit'

    - by directedition
    I have a mechanize script written in python that fills out a web form and is supposed to click on the 'create' button. But there's a problem, the form has two buttons. One for 'add attached file' and one for 'create'. Both are of type 'submit', and the attach button is the first one listed. So when I select the forum and do br.submit(), it clicks on the 'attach' button instead of 'create'. Extensive Googling has yielded nothing useful for selecting a specific button in a form. Does anyone know of any methods for skipping over the first 'submit' button and clicking the second?

    Read the article

  • How do I debug a Net::HTTPInternalServerError error when using Mechanize?

    - by Janak
    c:/ruby/lib/ruby/gems/1.8/gems/mechanize-1.0.0/lib/mechanize.rb:259:in `get': 500 => Net::HTTPInternalServerError (Mechanize::ResponseCodeError) I get the above error when I try to navigate to the following webpage http://fakewebsite.com//admin/edit_building.cfm?page=buildings&updateMode=yes&id=1251 I can navigate just fine when copy and paste the link into a browser. Note: This website does require a login, which I handle with code like the following $agent = Mechanize.new $agent.get('http://fakewebsite//admin/login.cfm?res=-5') form = $agent.page.forms.first form.EMail = "admin" form.Password = "password" form.submit This login works just fine, since I access other webpages protected by the login. I'm not sure where to go from here, any suggestions?

    Read the article

  • How can I get WWW-Mechanize to login to Wells Fargo's website?

    - by J Miller
    I am trying to use Perl's WWW::Mechanize to login to my bank and pull transaction information. After logging in through a browser to my bank (Wells Fargo), it briefly displays a temporary web page saying something along the lines of "please wait while we verify your identity". After a few seconds it proceeds to the bank's webpage where I can get my bank data. The only difference is that the URL contains several more "GET" parameters appended to the URL of the temporary page, which only had a sessionID parameter. I was able to successfully get WWW::Mechanize to login from the login page, but it gets stuck on the temporary page. There is a <meta http-equiv="Refresh"... tag in the header, so I tried $mech->follow_meta_redirect but it didn't get me past that temporary page either. Any help to get past this would be appreciated. Thanks in advance. Here is the barebones code that gets me stuck at the temporary page: #!/usr/bin/perl -w use strict; use WWW::Mechanize; my $mech = WWW::Mechanize->new(); $mech->agent_alias( 'Linux Mozilla' ); $mech->get( "https://www.wellsfargo.com" ); $mech->submit_form( form_number => 2, fields => { userid => "$userid", password => "$password" }, button => "btnSignon" );

    Read the article

  • how to manually add a cookie to Mechanize state?

    - by fearless_fool
    [I'm working in Ruby, but my question is valid for other languages as well.] I have a Mechanize-driven application. The server I'm talking to sets a cookie using javascript (rather than standard set-cookie), so Mechanize doesn't catch the cookie. I need to pass that cookie back on the next GET request. The good news is that I already know the value of the cookie, but I don't know how to tell Mechanize to include it in my next GET request. Pointers or suggestions are welcome. Thanks in advance. -ff

    Read the article

  • How can I make WWW:Mechanize to not fetch pages twice?

    - by planetp
    I have a web scraping application, written in OO perl. There's single WWW::Mechanize object used in the app. How can I make it to not fetch the same url twice, i.e. make the second get() with the same url nop: my $mech = WWW::Mechanize->new(); my $url = 'http:://google.com'; $mech->get( $url ); # first time, fetch $mech->get( $url ); # same url, do nothing

    Read the article

  • What is the preferred method of accessing WWW::Mechanize responses?

    - by sid_com
    Hello! Are both of these versions OK or is one of them to prefer? #!/usr/bin/env perl use strict; use warnings; use WWW::Mechanize; my $mech = WWW::Mechanize->new(); my $content; # 1 $mech->get( 'http://www.kernel.org' ); $content = $mech->content; say $content; # 2 my $res = $mech->get( 'http://www.kernel.org' ); $content = $res->content; say $content;

    Read the article

  • How can I select a value matching my input from a TypeAhead control using WWW::Mechanize?

    - by kyoob
    So ordinarily the element I want to pick from a list would be populated on a page and I'd just find it and pick it. But I'm dealing with a control that doesn't populate list elements until some input has been stuck into a text box, after which it gives me a list of recommendations. For an example of the kind of list I'm talking about think of Facebook's "People, Places, and Things" search field. I want to plug a string into this text box, select the same string from the list of recommendations, and submit the form. The issue I'm having right now is I can't seem to get Mechanize to even recognize the field is there. I examine a dump of $mech->find_all_inputs and it isn't listed. Is this kind of field just beyond Mechanize's jurisdiction?

    Read the article

  • Python - Problems using mechanize to log into a difficult website

    - by user1781599
    × 139886 I am trying to log in to betfair.com by using mechanize. I have tried several ways but it always fail. This is the code I have developed so far, can anyone help me to identify what is wrong with it and how I can improve it to log into my betfair account? Thanks, import cookielib import urllib import urllib2 from BeautifulSoup import BeautifulSoup import mechanize from mechanize import Browser import re bf_username_name = "username" bf_password_name = "password" bf_form_name = "loginForm" bf_username = "xxxxx" bf_password = "yyyyy" urlLogIn = "http://www.betfair.com/" accountUrl = "https://myaccount.betfair.com/account/home?rlhm=0&" # This url I will use to verify if log in has been successful br = mechanize.Browser(factory=mechanize.RobustFactory()) br.addheaders = [("User-Agent","Mozilla/5.0 (Macintosh; Intel Mac OS X 10_5_8) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.90 Safari/537.1")] br.open(urlLogIn) br.select_form(nr=0) print br.form br.form[bf_username_name] = bf_username br.form[bf_password_name] = bf_password print br.form #just to check username and psw have been recorded correctly responseSubmit = br.submit() response = br.open(accountUrl) text_file = open("LogInResponse.html", "w") text_file.write(responseSubmit.read()) #this file should show the home page with me logged in, but it show home page as if I was not logged it text_file.close() text_file = open("Account.html", "w") text_file.write(response.read()) #this file should show my account page, but it should a pop up with an error text_file.close()

    Read the article

  • Perl - WWW::Mechanize Cookie Session Id is being reset with every get(), how to make it stop?

    - by Phill Pafford
    So I'm scraping a site that I have access to via HTTPS, I can login and start the process but each time I hit a new page (URL) the cookie Session Id changes. How do I keep the logged in Cookie Session Id? #!/usr/bin/perl -w use strict; use warnings; use WWW::Mechanize; use HTTP::Cookies; use LWP::Debug qw(+); use HTTP::Request; use LWP::UserAgent; use HTTP::Request::Common; my $un = 'username'; my $pw = 'password'; my $url = 'https://subdomain.url.com/index.do'; my $agent = WWW::Mechanize->new(cookie_jar => {}, autocheck => 0); $agent->{onerror}=\&WWW::Mechanize::_warn; $agent->agent('Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.3) Gecko/20100407 Ubuntu/9.10 (karmic) Firefox/3.6.3'); $agent->get($url); $agent->form_name('form'); $agent->field(username => $un); $agent->field(password => $pw); $agent->click("Log In"); print "After Login Cookie: "; print $agent->cookie_jar->as_string(); print "\n\n"; my $searchURL='https://subdomain.url.com/search.do'; $agent->get($searchURL); print "After Search Cookie: "; print $agent->cookie_jar->as_string(); print "\n"; The output: After Login Cookie: Set-Cookie3: JSESSIONID=367C6D; path="/thepath"; domain=subdomina.url.com; path_spec; secure; discard; version=0 After Search Cookie: Set-Cookie3: JSESSIONID=855402; path="/thepath"; domain=subdomain.com.com; path_spec; secure; discard; version=0 Also I think the site requires a CERT (Well in the browser it does), would this be the correct way to add it? $ENV{HTTPS_CERT_FILE} = 'SUBDOMAIN.URL.COM'; ## Insert this after the use HTTP::Request... Also for the CERT In using the first option in this list, is this correct? X.509 Certificate (PEM) X.509 Certificate with chain (PEM) X.509 Certificate (DER) X.509 Certificate (PKCS#7) X.509 Certificate with chain (PKCS#7)

    Read the article

  • Why am I getting a new session ID on every page fetch in my Perl WWW::Mechanize script?

    - by Phill Pafford
    So I'm scraping a site that I have access to via HTTPS, I can login and start the process but each time I hit a new page (URL) the cookie Session Id changes. How do I keep the logged in Cookie Session Id? #!/usr/bin/perl -w use strict; use warnings; use WWW::Mechanize; use HTTP::Cookies; use LWP::Debug qw(+); use HTTP::Request; use LWP::UserAgent; use HTTP::Request::Common; my $un = 'username'; my $pw = 'password'; my $url = 'https://subdomain.url.com/index.do'; my $agent = WWW::Mechanize->new(cookie_jar => {}, autocheck => 0); $agent->{onerror}=\&WWW::Mechanize::_warn; $agent->agent('Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.3) Gecko/20100407 Ubuntu/9.10 (karmic) Firefox/3.6.3'); $agent->get($url); $agent->form_name('form'); $agent->field(username => $un); $agent->field(password => $pw); $agent->click("Log In"); print "After Login Cookie: "; print $agent->cookie_jar->as_string(); print "\n\n"; my $searchURL='https://subdomain.url.com/search.do'; $agent->get($searchURL); print "After Search Cookie: "; print $agent->cookie_jar->as_string(); print "\n"; The output: After Login Cookie: Set-Cookie3: JSESSIONID=367C6D; path="/thepath"; domain=subdomina.url.com; path_spec; secure; discard; version=0 After Search Cookie: Set-Cookie3: JSESSIONID=855402; path="/thepath"; domain=subdomain.com.com; path_spec; secure; discard; version=0 Also I think the site requires a CERT (Well in the browser it does), would this be the correct way to add it? $ENV{HTTPS_CERT_FILE} = 'SUBDOMAIN.URL.COM'; ## Insert this after the use HTTP::Request... Also for the CERT In using the first option in this list, is this correct? X.509 Certificate (PEM) X.509 Certificate with chain (PEM) X.509 Certificate (DER) X.509 Certificate (PKCS#7) X.509 Certificate with chain (PKCS#7)

    Read the article

  • Get Mechanize to handle cookies from an arbitrary POST (to log into https://www.t-mobile.com/ progra

    - by Horace Loeb
    I want to log into https://www.t-mobile.com/ programmatically. My first idea was to use Mechanize to submit the login form: However, it turns out that this isn't even a real form. Instead, when you click "Log in" some javascript grabs the values of the fields, creates a new form dynamically, and submits it. "Log in" button HTML: <button onclick="handleLogin(); return false;" class="btnBlue" id="myTMobile-login"><span>Log in</span></button> The handleLogin() function: function handleLogin() { if (ValidateMsisdnPassword()) { // client-side form validation logic var a = document.createElement("FORM"); a.name = "form1"; a.method = "POST"; a.action = mytmoUrl; // defined elsewhere as https://my.t-mobile.com/Login/LoginController.aspx var c = document.createElement("INPUT"); c.type = "HIDDEN"; c.value = document.getElementById("myTMobile-phone").value; // the value of the phone number input field c.name = "txtMSISDN"; a.appendChild(c); var b = document.createElement("INPUT"); b.type = "HIDDEN"; b.value = document.getElementById("myTMobile-password").value; // the value of the password input field b.name = "txtPassword"; a.appendChild(b); document.body.appendChild(a); a.submit(); return true } else { return false } } I could simulate this form submission by POSTing the form data to https://my.t-mobile.com/Login/LoginController.aspx with Net::HTTP#post_form, but I don't know how to get the resultant cookie into Mechanize so I can continue to scrape the UI available when I'm logged in. Any ideas?

    Read the article

  • How can I handle UTF-8 while posting to a vBulletin board with WWW::Mechanize?

    - by MrMirror
    I have a problem with some automating posting to bulletin board... If I send the posting form to the vBulletin board, I get corrupted entities. Feel free to copy-paste the script and try it... It looks like the board's expecting some decoded utf8, but if I send the message decoded the entities are still wrong. #!/usr/bin/perl use strict; use warnings; use WWW::Mechanize; use Digest::MD5 qw(md5_hex); my $mech = WWW::Mechanize->new(); my $base_url = 'http://www.boerse.bz/'; my $username = 'MrMirror'; my $password = 'test'; $mech->get($base_url); print "Login\n"; $mech->form_number(1); $mech->field('vb_login_username' => $username); $mech->field('vb_login_password' => $password); $mech->field('vb_login_md5password' => md5_hex($password)); $mech->field('vb_login_md5password_utf' => md5_hex($password)); $mech->submit(); unless ($mech->content() =~ m!Weiterleitung!gi) { print "No Rediction!\n"; exit; } print "Redict\n"; $mech->get($base_url); unless ($mech->content() =~ m!Logout!gi) { print "Login Failed!\n"; exit; } $mech->get($base_url .'/newthread.php?do=newthread&f=173'); $mech->form_number(3); $mech->field('subject' => 'MrMirror makes some testing ä ö ü ß'); $mech->field('message' => "ä ö ü ß"); ### everything allright here $mech->dump_forms(); ### preview submit, don't wanna spam around ;) $mech->click('preview'); print "\n\n\n---------------------------------------------------------------------\n\n\n"; ### same form, wrong entities :( $mech->dump_forms();

    Read the article

  • python urllib post question

    - by paul
    hello ALL im making some simple python post script but it not working well. there is 2 part to have to login. first login is using 'http://mybuddy.buddybuddy.co.kr/userinfo/UserInfo.asp' this one. and second login is using 'http://user.buddybuddy.co.kr/usercheck/UserCheckPWExec.asp' i can login first login page, but i couldn't login second page website. and return some error 'illegal access' such like . i heard this is related with some cooke but i don't know how to implement to resolve this problem. if anyone can help me much appreciated!! Thanks! import re,sys,os,mechanize,urllib,time import datetime,socket params = urllib.urlencode({'ID':'ph896011', 'PWD':'pk1089' }) rq = mechanize.Request("http://mybuddy.buddybuddy.co.kr/userinfo/UserInfo.asp", params) rs = mechanize.urlopen(rq) data = rs.read() logged_fail = r';history.back();</script>' in data if not logged_fail: print 'login success' try: params = urllib.urlencode({'PASSWORD':'pk1089'}) rq = mechanize.Request("http://user.buddybuddy.co.kr/usercheck/UserCheckPWExec.asp", params ) rs = mechanize.urlopen(rq) data = rs.read() print data except: print 'error'

    Read the article

  • Mechanize form submit not going to the correct response page, no errors as to why. Something I'm doing?

    - by Zack Shapiro
    I threw this all in one controller for testing purposes. My code fills out the form correctly for adding a new address to your Amazon account. There are two buttons that submit this form, one takes you to add a new address which is what I don't want, and the other is just a Save & Continue input/image. When I submit the form via that button, as I do below, the form is still on the page, filled out as I have with my code. Inspecting the page titles, they're the same. There are no discernible errors that Mechanize or Amazon spit back. Any ideas? class AmazonCrawler def initialize @agent = Mechanize.new do |agent| agent.user_agent_alias = 'Mac Safari' agent.follow_meta_refresh = true agent.redirect_ok = true end end def login # login_url = "https://www.amazon.com/ap/signin?_encoding=UTF8&openid.assoc_handle=usflex&openid.claimed_id=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.identity=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0%2Fidentifier_select&openid.mode=checkid_setup&openid.ns=http%3A%2F%2Fspecs.openid.net%2Fauth%2F2.0&openid.ns.pape=http%3A%2F%2Fspecs.openid.net%2Fextensions%2Fpape%2F1.0&openid.pape.max_auth_age=0&openid.return_to=https%3A%2F%2Fwww.amazon.com%2Fgp%2Fcss%2Fhomepage.html%3Fie%3DUTF8%26ref_%3Dgno_yam_ya" login_url = "https://www.amazon.com/gp/css/account/address/view.html?ie=UTF8&ref_=ya_add_address&viewID=newAddress" @agent.get(login_url) form = @agent.page.forms.first form.email = "[email protected]" form.radiobuttons.first.value = "0" form.radiobuttons.last.check form.password = "my_password" dashboard = @agent.submit(form) end end class UsersController < ApplicationController def index response = AmazonCrawler.new.login form = response.forms[1] # fill out form form.enterAddressFullName == "Your Name" form.enterAddressAddressLine1 = "123 Main Street" form.enterAddressAddressLine2 = "Apartment 34" form.enterAddressCity = "San Francisco" form.enterAddressStateOrRegion = "CA" form.enterAddressPostalCode = "94101" form.enterAddressPhoneNumber = "415-555-1212" form.AddressType = "RES" new_response = form.submit( form.button_with(value: /Save.*Continue/) ) end end

    Read the article

  • Ruby encoding problem

    - by Fossmo
    I'm just starting to learn Ruby and have a problem with encoding; require 'rubygems' require 'mechanize' agent = Mechanize.new agent.get('myurl.....') agent.page.search('#reciperesult a').each do |item| c = Mechanize.new c.get(item.attributes['href']) puts c.page.search('#ingredients li').text end The output text are shown like this h+©nsekj+©tt when it should have been shown like this hønsekjøtt. I'm using Ruby 1.8.7. Can anybody point me in the right direction?

    Read the article

  • find_all_links() Perl function don't find all links

    - by Malincy Montoya
    I'm starting with Perl and I'm trying to do a script that log me out of a page (I already did the login one). The think is, I'm trying to use find_link function from WWW::Mechanize (I also tried WWW::Mechanize::Firefox) to find the logout link, but it's not working. So, my question is: is the event onmouseover avoiding that link to be founded? I think WWW::Mechanize::Firefox supports javascript but maybe it's not correct. Logout Is there a way to do what I want? Any help or suggestions will be highly appreciated.

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 1 2 3 4 5  | Next Page >