saving / mirroring / crawling web pages that use javascript to generate content

Posted by Nick Nolan on Stack Overflow See other posts from Stack Overflow or by Nick Nolan
Published on 2009-03-24T23:07:47Z Indexed on 2010/03/17 6:31 UTC
Read the original article Hit count: 326

Filed under:

JavaScript

|

browser

|

web-crawler

I want to download web pages that use javascript to output the data. Wget can do everything else, but run javascript.

Even something like:firefox -remote "saveURL(www.mozilla.org, myfile.html)"

would be great (unfortunately that kind of command does not exist).

© Stack Overflow or respective owner

Related posts about JavaScript

CHAT ROOMs 7 by 6

as seen on Stack Overflow - Search for 'Stack Overflow'
I am looking for chatroom on one page with 7 loggedin users and 6+rows for say 42 users.these users will keep on adding wthnew users.Need urgent help.A PRETTY UNUSUAL Q FOR MOST OF U.What is MORE REQ new features: Usernames are unique to users currently chatting You can see a "currently chatting"… >>> More
Integrating JavaScript Unit Tests with Visual Studio

as seen on Stephen Walter - Search for 'Stephen Walter'
Modern ASP.NET web applications take full advantage of client-side JavaScript to provide better interactivity and responsiveness. If you are building an ASP.NET application in the right way, you quickly end up with lots and lots of JavaScript code. When writing server code, you should be writing… >>> More
Has Javascript developed beyond what it was originally designed to do?

as seen on Programmers - Search for 'Programmers'
I've been talking with a friend about the purpose of Javascript, when and how it should be used, etc. He quoted that: JavaScript was designed to add interactivity to HTML pages [...] JavaScript gives HTML designers a programming tool HTML authors are normally not programmers… >>> More
PHP, javascript, single quote problems with IE when passing variable from ajax post to javascript fu

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi! I have been trying to get this to work for a while, and I suspect there's an easy solution that I just can't find. My head feels like jelly and I would really appreciate any help. My main page.php makes a .post() to backend.php and fetches a list of cities which it echoes in the form of: <li… >>> More
Javascript in XSL that is loaded by Javascript

as seen on Stack Overflow - Search for 'Stack Overflow'
Is there anyway to have javascript run when a XSL sheet has been applied to an XML file by Javascript? I am using a JQuery plugin to apply the sheet to the xml but the javascript that is located inside of the XSL file will not run. I put the Javascript at the bottom of the file and it still does… >>> More

Related posts about browser

Couldn't attachto Firefox 3.x browser by using Browser.AttachTo<FireFox>method in WatiN 2.0 RC1

as seen on Stack Overflow - Search for 'Stack Overflow'
I am using HTTPWatch automation API to launch a new Firefox instance like that: HttpWatch.Controller ct = new HttpWatch.Controller(); HttpWatch.Plugin plugin = ct.FireFox.New(""); plugin.GotoURL("http://www.google.com"); These codes could start a Firefox browser successfully. Then I want to control… >>> More
Programmatically enable Javascript in Blackberry browser (not in Browser Field)

as seen on Stack Overflow - Search for 'Stack Overflow'
Is there a way to programmatically enable Javascript in Blackberry Browser? In my application I have to open an URL in Blackberry browser but for that I need to first enable Javascript. I do not wish to use Browser Field as the content displayed in it doesn't look good. Any help in this regard will… >>> More
The Windows Browser Ballot Screen Offers Web Browser Choice to European Users

as seen on How to geek - Search for 'How to geek'
Since March, our friends across the pond in Europe get to decide which browser they want to install with their Windows OS. Today we thought we would take a look at the ballot choices, some are well known, and others you may not have heard of. Windows users in European countries should start seeing… >>> More
To Change the Default web browser to other Browser in Asp.net

as seen on C# Corner - Search for 'C# Corner'
You can change the default web browser(IE,Opera,Safari) during web development in VS environment. you can test web page in different browser during development. >>> More
Use ASP.NET 4 Browser Definitions with ASP.NET 3.5

as seen on Stephen Walter - Search for 'Stephen Walter'
We updated the browser definitions files included with ASP.NET 4 to include information on recent browsers and devices such as Google Chrome and the iPhone. You can use these browser definition files with earlier versions of ASP.NET such as ASP.NET 3.5. The updated browser definition files, and instructions… >>> More