wget mirroring, subdomains and directories and cookies
Posted
by Jimmu
on Server Fault
See other posts from Server Fault
or by Jimmu
Published on 2010-02-03T07:14:47Z
Indexed on
2010/03/12
11:08 UTC
Read the original article
Hit count: 590
Hi all,
I have an account on a web page that is now "full" (ie I have used up all my allocated space) and I would like to make a mirror of that site. wget seems like the thing to use.
The problem is that I would only like to mirror the sites the lie within this directory http://user.domain.com/room/2324343/transcript/ (and sub-directories). Whilst saving the correct stylesheets, javascripts and css etc which exist in different directories.
There as also uploaded files that are linked to within the pages in the transcript directory (on different directories) that I would like to download/mirror (theses are in a variatey of formats .exe, .py, .png, .app (and many more)). There are also images that are on different severs that are on these pages.
Also I would like it if the links (which are sometimes relative , sometimes absoulute (but to internal things), sometimes external ) worked correctly so that if they link to things that have been downloaded(mirrored) they work fine (without internet connection), but if they link to things that are external or havent been mirrored they link to the external site. Basically so they work as expected.
Another problem is that you have to log in to acess the site.
Can wget be used to acomplish this or is there a better way?
either way how do I achive this?
(I have asked this question at stackoverflow.com/questions/2190115/wget-mirroring-subdomains-and-directories-and-cookies but it was recommended that I try asking it here)
© Server Fault or respective owner