While working in software development in general, and in web development for a couple of customers it is quite common that it is necessary to provide a test bed where
the client is able to get an image, or better said, a feeling for
the visions and ideas you are talking about. Usually here at IOS Indian Ocean Software Ltd. we set up a demo web site on one of our staging servers, and provide credentials to
the customer to access and review our progress and work ad hoc. This gives us
the highest flexibility on both sides, as
the test bed is simply online and available 24/7. We can update
the structure,
the UI and data at any time, and
the client is able to view it as it suits best for her/him.
Limited or lack of online connectivity
But what is going to happen when your client is not capable to be online - no matter for what reasons; here are some more obvious ones:
No internet connection (permanently or temporarily)
Expensive connection, ie. mobile data package, stay at a hotel, etc.
Presentation devices at an exhibition, ie. using tablets or iPads
Being abroad for a certain time, and only occasionally online
No network coverage, especially on mobile
Bad infrastructure, like ie. in Third World countries
Providing a catalogue on CD or USB pen drive
Anyway, it doesn't matter really. We should be able to provide a solution for
the circumstances of our customers.
Presentation during an exhibition
Recently, we had
the following request from a customer:
Is it possible to let us have a desktop version of ResortWork.co.uk that we can use for demo purposes at
the forthcoming Ski Shows? It would allow us to let stand visitors browse
the sites on an iPad to view jobs and training directory course listings.
Yes, sure we can do that. Eventually, you might think why don't they simply use 3G enabled iPads for that purpose? As stated above, there might be several reasons for that - low coverage, expensive data packages, etc. Anyway, it is not a question on how to circumvent
the request but to deliver a solution to that.
Possible solutions... or not?
We already did offline websites earlier, and even established complete mirrors of one or two web sites on our systems. There are actually several possibilities to handle this kind of request, and it mainly depends on
the system or device where
the offline site should be available on. Here, it is clearly expressed that we have to address this on an Apple iPad, well actually, I think that they'd like to use multiple devices during their exhibitions.
Following is an overview of possible solutions depending on
the technology or device in use, and how it can be done:
Replication of source files and database
The above mentioned web site is running on ASP.NET, IIS and SQL Server. In case that a laptop or slate runs a Windows OS,
the easiest way would be to take a snapshot of
the source files and database, and transfer them as local installation to those Windows machines. This approach would be fully operational on
the local machine.
Saving pages for offline usage
This is actually a quite tedious job but still practicable for small web sites
Tool based approach to 'harvest'
the web site
There quite some tools in
the wild that could handle this job, namely wget, httrack, web copier, etc.
Screenshots bundled as PDF document
Not really... ;-)
Creating screencast or video
Simply navigate through your website and record your desktop session. Actually, we are using this kind of approach to track down difficult problems in order to see and understand exactly what
the user was doing to cause an error.
Of course, this list isn't complete and I'd love to get more of your ideas in
the comments section below
the article.
Preparations for offline browsing
The original website is dynamically and data-driven by ASP.NET, and looks like this:
As we have to put
the result onto iPads we are going to choose
the tool-based approach to 'download'
the whole web site for offline usage. Again, depending on
the complexity of your web site you might have to check which of
the applications produces
the best results for you. My usual choice is to use wget but in this case, we run into problems related to
the rewriting of hyperlinks. As a consequence of that we opted for using HTTrack. HTTrack comes in different flavours, like console application but also as either GUI (WinHTTrack on Windows) or Web client (WebHTTrack on Linux/Unix/BSD). Here's a brief description taken from
the original website about HTTrack:
HTTrack is a free (GPL, libre/free software) and easy-to-use offline browser utility.
It allows you to download a World Wide Web site from
the Internet to a local directory, building recursively all directories, getting HTML, images, and other files from
the server to your computer. HTTrack arranges
the original site's relative link-structure. Simply open a page of
the "mirrored" website in your browser, and you can browse
the site from link to link, as if you were viewing it online.
And there is an extensive documentation for all options and switches online. General recommendation is to go through
the HTTrack Users Guide By Fred Cohen. It covers all
the initial steps you need to get up and running. Be aware that it will take quite some time to get all
the necessary resources down to your machine. Actually, for our customer we run
the tool directly on their web server to avoid unnecessary traffic and bandwidth. After a couple of runs and some additional fine-tuning - explicit inclusion or exclusion of various external linked web sites - we finally had a more or less complete offline version available.
A very handsome feature of HTTrack is
the error/warning log after completing
the download. It contains some detailed information about errors that appeared on
the pages and
the links within
the pages that have been processed.
Error: "Bad Request" (400) at link www.resortwork.co.uk/job-details_Ski_hire:tech_or_mgr_or_driver_37854.aspx (from www.resortwork.co.uk/Jobs_A_to_Z.aspx)Error: "Not Found" (404) at link www.247recruit.net/images/applynow.png (from www.247recruit.net/css/global.css)Error: "Not Found" (404) at link www.247recruit.net/activate.html (from www.247recruit.net/247recruit_tefl_jobs_network.html)
In our situation, we took
the records of HTTP 400/404 errors and passed them to
the web development department. Improvements are to be expected soon. ;-)
Quality assurance on
the full-featured desktop
Unfortunately,
the generated output of HTTrack was still incomplete but luckily there were only images missing. Being directly on
the web server we simply copied
the missing images from
the original source folder into our offline version. After that, we created an archive and transferred
the file securely to our local workspace for further review and checks.
From that point on, it wasn't necessary to get any more files from
the original web server, and we could focus ourselves completely on
the process of browsing and navigating through
the offline version to isolate visual differences and functional problems. As said,
the original web site runs on ASP.NET Web Forms and uses Postback calls for interaction like search, pagination and partly for navigation. This is
the main field of improving
the offline experience.
Of course, same as for standard web development it is advised to test with various browsers, and strangely we discovered that
the offline version looked pretty good on Firefox, Chrome and Safari, but not in Internet Explorer. A quick look at
the HTML source shed some light on this, and there are conditional CSS inclusions based on
the user agent. HTTrack is not acting as Internet Explorer and so we didn't have
the necessary overrides for this browser. Not problematic after all in our case, but you might have to pay attention to this and get
the IE-specific files explicitly.
And while having a view at
the source code, we also found out that HTTrack actually modifies
the generated HTML output. In several occasions we discovered that <div> elements were converted into <table> constructs for no obvious reason; even nested structures.
Search 'e'nd destroy - sed (or Notepad++) to
the rescue
During our intensive root cause analysis for a couple of HTML/CSS problems that needed some extra attention it is very helpful to be familiar with any editor that allows search and replace over multiple files like, ie. sed - stream editor for filtering and transforming text on Linux or my personal favourite Notepad++ on Windows. This allowed us to quickly fix a lot of anchors with onclick attributes and Javascript code that was addressed to ASP.NET files instead of their generated HTML counterparts, like so:
grep -lr -e '.aspx' * | xargs sed -e 's/.aspx/.html?/g'
The additional question mark after
the HTML extension helps to separate
the query string from
the actual target and solved all our missing hyperlinks very fast.
The same can be done in Notepad++ on Windows, too. Just use
the 'Replace in files' feature and you are settled. Especially, in combination with Regular Expressions (regex).
Landscape of browsers
Okay, after several runs of HTML/CSS code analysis, searching and replacing some strings in a pool of more than 4.000 files, we finally had a very good match of an offline browsing experience in Firefox and Chrome on Linux. Next, we transferred that modified set of files to a Windows 8 machine for review on Firefox, Chrome and Internet Explorer 7 to 10, and a Mac mini running Mac OS X 10.7 to check
the output on Safari and again on Chrome. Besides IE, for reasons already mentioned above,
the results were identical.
And last but not least it was about to check web site on tablets. Please continue to read on
the following articles:
Taking web sites offline for demonstration on Galaxy Tablet
Taking web sites offline for demonstration on iPad
