I'm a newbie to web stuff. However, I wish to scrape some data from multiple websites. I'm currently using the following technologies:
Selenium;
Python; and
BeautifulSoup;
I believe the site I am trying to scrape is using a combination of ASP.NET, javascript and the AjaxControlToolkit. I believe the key results I am looking for are in the following script:
<script type="text/javascript">
//<![CDATA[
Sys.Application.initialize();
Sys.Application.add_init(function() {
$create(AjaxControlToolkit.AutoCompleteBehavior, {"completionInterval":50,"completionListCssClass":"autocomplete_completionListElement","completionListItemCssClass":"autocomplete_listItem","completionSetCount":20,"delimiterCharacters":"","highlightedItemCssClass":"autocomplete_highlightedListItem","id":"ctl00_ContentPlaceHolder1_AutoCompleteExtender1","minimumPrefixLength":4,"serviceMethod":"GetSchoolNames","servicePath":"AutoComplete.asmx"}, {"itemSelected":ItemSelected}, null, $get("ctl00_ContentPlaceHolder1_SchoolNameTextBox"));
});
Sys.Application.add_init(function() {
$create(AjaxControlToolkit.AutoCompleteBehavior, {"completionInterval":50,"completionListCssClass":"autocomplete_completionListElement","completionListItemCssClass":"autocomplete_listItem","delimiterCharacters":"","highlightedItemCssClass":"autocomplete_highlightedListItem","id":"ctl00_ContentPlaceHolder1_AutoCompleteExtender2","minimumPrefixLength":2,"serviceMethod":"GetSuburbNames","servicePath":"AutoComplete.asmx"}, null, null, $get("ctl00_ContentPlaceHolder1_SuburbTownTextBox"));
});
//]]>
</script>
Is there an easy way to get the results of the above script processed using Selenium so that I may pass it using BeautifulSoup?
