In the larger scale of my program the goal of the below code is to filter out all dynamic html in a web-page source code
code snippet:
try:
deepreq3 = urllib.request.Request(deepurl3)
deepreq3.add_header("User-Agent","etc......")
deepdata3 = urllib.request.urlopen(deepurl3).read().decode("utf8", 'ignore')
The following code is looped 3 times in order to identify whether the target web-page is Dynamic (source code is changed at intervals) or not.
If the page IS dynamic, the above code loops another 15 times and attempts to filter out the dynamic content.
QUESTION:
While this filtering method works 80% of the time, some pages will reload ALL 15 times and STILL contain dynamic code. HOWEVER. If I manually close down the Python Shell and re-execute my program, the dynamic html that my 'refresh-page method' could not shake off is no longer there ... it's been replaced with new dynamic html that my 'refresh-page method' cannot shake off. So I need to know, what is going on here? How is re-running my program causing the dynamic content of a page to change. AND, is there any way, any 'reset connection' command I can use to recreate this ... without manually restarting my app.
Thanks for your response.
