Python: Is there a way to get HTML that was dynamically created by Javascript?
Posted
by
Joschua
on Stack Overflow
See other posts from Stack Overflow
or by Joschua
Published on 2014-08-25T16:16:10Z
Indexed on
2014/08/25
16:19 UTC
Read the original article
Hit count: 290
As far as I can tell, this is the case for LyricWikia. The lyrics (example) can be accessed from the browser, but can't be found in the source code (can be opened with CTRL + U in most browsers) or reading the contents of the site with Python:
from urllib.request import urlopen
URL = 'http://lyrics.wikia.com/Billy_Joel:Piano_Man'
r = urlopen(URL).read().decode('utf-8')
And the test:
>>> 'Now John at the bar is a friend of mine' in r
False
>>> 'John' in r
False
But when you select and look at the source code of the box in which the lyrics are displayed, you can see that there is: <div class="lyricbox">[...]</div>
Is there a way to get the contents of that div-element with Python?
© Stack Overflow or respective owner
