Python lxml - returns null list
- by Chris Finlayson
I cannot figure out what is wrong with the XPATH when trying to extract a value from a webpage table. The method seems correct as I can extract the page title and other attributes, but I cannot extract the third value, it always returns an empty list?
from lxml import html
import requests
test_url = 'SC312226'
page = ('https://www.opencompany.co.uk/company/'+test_url)
print 'Now searching URL: '+page
data = requests.get(page)
tree = html.fromstring(data.text)
print tree.xpath('//title/text()') # Get page title
print tree.xpath('//a/@href') # Get href attribute of all links
print tree.xpath('//*[@id="financial"]/table/tbody/tr/td[1]/table/tbody/tr[2]/td[1]/div[2]/text()')
Unless i'm missing something, it would appear the XPATH is correct:
Chrome screenshot
I checked Chrome console, appears ok! So i'm at a loss
$x ('//*[@id="financial"]/table/tbody/tr/td[1]/table/tbody/tr[2]/td[1]/div[2]/text()')
[
"£432,272"
]
