How to loop over nodes with xmlfeed using scrapy python
Posted
by
Kour ipm
on Stack Overflow
See other posts from Stack Overflow
or by Kour ipm
Published on 2012-09-04T15:23:40Z
Indexed on
2012/09/04
15:38 UTC
Read the original article
Hit count: 271
Hi i working on scrapy and trying xml feeds first time, below is my code
class TestxmlItemSpider(XMLFeedSpider):
name = "TestxmlItem"
allowed_domains = {"http://www.nasinteractive.com"}
start_urls = [
"http://www.nasinteractive.com/jobexport/advance/hcantexasexport.xml"
]
iterator = 'iternodes'
itertag = 'job'
def parse_node(self, response, node):
title = node.select('title/text()').extract()
job_code = node.select('job-code/text()').extract()
detail_url = node.select('detail-url/text()').extract()
category = node.select('job-category/text()').extract()
print title,";;;;;;;;;;;;;;;;;;;;;"
print job_code,";;;;;;;;;;;;;;;;;;;;;"
item = TestxmlItem()
item['title'] = node.select('title/text()').extract()
.......
return item
result:
File "/usr/lib/python2.7/site-packages/Scrapy-0.14.3-py2.7.egg/scrapy/item.py", line 56, in __setitem__
(self.__class__.__name__, key))
exceptions.KeyError: 'TestxmlItem does not support field: title'
Totally there are 200+ items so i need to loop over and assign the node text to item
but here all the results are displaying at once when we print, actually how can we loop over on nodes in scraping xml files with xmlfeedspider
© Stack Overflow or respective owner