How to reset Scrapy parameters? (always running under same parameters)
- by Jean Ventura
I've been running my Scrapy project with a couple of accounts (the project scrapes a especific site that requieres login credentials), but no matter the parameters I set, it always runs with the same ones (same credentials).
I'm running under virtualenv. Is there a variable or setting I'm missing?
Edit:
It seems that this problem is Twisted related.
Even when I run:
scrapy crawl -a user='user' -a password='pass' -o items.json -t json SpiderName
I still get an error saying:
ERROR: twisted.internet.error.ReactorNotRestartable
And all the information I get, is the last 'succesful' run of the spider.