I'm trying to use the Django ORM in some standalone screen scraping scripts. I know this question has been asked before, but I'm unable to figure out a good solution for my particular problem.
I have a Django project with defined models. What I would like to do is use these models and the ORM in my scraping script. My directory structure is something like this:
project
scrape
#scraping scripts
...
test.py
web
django_project
settings.py
...
#Django files
I tried doing the following in project/scrape/test.py:
print os.path.join(os.path.abspath('..'), 'web', 'django_project')
sys.path.append(os.path.join(os.path.abspath('..'), 'web', 'django_project'))
print sys.path
print "-------"
os.environ['DJANGO_SETTINGS_MODULE'] = 'django_project.settings'
#print os.environ
from django_project.myapp.models import MyModel
print MyModel.objects.count()
However, I get an ImportError when I try to run test.py:
Traceback (most recent call last):
File "test.py", line 12, in <module>
from django_project.myapp.models import MyModel
ImportError: No module named django_project.myapp.models
One solution I found around this problem is to create a symbolic link to ../web/govcheck in the scrape folder:
:scrape rmanocha$ ln -s ../web/govcheck ./govcheck
With this, I can then run test.py just fine. However, this seems like a hack, and more importantly, is not very portable (I will have to create this symbolic link everywhere I run this code).
So, I was wondering if anyone has any better solutions for my problem?
