GAE uploading to database - or - how to create big dictionary in GAE?
- by Koran
Hi,
I have a GAE application. A new request is to have a huge database to store many items for some functionality.
The database needs to populated every week or so - and populating it is done by querying many external links which overall takes around 1-2 hours. So, doing it via GAE Cron is now out of the question.
Another option I was thinking was to create it in external server and then uploading it to GAE DB. Is it possible to upload the data created externally to GAE DB straightaway? The question is because there will be close to 10 M items in it, so adding items individually to a DB is not going to work due to 30 second time limit.
Thanks
K