implementing dynamic query handler on historical data
- by user2390183
EDIT : Refined question to focus on the core issue
Context:
I have historical data about property (house) sales collected from various sources in a centralized/cloud data source (assume info collection is handled by a third party)
Planning to develop an application to query and retrieve data from this centralized data source
Example Queries:
Simple : for given XYZ post code, what is average house price for 3 bed room house?
Complex: What is estimated price for an house at "DD,Some Street,XYZ Post Code" (worked out from average values of historic data filtered by various characteristics of the house: house post code, no of bed rooms, total area, and other deeper insights like house building type, year of built, features)?
In addition to average price, the application should support other property info ** maximum, or minimum price..etc and trend (graph) on a selected property attribute over a period of time**. Hence, the queries should not enforce the search based on a primary key or few fixed fields
In other words, queries can be
What is the change in 3 Bed Room house price (irrespective of location) over last 30 days?
What kind of properties we can get for X price (irrespective of location or house type)
The challenge I have is identifying the domain (BI/ Data Analytical or DB Design or DB Query Interface or DW related or something else) this problem (dynamic query on historic data) belong to, so that I can do further exploration
My findings so far
I could be wrong on the following, so please correct me if you think so
I briefly read about BI/Data Analytics - I think it is heavy weight solution for my problem and has scalability issues.
DB Design - As I understand RDBMS works well if you know Data model at design time. I am expecting attributes about property or other entity (user) that am going to bring in, would evolve quickly. hence maintenance would be an issue. As I am going to have multiple users executing query at same time, performance would be a bottleneck
Other options like Graph DB (http://www.tinkerpop.com/) seems to be bit complex (they are good. but using those tools meant for generic purpose, make me think like assembly programming to solve my problem )
BigData related solution are to analyse data from multiple unrelated domains
So,
Any suggestion on the space this problem fit in ? (Especially if you have design/implementation experience of back-end for property listing or similar portals)
