Guidance required: FIrst time gonna work with real high end database (size = 50GB).

Posted by claws on Stack Overflow See other posts from Stack Overflow or by claws
Published on 2010-04-02T06:29:02Z Indexed on 2010/04/02 6:33 UTC
Read the original article Hit count: 213

Filed under:

database-design

|

database

|

mysql

I got a project of designing a Database. This is going to be my first big scale project. Good thing about it is information is mostly organized & currently stored in text files. The size of this information is 50GB. There are going to be few millions of records in each Table. Its going to have around 50 tables. I need to provide a web interface for searching & browsing. I'm going to use MySQL DBMS.

I've never worked with a database more than 200MB before. So, speed & performance was never a concern but I followed things like normalization & Indexes. I never used any kind of testing/benchmarking/queryOptimization/whatever because I never had to care about them.

But here the purpose of creating a database is to make it quickly searchable. So, I need to consider all possible aspects in design.

I was browsing archives & found:

http://stackoverflow.com/questions/1981526/what-should-every-developer-know-about-databases

http://stackoverflow.com/questions/621884/database-development-mistakes-made-by-app-developers

I'm gonna keep the points mentioned in above answers in mind.

What else should I know? What else should I keep in mind?

© Stack Overflow or respective owner

Related posts about database-design

(Database Design - products attributes): What is better option for product attribute database design

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi, I new in database design. What is better option for product attribute database design for cms?(Please suggest other options also). option 1: 1 table products{ id product_name color price attribute_name1 attribute_value1 attribute_name2 attribute_value2 attribute_name3 attribute_value3 } option… >>> More
Book Review: Pro SQL Server 2008 Relational Database Design and Implementation

as seen on SQL Blog - Search for 'SQL Blog'
Investing in proper database design is a very efficient way to cut maintenance costs. If we expect a system to last, we need to make sure it has a good solid foundation - high quality database design. Surely we can and sometimes do cut corners and save on database design to get things done faster… >>> More
Advice on database design / SQL for retrieving data with chronological order

as seen on Stack Overflow - Search for 'Stack Overflow'
I am creating a database that will help keep track of which employees have been on a certain training course. I would like to get some guidance on the best way to design the database. Specifically, each employee must attend the training course each year and my database needs to keep a history of… >>> More
Fiscal year handling strategies in database design

as seen on Stack Overflow - Search for 'Stack Overflow'
By fiscal year I mean all the data in the database (in all tables) that occurred in the particular year. Lets say that we are building an application that allows user to choose from different years. What way of implementing this would you prefer, and why: Separate fiscal year data based on multiple… >>> More
Database design for summarized data

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a new table I'm going to add to a bunch of other summarized data, basically to take some of the load off by calculating weekly avgs. My question is whether I would be better off with one model over the other. One model with days of the week as a column with an additional column for price… >>> More

Related posts about database

SQL SERVER Retrieve and Explore Database Backup without Restoring Database Idera virtual database

as seen on Dot net Slackers - Search for 'Dot net Slackers'
I recently downloaded Ideras SQL virtual database, and tested it. There are a few things about this tool which caught my attention.My ScenarioIt is quite common in real life that sometimes observing or retrieving older data is necessary; however, it had changed as time passed by. The full database… >>> More
Cloning A Database On The Same Server Using Rman Duplicate From Active Database

as seen on Oracle Blogs - Search for 'Oracle Blogs'
To clone a database using Rman we used to require an existing Rman backup, on 11g we can clone databases using the "from active" database option. In this case we do not require an existing backup, the active datafiles will be used as the source for the clone. In order to clone with the source database… >>> More
cPickle ImportError: No module named multiarray

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello, I'm using cPickle to save my Database into file. The code looks like that: def Save_DataBase(): import cPickle from scipy import * from numpy import * a=Results.VersionName #filename='D:/results/'+a[a.find('/')+1:-a.find('/')-2]+Results.AssType[:3]+str(random.randint(0,100))+Results.Distribution+"… >>> More
SQL SERVER – 2008 – Introduction to Snapshot Database – Restore From Snapshot

as seen on SQL Authority - Search for 'SQL Authority'
Snapshot database is one of the most interesting concepts that I have used at some places recently. Here is a quick definition of the subject from Book On Line: A Database Snapshot is a read-only, static view of a database (the source database). Multiple snapshots can exist on a source database and… >>> More
OTN ???? ?????? ???????

as seen on Oracle Blogs - Search for 'Oracle Blogs'
Database ?? Database ??????? Database ?????????? Java WebLogic Server/????????·???? SOA/BPM/????? ???????/???? ID??/?????? ?????EPM/BI EPM/BI ??????? EPM/BI ???? OS/??? ???? ????? MySQL Database ?? ???? ?? ????????? ??? ?? ORACLE MASTER… >>> More