I have tried to design the datastore schema for a very small application. That schema would have been very simple, if not trivial, using a relational database with foreign keys, many-to-many relations, joins, etc.
But the problem was that my application was targeted for Google App Engine and I had to design for a database that was not relational.…
Sorry if this question is somewhat subjective. I am new to 'could store', 'distributed store' or some concepts like this. I really wonder what do they have in common and want to get an overview on all of them. What do I need to prepare if I want to write a product similar to this?
We are looking for a suitable storage engine for our weblog history data. We looked at Bigtable's paper and understand it is suitable to us well.
However, I also understand that Document-oriented-DB such as MongoDB seems to provide a little more powerful schema power -- i.e, it can model our data as well.
I wonder how nowadays ppl choose a…
By what factor does the performance (read queries/sec) increase when a machine is added to a cluster of machines running either:
a Bigtable-like database
MySQL?
Google's research paper on Bigtable suggests that "near-linear" scaling is achieved can be achieved with Bigtable. This page here featuring MySQL's marketing jargon suggests…
I've been looking into writing a web app that will run on Google App Engine, but before I commit myself to the platform I'd like to know what, if any, limitations there are. I'm aware of the basic CPU/bandwidth restrictions that Google places on the free service, but I'm wondering more about development restrictions like how BigTable…
We have run some scripts that use LogParser to dump our IIS logs into a SQL Server database.
We can then query this to get simple stats on hits, usage etc. It's also good when linking it to error log databases and performance counter database to compare usage with errors, etc.
Having implemented this for just one system and for the…
We are using a mysql database w/ about 150,000 records (names) total. Our searches on the 'names' field is done through an autocomplete function in php. We have the table indexed but still feel that the searching is a bit sluggish (a few full seconds vs. something like Google Finance w/ near-instant response). We came up w/ 2…
I need to upload a .csv file and save the records in bigtable.
My application successfully parse 200 the records in the csv files and save to table.
Here is my code to save the data.
for (int i=0;i<lines.length -1;i++) //lines hold total records in csv file
{
String line = lines[i];
//The record have 3 columns…
Hi
I have a quite large table storing words contained in email messages
mysql> explain t_message_words;
+----------------+---------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+---------+------+-----+---------+----------------+
| mwr_key…
I have MySQL database, where I store the following BLOB (which contains JSON object) and ID (for this JSON object). JSON object contains a lot of different information. Say, "city:Los Angeles" and "state:California".
There are about 500k of such records for now, but they are growing. And each JSON object is quite big.
My…
I'm looking at solutions to store a massive quantity of information consuming the less possible disk space.
The information structure is very simple and the queries will also be very simple.
I've looked at solutions like Apache Cassandra and relations databases but couldn't find a comparison where disk usage is mentioned.…
I am trying to design tables to buildout a follower relationship.
Say I have a stream of 140char records that have user, hashtag and other text.
Users follow other users, and can also follow hashtags.
I am outlining the way I've designed this below, but there are two limitaions in my design. I was wondering if others…
I've noticed that google app engine seems to have a fair amount of downtime where they place the datastore into read-only mode. Frequently this downtime is in the middle of the day. Is this something that is happening only during early development, or is this something that I can expect to be always be occurring?
I've…
For various reasons, I need a unique running integer id for my entities stored on the Google AppEngine. The automatically generated key sort of has this behaviour, but it doesn't start from 1 (or 0) and doesn't guarantee that the generated integer part will come from a continuous sequence.
What would be the best way to…
I'm trying to overcome a very serious performance issue in which Sybase refuses to use the primary key index on a large table because one of the required fields is specified indirectly through another table - or, in other words;
SELECT ... FROM BIGTABLE WHERE KFIELD = 123
runs in ms but
SELECT ... FROM BIGTABLE,…
I have an SQL query that executes a LEFT JOIN on another table, then outputs all results that could be coupled into a designated table. I then have a second SQL query that executes the LEFT JOIN again, then outputs the results that could not be coupled to a designated table. In code, this is something like:
INSERT…
When we launched our first Subversion-on- Bigtable service in 2006 our goal was to scale to support hundreds of thousands of projects, with the idea that we could...