What is the most efficient way to store a mapping "key -> event stream"?
Posted
by jkff
on Stack Overflow
See other posts from Stack Overflow
or by jkff
Published on 2010-03-17T09:07:21Z
Indexed on
2010/03/17
9:11 UTC
Read the original article
Hit count: 281
Suppose there are ~10,000's of keys, where each key corresponds to a stream of events. I'd like to support the following operations:
push(key, timestamp, event)
- pushes event to the event queue for key, marked with the given timestamp. It is guaranteed that event timestamps for a particular key are pushed in sorted or almost sorted order.tail(key, timestamp)
- get all events for key since the given timestamp. Usually the timestamp requests for a given key are almost monotonically increasing, almost synchronously with pushes for the same key.
This stuff has to be persistent (although it is not absolutely necessary to persist pushes immediately and to keep tails with pushes strictly in sync), so I'm going to use some kind of database.
What is the optimal kind of database structure for this task? Would it be better to use a relational database, a key-value storage, or something else?
© Stack Overflow or respective owner