Search Results

Search found 222 results on 9 pages for 'redis sentinel'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 3/9 | < Previous Page | 1 2 3 4 5 6 7 8 9  | Next Page >

  • 100,000 complex structures that are accessed frequently by 100,000 users

    - by Saad
    If you are required to store 100,000 complex structures that are accessed frequently by 100,000 users, which of the following solutions would you use and why? Memcached, In-code python objects, Redis, or a relational database (MySQL). With the little knowledge that I have I think that memcached and In-code python object will not store permanent persistent data. so they don't qualify as the right answer for such a problem. And for complex data structures its best to use Redis. Please correct me if I am wrong.

    Read the article

  • MongoDB vs. Redis vs. Cassandra for a fast-write, temporary row storage solution

    - by Mark Bao
    Hi there, I'm building a system that tracks and verifies ad impressions and clicks. This means that there are a lot of insert commands (about 90/second average, peaking at 250) and some read operations, but the focus is on performance and making it blazing-fast. The system is currently on MongoDB, but I've been introduced to Cassandra and Redis since then. Would it be a good idea to go to one of these two solutions, rather than stay on MongoDB? Why or why not? Thank you

    Read the article

  • relating data stored in NoSQL DB to data stored in SQL DB

    - by seanbrant
    Whats the best way to use a SQL DB along side a NoSQL DB? I want to keep my users and other data in postgres but have some data that would be better suited for a NoSQL DB like redis. I see a lot of talk about switching to NoSQL but little talk on integrating it with existing systems. I think it would be foolish to throw the baby out with the bath water and ditch SQL all together, unless it makes things easier to maintain and develop. I'm wondering what the best approach is for relating data stored in SQL to my data in redis. I was thinking of something along the line of this. User object stored in SQL Book object in redis, key sh1 hash of value, value is a JSON string Relations stored in redis, key User.pk:books, value redis set of sha1's Anyone have experience, tips, better ways?

    Read the article

  • In-memory DB to perform intersects on set slices

    - by IanC
    I have a specific programming need where I need to efficiently store large sorted sets in memory, query them for ranges, and intersect them against other sets that are also queried for ranged. I am looking at Redis, but I can't see a range slice command. MongoDB can only use 1 index, so it has to perform row-level scans, whereas I wish to process using columns that are intersected. I'm also looking at Counchbase, but can't easily determine from the documentation if it is suited to this. I know it uses Memcached, which is AFAIK not suited to this usage. Could anyone share potential solutions for this specific problem? EDIT For example, I need to perform the following: Get the IDs of all cars where the price is between 2000 and 3000, and intersect that will all cars where the engine capacity is between 3000 and 4000.

    Read the article

  • C++ Sentinel/Count Controlled Loop beginning programming

    - by Bryan Hendricks
    Hello all this is my first post. I'm working on a homework assignment with the following parameters. Piecework Workers are paid by the piece. Often worker who produce a greater quantity of output are paid at a higher rate. 1 - 199 pieces completed $0.50 each 200 - 399 $0.55 each (for all pieces) 400 - 599 $0.60 each 600 or more $0.65 each Input: For each worker, input the name and number of pieces completed. Name Pieces Johnny Begood 265 Sally Great 650 Sam Klutz 177 Pete Precise 400 Fannie Fantastic 399 Morrie Mellow 200 Output: Print an appropriate title and column headings. There should be one detail line for each worker, which shows the name, number of pieces, and the amount earned. Compute and print totals of the number of pieces and the dollar amount earned. Processing: For each person, compute the pay earned by multiplying the number of pieces by the appropriate price. Accumulate the total number of pieces and the total dollar amount paid. Sample Program Output: Piecework Weekly Report Name Pieces Pay Johnny Begood 265 145.75 Sally Great 650 422.50 Sam Klutz 177 88.5 Pete Precise 400 240.00 Fannie Fantastic 399 219.45 Morrie Mellow 200 110.00 Totals 2091 1226.20 You are required to code, compile, link, and run a sentinel-controlled loop program that transforms the input to the output specifications as shown in the above attachment. The input items should be entered into a text file named piecework1.dat and the ouput file stored in piecework1.out . The program filename is piecework1.cpp. Copies of these three files should be e-mailed to me in their original form. Read the name using a single variable as opposed to two different variables. To accomplish this, you must use the getline(stream, variable) function as discussed in class, except that you will replace the cin with your textfile stream variable name. Do not forget to code the compiler directive #include < string at the top of your program to acknowledge the utilization of the string variable, name . Your nested if-else statement, accumulators, count-controlled loop, should be properly designed to process the data correctly. The code below will run, but does not produce any output. I think it needs something around line 57 like a count control to stop the loop. something like (and this is just an example....which is why it is not in the code.) count = 1; while (count <=4) Can someone review the code and tell me what kind of count I need to introduce, and if there are any other changes that need to be made. Thanks. [code] //COS 502-90 //November 2, 2012 //This program uses a sentinel-controlled loop that transforms input to output. #include <iostream> #include <fstream> #include <iomanip> //output formatting #include <string> //string variables using namespace std; int main() { double pieces; //number of pieces made double rate; //amout paid per amount produced double pay; //amount earned string name; //name of worker ifstream inFile; ofstream outFile; //***********input statements**************************** inFile.open("Piecework1.txt"); //opens the input text file outFile.open("piecework1.out"); //opens the output text file outFile << setprecision(2) << showpoint; outFile << name << setw(6) << "Pieces" << setw(12) << "Pay" << endl; outFile << "_____" << setw(6) << "_____" << setw(12) << "_____" << endl; getline(inFile, name, '*'); //priming read inFile >> pieces >> pay >> rate; // ,, while (name != "End of File") //while condition test { //begining of loop pay = pieces * rate; getline(inFile, name, '*'); //get next name inFile >> pieces; //get next pieces } //end of loop inFile.close(); outFile.close(); return 0; }[/code]

    Read the article

  • Reversing a circular deque without a sentinel

    - by SDLFunTimes
    Hey Stackoverflow I'm working on my homework and I'm trying to reverse a circular-linked deque without a sentinel. Here are my data structures: struct DLink { TYPE value; struct DLink * next; struct DLink * prev; }; struct cirListDeque { int size; struct DLink *back; }; Here's my approach to reversing the deque: void reverseCirListDeque(struct cirListDeque* q) { struct DLink* current; struct DLink* temp; temp = q->back->next; q->back->next = q->back->prev; q->back->prev = temp; current = q->back->next; while(current != q->back->next) { temp = current->next; current->next = current->prev; current->prev = temp; current = current->next; } } However when I run it and put values 1, 2 and 3 on it (TYPE is just a alias for int in this case) and reverse it I get 2, 3, null. Does anyone have any ideas as to what I may be doing wrong? Thanks in advance.

    Read the article

  • search algorithm using sentinel

    - by davit-datuashvili
    i am trying to do search algorithm using sentinel which reduce time to 3.87n nanoseconds for example compare to this code int search (int t ){ for (int i=0;i<n;i++) if (x[i]==t) return i; return -1; } it takes 4.06 nanoseconds so i am trying to optimize it here is code public class Search{ public static int search(int a[],int t){ int i; int p=0; int n=a.length; int hold; hold=a[n-1]; a[n-1]=t; for ( i=0;;i++) if (a[i]==t) break; a[n-1]=t; if (i==n){ p= -1; } else{ p= i; } return p; } public static void main(String[]args){ int t=-1; int a[]=new int[]{4,5,2,6,8,7,9}; System.out.println(search(a,t)); } } but is show me that 9 is at position 6 which is correct but if t =1 or something else which is not array it show me position 6 too please help

    Read the article

  • MongoDB on 128mb 32-bit VPS (plus Tornado and Redis)

    - by apito
    i am curious about how mongodb will perform in a limited vps. specifically, i'll deploy this configuration on 32-bit ubuntu 9.04 server with 128Mb memory (UPDATE: now i'm considering 360mb too). nginx and redis three instances of tornado apps (one is for mobile site; limited app, not my primary audience); has around 8 Collections. social webapp for my community. mongodb all beside mongodb seems to have small footprint. memory-mapping-wise, i dont know how mongodb will behave. i know it's a little bit a stretch to use this kind of config on a tiny vps, but that's what i can afford for now. i expect to have.. hmm.. maybe ~50 15rps. i did my homework doing a lot of frontend optimizations and yslow says grade A 91 (ruleset V2) :-) anyone willing to share experiences? eg. how big the data set size when mongo hit the ceiling, performance when mongo do a lot of disk IO, etc. thanks. UPDATE: this is my pet project. i'll get back to you when i have next spare time to do same httperf in a vbox with exact spec. suggestion how to do stress testing welcomed. i'm new to this kind of stuff.

    Read the article

  • Twisted: why is it that passing a deferred callback to a deferred thread makes the thread blocking a

    - by surtyaarthoughts
    I unsuccessfully tried using txredis (the non blocking twisted api for redis) for a persisting message queue I'm trying to set up with a scrapy project I am working on. I found that although the client was not blocking, it became much slower than it could have been because what should have been one event in the reactor loop was split up into thousands of steps. So instead, I tried making use of redis-py (the regular blocking twisted api) and wrapping the call in a deferred thread. It works great, however I want to perform an inner deferred when I make a call to redis as I would like to set up connection pooling in attempts to speed things up further. Below is my interpretation of some sample code taken from the twisted docs for a deferred thread to illustrate my use case: #!/usr/bin/env python from twisted.internet import reactor,threads from twisted.internet.task import LoopingCall import time def main_loop(): print 'doing stuff in main loop.. do not block me!' def aBlockingRedisCall(): print 'doing lookup... this may take a while' time.sleep(10) return 'results from redis' def result(res): print res def main(): lc = LoopingCall(main_loop) lc.start(2) d = threads.deferToThread(aBlockingRedisCall) d.addCallback(result) reactor.run() if __name__=='__main__': main() And here is my alteration for connection pooling that makes the code in the deferred thread blocking : #!/usr/bin/env python from twisted.internet import reactor,defer from twisted.internet.task import LoopingCall import time def main_loop(): print 'doing stuff in main loop.. do not block me!' def aBlockingRedisCall(x): if x<5: #all connections are busy, try later print '%s is less than 5, get a redis client later' % x x+=1 d = defer.Deferred() d.addCallback(aBlockingRedisCall) reactor.callLater(1.0,d.callback,x) return d else: print 'got a redis client; doing lookup.. this may take a while' time.sleep(10) # this is now blocking.. any ideas? d = defer.Deferred() d.addCallback(gotFinalResult) d.callback(x) return d def gotFinalResult(x): return 'final result is %s' % x def result(res): print res def aBlockingMethod(): print 'going to sleep...' time.sleep(10) print 'woke up' def main(): lc = LoopingCall(main_loop) lc.start(2) d = defer.Deferred() d.addCallback(aBlockingRedisCall) d.addCallback(result) reactor.callInThread(d.callback, 1) reactor.run() if __name__=='__main__': main() So my question is, does anyone know why my alteration causes the deferred thread to be blocking and/or can anyone suggest a better solution?

    Read the article

  • Trying to sentinel loop this program.

    - by roger34
    Okay, I spent all this time making this for class but I have one thing that I can't quite get: I need this to sentinel loop continuously (exiting upon entering x) so that the System.out.println("What type of Employee? Enter 'o' for Office " + "Clerical, 'f' for Factory, or 's' for Saleperson. Enter 'x' to exit." ); line comes back up after they enter the first round of information. Also, I can't leave this up long on the (very) off chance a classmate might see this and steal the code. Full code following: import java.util.Scanner; public class Project1 { public static void main (String args[]){ Scanner inp = new Scanner( System.in ); double totalPay; System.out.println("What type of Employee? Enter 'o' for Office " + "Clerical, 'f' for Factory, or 's' for Saleperson. Enter 'x' to exit." ); String response= inp.nextLine(); while (!response.toLowerCase().equals("o")&&!response.toLowerCase().equals("f") &&!response.toLowerCase().equals("s")&&!response.toLowerCase().equals("x")){ System.out.print("\nInvalid selection,please enter your choice again:\n"); response=inp.nextLine(); } char choice = response.toLowerCase().charAt( 0 ); switch (choice){ case 'o': System.out.println("Enter your hourly rate:"); double officeRate=inp.nextDouble(); System.out.println("Enter the number of hours worked:"); double officeHours=inp.nextDouble(); totalPay = officeCalc(officeRate,officeHours); taxCalc(totalPay); break; case 'f': System.out.println("How many Widgets did you produce during the week?"); double widgets=inp.nextDouble(); totalPay=factoryCalc(widgets); taxCalc(totalPay); break; case 's': System.out.println("What were your total sales for the week?"); double totalSales=inp.nextDouble(); totalPay=salesCalc(totalSales); taxCalc(totalPay); break; } } public static double taxCalc(double totalPay){ double federal=totalPay*.22; double state =totalPay*.055; double netPay = totalPay - federal - state; federal =federal*Math.pow(10,2); federal =Math.round(federal); federal= federal/Math.pow(10,2); state =state*Math.pow(10,2); state =Math.round(state); state= state/Math.pow(10,2); totalPay =totalPay*Math.pow(10,2); totalPay =Math.round(totalPay); totalPay= totalPay/Math.pow(10,2); netPay =netPay*Math.pow(10,2); netPay =Math.round(netPay); netPay= netPay/Math.pow(10,2); System.out.printf("\nTotal Pay \t: %1$.2f.\n", totalPay); System.out.printf("State W/H \t: %1$.2f.\n", state); System.out.printf("Federal W/H : %1$.2f.\n", federal); System.out.printf("Net Pay \t: %1$.2f.\n", netPay); return totalPay; } public static double officeCalc(double officeRate,double officeHours){ double overtime=0; if (officeHours>=40) overtime = officeHours-40; else overtime = 0; if (officeHours >= 40) officeHours = 40; double otRate = officeRate * 1.5; double totalPay= (officeRate * officeHours) + (otRate*overtime); return totalPay; } public static double factoryCalc(double widgets){ double totalPay=widgets*.35 +300; return totalPay; } public static double salesCalc(double totalSales){ double totalPay = totalSales * .05 + 500; return totalPay; } }

    Read the article

  • Trying to sentinel loop this program. [java]

    - by roger34
    Okay, I spent all this time making this for class but I have one thing that I can't quite get: I need this to sentinel loop continuously (exiting upon entering x) so that the System.out.println("What type of Employee? Enter 'o' for Office " + "Clerical, 'f' for Factory, or 's' for Saleperson. Enter 'x' to exit." ); line comes back up after they enter the first round of information. Also, I can't leave this up long on the (very) off chance a classmate might see this and steal the code. Full code following: import java.util.Scanner; public class Project1 { public static void main (String args[]){ Scanner inp = new Scanner( System.in ); double totalPay; System.out.println("What type of Employee? Enter 'o' for Office " + "Clerical, 'f' for Factory, or 's' for Saleperson. Enter 'x' to exit." ); String response= inp.nextLine(); while (!response.toLowerCase().equals("o")&&!response.toLowerCase().equals("f") &&!response.toLowerCase().equals("s")&&!response.toLowerCase().equals("x")){ System.out.print("\nInvalid selection,please enter your choice again:\n"); response=inp.nextLine(); } char choice = response.toLowerCase().charAt( 0 ); switch (choice){ case 'o': System.out.println("Enter your hourly rate:"); double officeRate=inp.nextDouble(); System.out.println("Enter the number of hours worked:"); double officeHours=inp.nextDouble(); totalPay = officeCalc(officeRate,officeHours); taxCalc(totalPay); break; case 'f': System.out.println("How many Widgets did you produce during the week?"); double widgets=inp.nextDouble(); totalPay=factoryCalc(widgets); taxCalc(totalPay); break; case 's': System.out.println("What were your total sales for the week?"); double totalSales=inp.nextDouble(); totalPay=salesCalc(totalSales); taxCalc(totalPay); break; } } public static double taxCalc(double totalPay){ double federal=totalPay*.22; double state =totalPay*.055; double netPay = totalPay - federal - state; federal =federal*Math.pow(10,2); federal =Math.round(federal); federal= federal/Math.pow(10,2); state =state*Math.pow(10,2); state =Math.round(state); state= state/Math.pow(10,2); totalPay =totalPay*Math.pow(10,2); totalPay =Math.round(totalPay); totalPay= totalPay/Math.pow(10,2); netPay =netPay*Math.pow(10,2); netPay =Math.round(netPay); netPay= netPay/Math.pow(10,2); System.out.printf("\nTotal Pay \t: %1$.2f.\n", totalPay); System.out.printf("State W/H \t: %1$.2f.\n", state); System.out.printf("Federal W/H : %1$.2f.\n", federal); System.out.printf("Net Pay \t: %1$.2f.\n", netPay); return totalPay; } public static double officeCalc(double officeRate,double officeHours){ double overtime=0; if (officeHours>=40) overtime = officeHours-40; else overtime = 0; if (officeHours >= 40) officeHours = 40; double otRate = officeRate * 1.5; double totalPay= (officeRate * officeHours) + (otRate*overtime); return totalPay; } public static double factoryCalc(double widgets){ double totalPay=widgets*.35 +300; return totalPay; } public static double salesCalc(double totalSales){ double totalPay = totalSales * .05 + 500; return totalPay; } }

    Read the article

  • Tips for maximizing Nginx requests/sec?

    - by linkedlinked
    I'm building an analytics package, and project requirements state that I need to support 1 billion hits per day. Yep, "billion". In other words, no less than 12,000 hits per second sustained, and preferably some room to burst. I know I'll need multiple servers for this, but I'm trying to get maximum performance out of each node before "throwing more hardware at it". Right now, I have the hits-tracking portion completed, and well optimized. I pretty much just save the requests straight into Redis (for later processing with Hadoop). The application is Python/Django with a gunicorn for the gateway. My 2GB Ubuntu 10.04 Rackspace server (not a production machine) can serve about 1200 static files per second (benchmarked using Apache AB against a single static asset). To compare, if I swap out the static file link with my tracking link, I still get about 600 requests per second -- I think this means my tracker is well optimized, because it's only a factor of 2 slower than serving static assets. However, when I benchmark with millions of hits, I notice a few things -- No disk usage -- this is expected, because I've turned off all Nginx logs, and my custom code doesn't do anything but save the request details into Redis. Non-constant memory usage -- Presumably due to Redis' memory managing, my memory usage will gradually climb up and then drop back down, but it's never once been my bottleneck. System load hovers around 2-4, the system is still responsive during even my heaviest benchmarks, and I can still manually view http://mysite.com/tracking/pixel with little visible delay while my (other) server performs 600 requests per second. If I run a short test, say 50,000 hits (takes about 2m), I get a steady, reliable 600 requests per second. If I run a longer test (tried up to 3.5m so far), my r/s degrades to about 250. My questions -- a. Does it look like I'm maxing out this server yet? Is 1,200/s static files nginx performance comparable to what others have experienced? b. Are there common nginx tunings for such high-volume applications? I have worker threads set to 64, and gunicorn worker threads set to 8, but tweaking these values doesn't seem to help or harm me much. c. Are there any linux-level settings that could be limiting my incoming connections? d. What could cause my performance to degrade to 250r/s on long-running tests? Again, the memory is not maxing out during these tests, and HDD use is nil. Thanks in advance, all :)

    Read the article

  • How to format the node_redis info function output?

    - by hh54188
    I want check the Redis info on my pc with node, so I use node_redis and run the info function: var redis = require("redis"), client = redis.createClient(); client.on("connect", function () { client.info(function (err, replay) { console.log(replay); }) }) but the response is un-format: `#Server\r\nredis_version:2.6.16\r\nredis_git_sha1:00000000\r\nredis_git_dirty:0\r\nredis_mode:standalone\r\nos:Linux 3.8.0-29-generic x86_64\r\narch_bits:64\r\nmultiplexing_api:epoll\r\ngcc_version:4.6.3\r\nprocess_id:2941\r\nrun_id:e60f261a6f4f6f081563a47961315eff6b1c005d\r\ntcp_port:6379\r\nuptime_in_seconds:1777\r\nuptime_in_days:0\r\nhz:10\r\nlru_clock:2040689\r\n\r\n# Clients\r\nconnected_clients:2\r\nclient_longest_output_list:0\r\nclient_biggest_input_buf:0\r\nblocked_clients:0\r\n\r\n# Memory\r\nused_memory:562584\r\nused_memory_human:549.40K\r\nused_memory_rss:2031616\r\nused_memory_peak:561784\r\nused_memory_peak_human:548.62K\r\nused_memory_lua:31744\r\nmem_fragmentation_ratio:3.61\r\nmem_allocator:jemalloc-3.2.0\r\n\r\n# Persistence\r\nloading:0\r\nrdb_changes_since_last_save:0\r\nrdb_bgsave_in_progress:0\r\nrdb_last_save_time:1383553917\r\nrdb_last_bgsave_status:ok\r\nrdb_last_bgsave_time_sec:-1\r\nrdb_current_bgsave_time_sec:-1\r\naof_enabled:0\r\naof_rewrite_in_progress:0\r\naof_rewrite_scheduled:0\r\naof_last_rewrite_time_sec:-1\r\naof_current_rewrite_time_sec:-1\r\naof_last_bgrewrite_status:ok\r\n\r\n# Stats\r\ntotal_connections_received:3\r\ntotal_commands_processed:5\r\ninstantaneous_ops_per_sec:0\r\nrejected_connections:0\r\nexpired_keys:0\r\nevicted_keys:0\r\nkeyspace_hits:0\r\nkeyspace_misses:0\r\npubsub_channels:0\r\npubsub_patterns:0\r\nlatest_fork_usec:0\r\n\r\n# Replication\r\nrole:master\r\nconnected_slaves:0\r\n\r\n# CPU\r\nused_cpu_sys:0.13\r\nused_cpu_user:0.19\r\nused_cpu_sys_children:0.00\r\nused_cpu_user_children:0.00\r\n\r\n# Keyspace\r\n' How can I turn it to an object? like: { redis_version:2.6.16, redis_git_sha1:00000000, redis_git_dirty:0, ...... } so that I can read each property's value, get information I need

    Read the article

  • What database strategy to choose for a large web application

    - by Snoopy
    I have to rewrite a large database application, running on 32 servers. The hardware is up to date, each machine has two quad core Xeon and 32 GByte RAM. The database is multi-tenant, each customer has his own file, around 5 to 10 GByte each. I run around 50 databases on this hardware. The app is open to the web, so I have no control on the load. There are no really complex queries, so SQL is not required if there is a better solution. The databases get updated via FTP every day at midnight. The database is read-only. C# is my favourite language and I want to use ASP.NET MVC. I thought about the following options: Use two big SQL servers running SQL Server 2012 to serve the 32 servers with data. On the 32 servers running IIS hosting providing REST services. Denormalize the database and use Redis on each webserver. Use booksleeve as a Redis client. Use a combination of SQL Server and Redis Use SQL Server 2012 together with Hadoop Use Hadoop without SQL Server What is the best way for a read-only database, to get the best performance without loosing maintainability? Does Map-Reduce make sense at all in such a scenario? The reason for the rewrite is, the old app written in C++ with ISAM technology is too slow, the interfaces are old fashioned and not nice to use from an website, especially when using ajax. The app uses a relational datamodel with many tables, but it is possible to write one accerlerator table where all queries can be performed on, and all other information from the other tables are possible by a simple key lookup.

    Read the article

  • phpredis + pconnect

    - by john smith
    I am using phpredis on my php based website. The webserver I am using is a the simplest apache apt-get installation, no configuration involved, as this is only a development environment. The issue I am facing is that basically, while using phpredis, there is no difference between "connect" and "pconnect" commands: they both create a new connection everytime, as I can see from the "info" command on redis-cli. Now, I am pretty sure it is because of the apache configuration and the fact that it probably (most likely) is a multi-threaded env, therefore can't enstablish a single connection. My question is basically for when I will turn into production: what would the best choice of a webserver be to avoid this problem? I remember using lighttpd with thousands of users and still get only a very few (like 2 or 3) connections on mongoDB. Any ideas? Thanks in advance.

    Read the article

  • How to choose NoSQL database engine?

    - by Poma
    We have a database with following specs: 30k records, 7mb in size 20 inserts/second 1000 updates/second 1000 range selects/second, by secondary index, approx 10 rows each needs at least one secondary index needs some mechanism to expire keys if they are not updated for 75 secs (can be done via programmatic garbage collector but will require additional 'last_update' index and will add some load) consistency is not required durability is not required db should be stored in memory For now we use Redis, but it does not have secondary index and it's keys index:foo:* is too slow. Membase also does not have secondary index (as far as I know). MongoDB and MySQL memory engine have table-level locks. What engine will fit our use case?

    Read the article

  • How long does it take in practice to warm up large in-memory databases?

    - by Sim
    Companies such as Peak Hosting are offering 64 core machines with 512Gb RAM for $2K/month. This is a very interesting choice for in-memory databases such as Memcached/Redis as well as databases whose performance degrades rapidly when the data & indexes don't fit in RAM, such as MongoDB. My main concern with monster machines such as these is the time it takes to warm up an in-memory database. In my experience, theoretical metrics, e.g., that SATA can load 100Mb/sec, fall short of what happens in practice. Even at that rate, 100Mb/sec means that loading up 512Gb RAM machine from SATA disks can take over 1 1/2 hours (!). I am looking for real-world reports of warm-up times for machines with very large memory. Please, share details of the software on the machine, data size, storage configuration, e.g., SATA or SSD, network, hosting/cloud provider, if relevant, etc.

    Read the article

  • Developing a high-performance, scalable Comet application

    - by Rob
    Well, the title says most of it. I'm looking to develop a chat application that will hopefully become something more, and currently I'm considering my options for what I should build it on top of. I've taken a look at Tornado with Redis as my primary option - Tornado, being a Comet server, is perfect for long polling to retrieve the messages on Redis, which I have the intention of using as both a persistent data store, as well as a message queue with its nifty subpub features. However, I've also heard good things about Django, RabbitMQ, MongoDB and Orbited. JavaScript isn't a big problem for me, so Orbited's JavaScript support isn't too much of a boon. Really, I'd probably be happy to develop on the route I've chosen for myself, but if there are any gaping deficiencies in my plan, I'd like some kind person to point them out before I find I've wasted months on this.

    Read the article

  • Caching for database questions.

    - by SeanD
    When we say caching like using memcahe or Redis, is this a 1:1 caching between the user and the cache or can we cache 1 item and use it for all user? Some items like a Friend list will be 1:1 a that is unique per user. But if i want to cache the auto complete list for city lookups which can be used by any user, will it just store 1 list in the cache used by all users at same time or doe it need to store 1 list per user? Is it possible to cache the entire database, all the lookups, all the users, all their photos, etc using memache or redis? So from the above example: a friend list will be cleared from the cache when the user logs off. But something like city auto complete will stay in the cache 24-7-365, am i correct?

    Read the article

  • Nginx dynamic upstream configuration / routing

    - by Dan Sosedoff
    I was experimenting with dynamic upstream configuration for nginx and cant find any good solution to implement upstream configuration from third-party source like redis or mysql. The idea behind it is to have a single file configuration in primary server and proxy requests to various app servers based on environment conditions. Think of dynamic deployments where you have X servers that are running Y workers on different ports. For instance, i create a new app and deploy. App manager selects a server and then rolls out a worker (Ruby/PHP/Python) and then reports the ip:port to the central database with status "up". At this time when i go to the given url nginx should proxy all requests to the specified ip:port upstream. The whole thing is pretty similar to what heroku does, except this proof-of-concept is not supposed to be production ready, mostly for internal needs. The easiest solution i found was using resolver with ruby-based DNS server. It works, nginx gets the IP address correctly, but the only problem is that you cant define port number for that IP. Second solution (which i havent tried yet) is to roll something else as a proxy server, maybe written in Erlang. In this case we need to use something to serve static content. Any ideas how to implement this in more flexible and stable way? P.S. Some research options: http://openresty.org/#DynamicRoutingBasedOnRedis https://github.com/nodejitsu/node-http-proxy

    Read the article

  • PHP on several servers with session-sharing

    - by Etu
    there's certanly other threads about this, but I have one more question. We are about to scale the website at work to have more than one server. And we need to share the sessions between the servers. We have been looking into different solutions, one in memcached and use Memcached as sessionhandler in PHP. That will probably work. And the idea would be to run memcached on every machine and let all webservers access all other servers memcached servers, and then we have shared sessions between the machines, yay. (we have no resources to setup with sticky-sessions yet, that's a later project. we need this running, and we need this running now. and we will loadbalance with DNS for a starter) But then... If I want to take one server down, say, for maintenance, or a server crashes, or whatever reason. I don't want the users to just loose their sessions and have to start from the beginning... That's why we need some kind of replication, which Memcached does not support. Then I found http://repcached.lab.klab.org/ -- which has multi-master replication of memcached, which is great, and is what I want. But does it work with 2 machines? Say 3, 5, 10? For future scaling. I also looked into redishttp://redis.io/ -- which also seems great, but is a bit more "shaky" with the php-session-handler support, and no multi-master-replication. The thing is that I like to use memcached, but I want to be able to power down one of two boxes without loosing half of the sessions. Any suggestions?

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 1 2 3 4 5 6 7 8 9  | Next Page >