Search Results

Search found 1015 results on 41 pages for 'mongodb csharp'.

Page 14/41 | < Previous Page | 10 11 12 13 14 15 16 17 18 19 20 21  | Next Page >

  • Real-time log parsing and reporting

    - by Alienfluid
    We have a small project we are working on part-time that runs on Nginx/MongoDB on Ubuntu 10.04 LTS Server. We'd like to be able to see reports on things like server load, requests/sec, response time, DB load, DB response time, etc. Is there an open source or free (as in beer) tool that can parse such logs and provide a real-time report? I looked into Splunk briefly, but I wanted to see if there are any others that are highly recommended.

    Read the article

  • How would I template an SLS using saltstack

    - by user180041
    I'm trying to do proof of concept with Mongodb(sharding) and Id like to run a command every time I spin up a new cluster without having to add lines in all my sls files. My current init is as follows: mongo Replica4:27000 /usr/lib/mongo/init_addshard.js: cmd: - run - user: present The word Replica4 is not templated id like to know a way I would be able to do so, that way when I spin up a new cluster I wouldn't have to touch anything in this file.

    Read the article

  • which NoSQL for billions of records [closed]

    - by airtruk
    There are plenty of discussions around NoSQL databases around and a lot of them are about data logging in the social media section. The problem I'm trying to solve falls more into the scientific computing section, where I have several 1000s of billions of pieces of information that I want to query with different a different criteria for each query. All data is at least a 4 dimensional space, which means I have a 3D location (x,y,z) and a time component - plus the value and unit. Say temperature at xyz and 10min in degree Celcius. A typical query result may contain several million results ... I have read about pretty much all NoSQL solutions being exceptionally fast for inserting records, but when it comes to querying them it's a different story. I'm leaning towards MongoDB for the implementation and platform for developing the necessary code since it is more closely related to the current solution using MySQL. Happy to be proven wrong though when it comes to the choice of the NoSQL solution.

    Read the article

  • Disk / system configuration for log collection / syslog server

    - by Konrads
    I am looking into building a syslog / logging infrastructure and am pondering about some architecture best practices. Essentially, I see that a syslog system needs to support two conflicting workloads: log collection. Potentially massive streams of data need to be written quickly to disks and indexed. log querying. logs will be queried by both fixed fields such as date and source as well as text search. What is the best disk/system setup assuming I'd like to keep it to a single server for now? Should I use SSDs or ramdisk to off-load some processing? some disks in stripe and some in raid5? I am particularly eyeing Graylog2 with ElasticSearch/MongoDB

    Read the article

  • PostgresQL on Amazon EBS volume, realistic performance, or move to something more lightweight?

    - by Peck
    Hi, I'm working on a little research project, currently running as an instance on ec2, and I'm hoping to figure out whether I'm going down the right path. We, like a thousand other people, are making use of some of twitters streaming feeds to do gather some data to have fun with and my db seems to be having problems keeping up, and queries take what seems to be a very long time. I'm not a DBA by trade, so I'll just dump some info here and add more if need be. System specs: ec2 xl, 15 gigs of ram ebs: 4 100 gb drives, raid 0. The stream we're getting we're looking at around 10k inserts per minute. 3 main tables, with the users we're tracking somewhere in the neighborhood of 26M rows currently. Is this amount of inserts on this hardware too much to ask out of ebs? Should take a look at some things with less overhead like mongodb?

    Read the article

  • How to upgrade a single instance's size without downtime

    - by Justin Meltzer
    I'm afraid there may not be a way to do this since we're not load balancing, but I'd like to know if there is any way to upgrade an EC2 EBS backed instance to a larger size without downtime. First of all, we have everything on one instance: both our app and our database (mongodb). This is along the lines i'm thinking: I know you can create snapshots of your EBS and an AMI of your instance. We already have an AMI and we create hourly snapshots. If I spin up a new separate instance of a larger size and then implement (not sure what the right term is here) the snapshots so that our database is up to date, then I could switch the A record of our domain from the old ip address to the new one. However, I'm afraid that after copying over the data from the snapshot, by the time it takes to change the A record and have that change propagate, the data could potentially be stale. Is there a way to prevent this, and is there a better way to do this than I am suggesting?

    Read the article

  • MMS gets hostname from uname and can't connect to it

    - by Adam Monsen
    I'm trying to get 10gen's MongoDB Monitoring Service monitoring my 3-node replica set. The replica set running in an AWS VPC. Each node runs on a different [virtual] machine. Assume their IPs are 192.168.1.1 (primary or secondary), 192.168.1.2 (primary or secondary), 192.168.1.3 (arbiter). From a quick look at the source, MMS appears to get the hostname of the machine it is running on like so: platform.uname()[1] For my VPC EC2 instance, this returns something like ip-192-168-1-1 MMS then tries to connect to this hostname, which does not resolve. I'd rather just use IP addresses (since they're always static), but it seems like the hardcoded use of platform.uname()[1] in mmsAgent.py precludes that. So, what's an elegant way out of this? Hack /etc/hosts? I'm not setting up a DNS server just for this. Maybe I'm just misunderstanding how to configure MMS.

    Read the article

  • Mongo Client RedHat EL5 UT8 Support

    - by Michael Irey
    # mongo MongoDB shell version: 1.6.4 Fri Mar 16 11:55:46 *** warning: spider monkey build without utf8 support. consider rebuilding with utf8 support connecting to: test Mongo Server seems to handle the utf8 characters fine, as well as my php-mongo-client driver. But when I try to query a record that has a utf8 character from the mongo command line client I get: > db.Users.find({age:33}); error:non ascii character detected Fri Mar 16 11:55:43 mongo got signal 11 (Segmentation fault), stack trace: Fri Mar 16 11:55:43 0x440b50 0x3664c302d0 0x3f47e7b6e0 0x3f47e83bbd 0x3f47e254f3 0x3f47e25660 0x3f47e256ee 0x3f47e25792 0x3f47e2876e 0x4b031d 0x443b72 0x445476 0x3664c1d994 0x43fd39 mongo(_Z12quitAbruptlyi+0x3b0) [0x440b50] /lib64/libc.so.6 [0x3664c302d0] /usr/lib64/libjs.so.1 [0x3f47e7b6e0] /usr/lib64/libjs.so.1(js_CompileTokenStream+0x3d) [0x3f47e83bbd] /usr/lib64/libjs.so.1 [0x3f47e254f3] /usr/lib64/libjs.so.1(JS_CompileUCScriptForPrincipals+0x60) [0x3f47e25660] /usr/lib64/libjs.so.1(JS_EvaluateUCScriptForPrincipals+0x3e) [0x3f47e256ee] /usr/lib64/libjs.so.1(JS_EvaluateUCScript+0x22) [0x3f47e25792] /usr/lib64/libjs.so.1(JS_EvaluateScript+0x6e) [0x3f47e2876e] mongo(_ZN5mongo7SMScope4execERKSsS2_bbbi+0xed) [0x4b031d] mongo(_Z5_mainiPPc+0x14a2) [0x443b72] mongo(main+0x26) [0x445476] /lib64/libc.so.6(__libc_start_main+0xf4) [0x3664c1d994] mongo(__gxx_personality_v0+0x269) [0x43fd39] Any ideas or suggestions would be welcome

    Read the article

  • How to choose NoSQL database engine?

    - by Poma
    We have a database with following specs: 30k records, 7mb in size 20 inserts/second 1000 updates/second 1000 range selects/second, by secondary index, approx 10 rows each needs at least one secondary index needs some mechanism to expire keys if they are not updated for 75 secs (can be done via programmatic garbage collector but will require additional 'last_update' index and will add some load) consistency is not required durability is not required db should be stored in memory For now we use Redis, but it does not have secondary index and it's keys index:foo:* is too slow. Membase also does not have secondary index (as far as I know). MongoDB and MySQL memory engine have table-level locks. What engine will fit our use case?

    Read the article

  • How long does it take in practice to warm up large in-memory databases?

    - by Sim
    Companies such as Peak Hosting are offering 64 core machines with 512Gb RAM for $2K/month. This is a very interesting choice for in-memory databases such as Memcached/Redis as well as databases whose performance degrades rapidly when the data & indexes don't fit in RAM, such as MongoDB. My main concern with monster machines such as these is the time it takes to warm up an in-memory database. In my experience, theoretical metrics, e.g., that SATA can load 100Mb/sec, fall short of what happens in practice. Even at that rate, 100Mb/sec means that loading up 512Gb RAM machine from SATA disks can take over 1 1/2 hours (!). I am looking for real-world reports of warm-up times for machines with very large memory. Please, share details of the software on the machine, data size, storage configuration, e.g., SATA or SSD, network, hosting/cloud provider, if relevant, etc.

    Read the article

  • replica set with multiple primary nodes

    - by miguel
    I am trying to configure a replica set with three nodes: node A, B and C. I execute the rs.add()'s from node A and after that rs.status() shows that the three nodes are PRIMARY. Moreover node B and C have 0 pingMs. If I execute rs.status() from node B or C the only node listed is the self (As PRIMARY). I tried adding an arbiter but it didn't work (it behaved as the nodes B and C). I think this can be a networking problem but I can't figure it out. Edit: This is the output for netstat -anp|grep 27017: tcp 0 0 0.0.0.0:27017 0.0.0.0:* LISTEN - tcp 0 0 10.0.1.211:55772 10.0.1.213:27017 TIME_WAIT - tcp 0 0 127.0.0.1:50509 127.0.0.1:27017 ESTABLISHED - tcp 0 0 127.0.0.1:27017 127.0.0.1:50509 ESTABLISHED - tcp 0 0 10.0.1.211:55774 10.0.1.213:27017 TIME_WAIT - tcp 0 0 10.0.1.211:55776 10.0.1.213:27017 ESTABLISHED - tcp 0 0 10.0.1.211:39180 10.0.1.212:27017 ESTABLISHED - tcp 0 0 10.0.1.211:39178 10.0.1.212:27017 TIME_WAIT - tcp 0 0 10.0.1.211:39176 10.0.1.212:27017 TIME_WAIT - unix 2 [ ACC ] STREAM LISTENING 3291267 - /tmp/mongodb-27017.sock the private ips for the node B and C are 10.0.1.212 and 10.0.1.213 respectively (they appear to have an established connection in the 27017 port according to the netstat output).

    Read the article

  • Large, high performance object or key/value store for HTTP serving on Linux

    - by Tommy
    I have a service that serves images to end users at a very high rate using plain HTTP. The images vary between 4 and 64kbytes, and there are 1.300.000.000 of them in total. The dataset is about 30TiB in size and changes (new objects, updates, deletes) make out less than 1% of the requests. The number of requests pr. second vary from 240 to 9000 and is dispersed pretty much all over, with few objects being especially "hot". As of now, these images are files on a ext3 filesystem distributed read only across a large amount of mid range servers. This poses several problems: Using a fileysystem is very inefficient since the metadata size is large, the inode/dentry cache is volatile on linux and some daemons tend to stat()/readdir() it's way through the directory structure, which in my case becomes very expensive. Updating the dataset is very time consuming and requires remounting between set A and B. The only reasonable handling is operating on the block device for backup, copying, etc. What I would like is a deamon that: speaks HTTP (get, put, delete and perhaps update) stores data it in an efficient structure. The index should remain in memory, and considering the amount of objects, the overhead must be small. The software should be able to handle massive connections with slow (if any) time needed to ramp up. Index should be read in memory at startup. Statistics would be nice, but not mandatory. I have experimented a bit with riak, redis, mongodb, kyoto and varnish with persistent storage, but I haven't had the chance to dig in really deep yet.

    Read the article

  • what is best multi-server configuration with OpenVPN

    - by sebut
    We have a number of Database severs running MongoDB on Debian plus a number of Application servers also on Debian. The db servers hold replicating db clusters, so they need to talk to each other. Application servers need to talk to all db servers (for reasons of fault tolerance). The servers are potentially spread across multiple hosting centers, so we need secure channels between all servers. The number of servers is bound to grow, so we need a VPN solution that's easy to maintain and expand. This is why I feel that SSH that we use for testing might not be up to the task and OpenVPN seems the way to go. I have ruled out TAP, since I understand that this would mean all traffic going to all the servers - perhaps this is a misunderstanding and TAP acts more like a switch? With TUN devices I imagine that all DB servers would live in their own separate subnet, they would also need a client configured to be able to connect to each of their peers. The application servers could live in a common subnet range with a client config only. Does this sound like a reasonable setup? Strangely, on the web I did not find anything about multi-server with OpenVPN. Thanks for all insights and ideas!

    Read the article

  • Dynamic attributes with Rails and Mongoid

    - by japancheese
    Hello, I'm learning MongoDB through the Mongoid Ruby gem with Rails (Rails 3 beta 3), and I'm trying to come up with a way to create dynamic attributes on a model based on fields from another model, which I thought a schema-less database would be a good choice for. So for example, I'd have the models: class Account include Mongoid::Document field :name, :type => String field :token, :type => String field :info_needed, :type => Array embeds_many :members end class Member include Mongoid::Document embedded_in :account, :inverse_of => :members end I'm looking to take the "info_needed" attribute of the Account model and created dynamic attributes on the Member model based on what's inside. If club.info_needed was ["first_name", "last_name"], I'm trying to create a form that would save first_name and last_name attributes to the Member model. However, upon practice, I just keep getting "undefined method first_name=" errors on the Member model when trying to do this. I know MongoDB can handle dynamic attributes per record, but how can I get Mongoid to do this without an undefined method error?

    Read the article

  • Mongo connect problem using asp.net

    - by Amr ElGarhy
    I wrote these lines in My Application start event: var mongo = new Mongo(); mongo.Connect(); var blog = mongo.GetDatabase("Blog"); mongo.Disconnect(); but on this line: mongo.Connect(); it gave me this error: No connection could be made because the target machine actively refused it 127.0.0.1:27017 I am a beginner using Mongodb and i am just try to make sample code to see it's power. So i have no idea how to solve this problem. I am using VS2008 with MongoDB.Driver on Windows7.

    Read the article

  • Difference between Document-oriented-DB and Bigtable clones

    - by chen
    We are looking for a suitable storage engine for our weblog history data. We looked at Bigtable's paper and understand it is suitable to us well. However, I also understand that Document-oriented-DB such as MongoDB seems to provide a little more powerful schema power -- i.e, it can model our data as well. I wonder how nowadays ppl choose a scalable NoSQL DB --- I read enough articles like "we looked at A, B and C, and we decided to use C". But I'd like to see some benchmark number. What I am saying is that if MongoDB and the like can provide same level of performance as Bigtable clones, why don't web companies choose it (preparing to deal with various potentially more complex data problem)? Thanks, By the way, I read an article (which convinced me at the moment) saying Cassandra does not fit the M/R operation, any comments?

    Read the article

  • how to have separate keys per record in mongo_mapper + Rails

    - by Vitaly Kushner
    When I'm adding a record in mongodb I can specify whatever keys I want and it will store it in the db. The problem is that it will remember those keys for the next time I insert another record. so for example if I do the following: Product.create :foo => 123 and then Product.create :bar => 456 I get :foo => nil field in the 2nd record. This is definitely not a limitation of mongodb itself, since if I restart the rails console and create yet another record with different set of columns, it will not add the columns from the 1st 2 records. So it seems like mongomapper remembers all the keys used and inserts them all into all records, even if values are not provided. The question is obviously: how do I disable this crazy attributes explosion? Basically I want only the 'permanent' keys that I specify in the model to be in every record, but all the 'extra' attributes to be specified per record and not to mess the consequent records.

    Read the article

  • how to test if a string is a valid UTF16 string?

    - by superb
    I am using mongodb and javascript to do some string processing. Now I got some error like: Sun May 23 07:42:20 Assertion failure JS_EncodeCharacters( _context , s , srclen , dst , &len) scripting/engine_spidermonkey.cpp 152 0x80f4f7e 0x80f8794 0x811525b 0x811a953 0x8119fc4 0x8111bc5 0x81b408e 0x81c4ee7 0x81b4a10 0x817a881 0x817a7d8 0x817a6e2 0x811e1bb 0x80a777b 0x80a8f8a 0xb7cb2455 0x80a37a1 mongodb-linux-i686-1.4.2/bin/mongo(_ZN5mongo12sayDbContextEPKc+0xfe) [0x80f4f7e] After doing some google, I find that JS_EncodeCharacters return false if the input is not a valid UTF16 string. (if spidermonkey is build with UTF-8 enabled) So I was wondering how to test if the input string if a proper UTF16 string? so I can skip such kind of string to avoid problem ... Thanks

    Read the article

  • Scala case class generated field value

    - by Petteri Hietavirta
    I have an existing Scala application and it uses case classes which are then persisted in MongoDB. I need to introduce a new field to a case class but the value of it is derived from existing field. For example, there is phone number and I want to add normalised phone number while keeping the original phone number. I'll update the existing records in MongoDB but I would need to add this normalisation feature to existing save and update code. So, is there any nice shortcut in Scala to add a "hook" to a certain field of a case class? For example, in Java one could modify setter of the phone number.

    Read the article

  • Is shortening properties names worth it?

    - by raam86
    in how to node Blog rolling with node.js and mongoDB the author mentions it's a good idea to shorten proprieties names: ....oft-reported issue with mongoDB is the size of the data on the disk... each and every record stores all the field-names .... This means that it can often be more space-efficient to have properties such as 't', or 'b' rather than 'title' or 'body', however for fear of confusion I would avoid this unless truly required! I am aware of solutions of how to do it I am more intrested in when is it truly required?

    Read the article

  • Understanding NoSQL Data Modeling - blog application

    - by Rushabh RajeshKumar Padalia
    I am creating an blogging application in Node.js + MongoDB Database. I have used relational Database like MySQL before but this is my first experience with NoSQL database. So I would like to conform my MongoDB data models before I move further. I have decided my blogDB to have 3 collections post_collection - stores information about that article comment_collection - store information about comments on articles user_info_collection - contains user inforamtion PostDB { _"id" : ObjectID(...), "author": "author_name", "Date": new Date(....), "tag" : ["politics" , "war"], "post_title": "My first Article", "post_content": "Big big article" "likes": 23 "access": "public" } CommentDB { "_id" : Objectid(...), "POST": "My First Article", "comment_by": "User_name", "comment": "MY comments" } UserInfoDB { "_id": ObjectID(...), "user": "User_name", "password": "My_password" } I would appreciate your comments.

    Read the article

  • Why is Javascript used in MongoDB and CouchDB instead of other languages such as Java, C++?

    - by startup007
    I asked this question on SO but was suggested to try here. So here it goes: My understanding of Javascript so far has been that it is a client-side language that capture events and makes a web-page dynamic. But on reading the comparison between MongoDB and CouchDB I noticed that both are using Javascript. This makes me wonder the reason behind the choice of JavaScript over other conventional languages. I guess I am trying to understand the role of JavaScript and its advantages over other languages. Update: I am not asking about the languages / drivers supported by the two databases. The comparison says: Both CouchDB and MongoDB make use of Javascript. CouchDB uses Javascript extensively including in the building of views. MongoDB also supports running arbitrary javascript functions server-side and uses javascript for map/reduce operations. My lack of understanding pertains to why is Javascript being used at all for the backend work. Why is it preferred for building views in CouchDB, or for using map/reduce operations? Why C/C++ or Java were not used? What are the advantages in using Javascript for such back-end work?

    Read the article

  • Using map/reduce for mapping the properties in a collection

    - by And
    Update: follow-up to MongoDB Get names of all keys in collection. As pointed out by Kristina, one can use Mongodb 's map/reduce to list the keys in a collection: db.things.insert( { type : ['dog', 'cat'] } ); db.things.insert( { egg : ['cat'] } ); db.things.insert( { type : [] }); db.things.insert( { hello : [] } ); mr = db.runCommand({"mapreduce" : "things", "map" : function() { for (var key in this) { emit(key, null); } }, "reduce" : function(key, stuff) { return null; }}) db[mr.result].distinct("_id") //output: [ "_id", "egg", "hello", "type" ] As long as we want to get only the keys located at the first level of depth, this works fine. However, it will fail retrieving those keys that are located at deeper levels. If we add a new record: db.things.insert({foo: {bar: {baaar: true}}}) And we run again the map-reduce +distinct snippet above, we will get: [ "_id", "egg", "foo", "hello", "type" ] But we will not get the bar and the baaar keys, which are nested down in the data structure. The question is: how do I retrieve all keys, no matter their level of depth? Ideally, I would actually like the script to walk down to all level of depth, producing an output such as: ["_id","egg","foo","foo.bar","foo.bar.baaar","hello","type"] Thank you in advance!

    Read the article

  • getting data from dynamic schema

    - by coure2011
    I am using mongoose/nodejs to get data as json from mongodb. For using mongoose I need to define schema first like this var mongoose = require('mongoose'); var Schema = mongoose.Schema; var GPSDataSchema = new Schema({ createdAt: { type: Date, default: Date.now } ,speed: {type: String, trim: true} ,battery: { type: String, trim: true } }); var GPSData = mongoose.model('GPSData', GPSDataSchema); mongoose.connect('mongodb://localhost/gpsdatabase'); var db = mongoose.connection; db.on('open', function() { console.log('DB Started'); }); then in code I can get data from db like GPSData.find({"createdAt" : { $gte : dateStr, $lte: nextDate }}, function(err, data) { res.writeHead(200, { "Content-Type": "application/json", "Access-Control-Allow-Origin": "*" }); var body = JSON.stringify(data); res.end(body); }); How to define scheme for a complex data like this, you can see that subSection can go to any deeper level. [ { 'title': 'Some Title', 'subSection': [{ 'title': 'Inner1', 'subSection': [ {'titile': 'test', 'url': 'ab/cd'} ] }] }, .. ]

    Read the article

< Previous Page | 10 11 12 13 14 15 16 17 18 19 20 21  | Next Page >