Search Results

Search found 5318 results on 213 pages for 'cloud computing'.

Page 45/213 | < Previous Page | 41 42 43 44 45 46 47 48 49 50 51 52  | Next Page >

• Cluster Nodes as RAID Drives

  - by BuckWoody

  I'm unable to sleep tonight so I thought I would push this post out VERY early. When you don't sleep your mind takes interesting turns, which can be a good thing. I was watching a briefing today by a couple of friends as they were talking about various ways to arrange a Windows Server Cluster for SQL Server. I often see an "active" node of a cluster with a "passive" node backing it up. That means one node is working and accepting transactions, and the other is not doing any work but simply "standing by" waiting for the first to fail over. The configuration in the demonstration I saw was a bit different. In this example, there were three nodes that were actively working, and a fourth standing by for all three. I've put configurations like this one into place before, but as I was looking at their architecture diagram, it looked familar - it looked like a RAID drive setup! And that's not a bad way to think about your cluster arrangements. The same concerns you might think about for a particular RAID configuration provides a good way to think about protecting your systems in general. So even if you're not staying awake all night thinking about SQL Server clusters, take this post as an opportunity for "lateral thinking" - a way of combining in your mind the concepts from one piece of knowledge to another. You might find a new way of making your technical environment a little better. Share this post: email it! | bookmark it! | digg it! | reddit! | kick it! | live it!

  Read the article

• Can an issue tracking system be distributed?

  - by Klaim

  I was thinking about issue tracking software like Redmine, Trac or even the one that is in Fossil and something hit me: Is there a reason why Redmine and Trac are not possible to be distributed? Or maybe it's possible and I just don't know how it's possible? If it's not possible, why? By distributed I mean like Facebook or Google or other applications that effectively runs on multiple hardware a the same time but share data.

  Read the article

• Have You Heard About Project Lucy?

  - by KKline

  Lucy, You Got Some 'Splainin to Do!' Quest Software's latest community initiative, Windows Azure-based Project Lucy, has debuted! Project Lucy is part infrastructure analytics, part social media experiment, and part performance data warehouse. The best things about Project Lucy include: It’s Free - just like our SQLServerPedia website, Project Lucy is free to anyone who wants to upload a trace file It’s 1oo% web-based - you don’t have to download or maintain anything and updates roll out seamlessly,...(read more)

  Read the article

• Toad Just Keeps Getting Better

  - by KKline

  Toad for SQL Server just keeps getting better. And with its win as Best of TechEd in the Database Development category, there's even more to love....(read more)

  Read the article

• Building massively scalable systems, where to start? [closed]

  - by Mahmoud Hossam

  Recently, I've been seeing these job postings about building scalable systems using Java, and some of the technologies mentioned were: Cassandra Thrift Hadoop MapReduce Among others. How can I get started with these technologies? Is there something else I need to know before actually learning any of these technologies? Maybe some general concepts about building highly available and scalable systems? I already know Java SE, so I won't be starting from scratch.

  Read the article

• Join Me May 19th for 24 Hours of PASS

  - by KKline

  Check out all the great SQL Server sessions you can attend for FREE in this on-line event....(read more)

  Read the article

• Design pattern for client/server sessions?

  - by nonot1

  Are there any common patterns or general guidance I can learn from for how to design a client/server system where the both the client and server must maintain some kind per-client session state? I've found any number of libraries that can help with some of the plumbing, but it's the overall design I'm wondering about. Open issues in my mind: How to structure the client/server communication so that bidirectional synchronous and asynchronous requests are possible? The server side needs to spawn a couple of per-connected-client session-long helper process. How to manage that? How to manage the mapping from a given client (and any of it's requests) to server state and helper process instances in the face of multiple clients and intermittent network connectivity. Most communication can be simple blocking request/reply, but some will be long running processing tasks that the client will want to keep tabs on. To the extent that it matters, the platform is Linux/C/C++. Not web based. Just an existing thick-client software app being modified to talk to backend servers for some tasks.

  Read the article

• Optimistic work sharing on sparsely distributed systems

  - by Asti

  What would a system like BOINC look like if it were written today? At the time BOINC was written, databases were the primary choice for maintaining a shared state and concurrency among nodes. Since then, many approaches have been developed for tasking with optimistic concurrency (OT, partial synchronization primitives, shared iterators etc.) Is there an optimal paradigm for optimistically distributing units of work on sparsely distributing systems which communicate through message passing? Sorry if this is a bit vague. P.S. The concept of Tuple-spaces is great, but locking is inherent to its definition. Edit: I already have a federation system which works very well. I have a reactive OT system is implemented on top of it. I'm looking to extend it to get clients to do units of work.

  Read the article

• Building a distributed system on Amazon Web Services

  - by Songo

  Would simply using AWS to build an application make this application a distributed system? For example if someone uses RDS for the database server, EC2 for the application itself and S3 for hosting user uploaded media, does that make it a distributed system? If not, then what should it be called and what is this application lacking for it to be distributed? Update Here is my take on the application to clarify my approach to building the system: The application I'm building is a social game for Facebook. I developed the application locally on a LAMP stack using Symfony2. For production I used an a single EC2 Micro instance for hosting the app itself, RDS for hosting my database, S3 for the user uploaded files and CloudFront for hosting static content. I know this may sound like a naive approach, so don't be shy to express your ideas.

  Read the article

• Exclusive use of a Jini server during long-running call

  - by Matthew Flint

  I'm trying to use Jini, in a "Masters/Workers" arrangement, but the Worker jobs may be long running. In addition, each worker needs to have exclusive access to a singleton resource on that machine. As it stands, if a Worker receives another request while a previous request is running, then the new request is accepted and executed in a second thread. Are there any best-practices to ensure that a Worker accepts no further jobs until the current job is complete? Things I've considered: synchronize the job on the server, with a lock on the singleton resource. This would work, but is far from ideal. A call from a Master would block until the current Worker thread completes, even if other Workers become free in the meantime unregister the Worker from the registry while the job is running, then re-register when it completes. Might work OK, but something doesn't smell right with this idea... Of course, I'm quite happy to be told that there are more appropriate technologies than Jini... but I wouldn't want anything too heavyweight, like rolling out EJB containers all over the place.

  Read the article

• Scalable Architecture for modern Web Development [on hold]

  - by Jhilke Dai

  I am doing research about Scalable architecture for Web Development, the research is solely to support Modern Web Development with flexible architecture which can scale up/down according to the needs without losing any core functionality. By Modern Web I mean to support all the Devices used to access websites, but the loading mechanism for all devices would be different. My quest of architecture is: For PC: Accessing web in PC is faster but it also depends on the Geo-location, so, the application would check by default the capacity of Internet/Browser and load the page according to it. For Mobile: Most of the mobile design these days either hide information or use different version of same application. eg: facebook uses m.facebook.com which is completely different than PC version. Hiding the things from Mobile using JavaScript or CSS is not a solution as it'll consume the bandwidth and make the application slow. So, my architecture research is about Serving one Application, which has different stack. When the application receives the request it'd send the Packaged Stack to the received request. This way the load time for end users would be faster and maintenance of application for developers would be easier. I am researching about for 4-tier(layered) architecture like: Presentation Layer Application Logic Layer -- The main Logic layer which stores the Presentation Stack Business Logic Layer Data Layer Main Question: Have you come across of similar architecture? If so, then can you list the links here, I'm very much interested to learn about those implementations specially in real world scenario. Have you thought about similar architectures and tried your own ideas, or if you have any ideas regarding this, then I urge to share. I am open to any discussions regarding this, so, please feel free to comment/answer.

  Read the article

• How to mange big amount users at server side?

  - by Rami

  I built a social android application in which users can see other users around them by gps location. at the beginning thing went well as i had low number of users, But now that I have increasing number of users (about 1500 +100 every day) I revealed a major problem in my design. In my Google App Engine servlet I have static HashMap that holding all the users profiles objects, currenty 1500 and this number will increase as more users register. Why I'm doing it Every user that requesting for the users around him compares his gps with other users and check if they are in his 10km radius, this happens every 5 min on average. That is why I can't get the users from db every time because GAE read/write operation quota will tare me apart. The problem with this desgin is As the number of users increased the Hashmap turns to null every 4-6 hours, I thing that this time is getting shorten but I'm not sure. I'm fixing this by reloading the users from the db every time I detect that it became null, But this causes DOS to my users for 30 sec, So I'm looking for better solution. I'm guessing that it happens because the size of the hashmap, Am I right? I have been advised to use spatial database, but that mean that I can't work with GAE any more and that mean that I need to build my big server all over again and lose my existing DB. Is there something I can do with the existing tools? Thanks.

  Read the article

• Interconnect nodes in a Java distributed infrastructure for tweet processing

  - by David Moreno García

  I'm working in a new version of an old project that I used to download and process user statuses from Twitter. The main problem of that project was its infrastructure. I used multiple instances of a java application (trackers) to download from Twitter given an specific task (basically terms to search for), connected with a central node (a web application) that had to process all tweets once per day and generate a new task for each trackers once each 15 minutes. The central node also had to monitor all trackers and enable/disable them under user petition. This, as I said, was too slow because I had multiple bottlenecks, so in this new version I want to improve the infrastructure and isolate all functionalities in specific nodes. I also need a good notification system to receive notifications for any node. So, in the next diagram I show the components that I'll need in this new version: As you can see, there are more nodes. Here are some notes about them: Dashboard: Controls trackers statuses and send a single task to each of them (under user request). The trackers will use this task until replaced with a new one (if done, not each 15 minutes like before). Search engine: I need to store all the tweets. They are firstly stored in a local database for each tracker but after that I'm thinking on using something like Elasticsearch to be able to do fast searches. Tweet processor: Just and isolated component with its own database (maybe something like the search engine to have fast access to info generated by the module). In the future more could be added. Application UI: A web application with a shared database with the Dashboard (mainly to store users information and preferences). Indeed, both could be merged into a single web. The main difference with the previous version of the project is that now they will be isolated and they will only show information and send requests. I will not do any heavy task in them (like process tweets as I did before). So, having this components, my main headache is how to structure all to not have to rewrite a lot of code every time I need to access any new data. Another headache is how can I interconnect nodes. I could use sockets but that is a pain in the ass. Maybe a REST layer? And finally, if all the nodes are isolated, how could I generate notifications for each user which info is only in the database used by the Application UI? I'm programming this using Java and Spring (at least I used them in the last version) but I have no problems with changing the language if I can take advantage of a tool/library/engine to make my life easier and have a better platform. Any comment will be appreciated.

  Read the article

• Synchronizing local and remote cache in distributed caching

  - by ltfishie

  With a distributed cache, a subset of the cache is kept locally while the rest is held remotely. In a get operation, if the entry is not available locally, the remote cache will be used and and the entry is added to local cache. In a put operation, both the local cache and remote cache are updated. Other nodes in the cluster also need to be notified to invalidate their local cache as well. What's a simplest way to achieve this if you implemented it yourself, assuming that nodes are not aware of each other. Edit My current implementation goes like this: Each cache entry contains a time stamp. Put operation will update local cache and remote cache Get operation will try local cache then remote cache A background thread on each node will check remote cache periodically for each entry in local cache. If the timestamp on remote is newer overwrite the local. If entry is not found in remote, delete it from local.

  Read the article

• Scalable distributed file system for blobs like images and other documents

  - by Pinnacle

  Cassandra & HBase both do not efficiently support storage of blobs like images. Storing directly on HDFS stresses the Namenode because of huge number of files. Facebook uses Haystack for images and attachments storage, but this is not open source. So is Lustre a good choice for distributed blob storage? I have read that Amazon S3 is used by many, but this would cost money and personally, I would not like to rely on third party system. What are other suggestions?

  Read the article

• Using EC2 instance as main development platform

  - by David

  My problem I am working as a consultant for various companies. Each company provides me with a laptop with their software on and I also have my own, where I have my development environment. I tend to buy a new laptop every second year and find myself spending lots of time configuring and installing software. I also spend a lot of time waiting for my laptop to process things. To solve all these issues, I am now considering using EC2 (running windows instances) as my main development platform and just access this from any PC I happen to be at. I calculated that running the Large instance (cheapest 64-bit) for 8 hours a day for a year costs me 960$ per year, which is acceptable. I imagine that when I approach the workplace each day, I will make a single tap on my phone to fire up the instance, so it is ready when I get to work. I should have different icons on my phone to fire up the various instance types. The same software should of course automatically be loaded on the various hardware (sometimes I would even need their instance with 68.4 GB of memory). Another advantage is that if I am having a specific problem with my instance, I could fire up another instance and have someone look into the problem and update the image. My question: Does anyone have experience with such a setup on EC2? What kind of problems do you foresee?

  Read the article

• Erlang node acts like it connects, but doesn't [migrated]

  - by Malfist

  I'm trying to setup a distributed network of nodes across a few firewalls and it's not going so well. My application is structured like this: there is a central server that always running a node ([email protected]) and my co-worker's laptops connect to it on startup. This works if we're all in the office, but if someone is at home, they can connect to the masternode, but they fail to connect to the other nodes in the swarm. I.E., erlang fails to gossip correctly. To correct this, I've change epmd's port number and changed the inet_dist_listen ports to a known open port (1755 and 7070 respectively). However, something fishy is going on. I can run net_adm:world() and it reports that it connects to master node, but when I run nodes() I get an empty array. Same with net_adm:ping('[email protected]'). See: Eshell V5.9 (abort with ^G) ([email protected])1> net_adm:world(). ['[email protected]'] ([email protected])2> nodes(). [] ([email protected])3> net_adm:ping('[email protected]'). pong ([email protected])4> nodes(). [] ([email protected])5> What's going on, and how can I fix it?

  Read the article

• How to determine the source of a request in a distributed service system?

  - by Kabumbus

  Map/Reduce is a great concept for sorting large quantities of data at once. What to do if you have small parts of data and you need to reduce it all the time? Simple example - choosing a service for request. Imagine we have 10 services. Each provides services host with sets of request headers and post/get arguments. Each service declares it has 30 unique keys - 10 per set. service A: name id ... Now imagine we have a distributed services host. We have 200 machines with 10 services on each. Each service has 30 unique keys in there sets. but now to find to which service to map the incoming request we make our services post unique values that map to that sets. We can have up to or more than 10 000 such values sets on each machine per each service. service A machine 1 name = Sam id = 13245 ... service A machine 1 name = Ben id = 33232 ... ... service A machine 100 name = Ron id = 777888 ... So we get 200 * 10 * 30 * 30 * 10 000 == 18 000 000 000 and we get 500 requests per second on our gateway each containing 45 items 15 of which are just noise. And our task is to find a service for request (at least a machine it is running on). On all machines all over cluster for same services we have same rules. We can first select to which service came our request via rules filter 10 * 30. and we will have 200 * 30 * 10 000 == 60 000 000. So... 60 mil is definitely a problem... I hope to get on idea of mapping 30 * 10 000 onto some artificial neural network alike Perceptron that outputs 1 if 30 words (some hashes from words) from the request are correct or if less than Perceptron should return 0. And I’ll send each such Perceptron for each service from each machine to gateway. So I would have a map Perceptron <-> machine for each service. Can any one tall me if my Perceptron idea is at least “sane”? Or normal people do it some other way? Or if there are better ANNs for such purposes?

  Read the article

• How can i manage my personal notes , code snippets files in one place online [closed]

  - by user1758043

  Whenever i work on any project , then i have so much notes , diagrams files , image s, brainstorming ideas which i want to keep. i want to put them in one place so that i can see the history of my work. Is there any toll whichere i can store this online. my company is using confluence but thats costly for me. I want something for single user but online in clou where i can store Notes Code snippets Diagrams , flowchart Attah files , images Books marks , sites

  Read the article

• Upcoming Speaking Engagements

  - by gsusx

  This summer, I took a brief break from speaking engagements to focus on shipping our new software in Tellago Studios and not stress my already hectic travel schedule. However, I’ve accepted a few invites to speak at different conferences during the fall and winter. Here is a brief list of the ones that are already confirmed: Software Architect Conference (London) http://www.software-architect.co.uk NodeJS for the .Net Developer I am a .NET developer but I have an iPhone and an Android Oredev (Malmö...(read more)

  Read the article

• How to manage many mobile device users at server side?

  - by Rami

  I built a social Android application in which users can see other users around them by GPS location. At the beginning thing went well as I had low number of users, but now that I have increasing number of users (about 1500 +100 every day) it has revealed a major problem in my design. In my Google App Engine servlet I have static HashMap that holds all the users profiles objects, currently 1500 and this number will increase as more users register. Why I'm doing it? Every user that requests for the users around him compares his GPS with other users and checks if they are in his 10km radius. This happens every five minutes on average. Consequently, I can't get the users from db every time because GAE read/write operation quota will tear me apart. The problem with this design is? As the number of users increases, the Hashmap turns to null every 4-6 hours, I think that this time is getting shorter, but I'm not sure. I'm fixing this by reloading the users from the db every time I detect that it becomes null, but this causes DOS to my users for 30 sec, so I'm looking for better solution. I'm guessing that it happens because the size of the hashmap. Am I right? I have been advised to use a spatial database, but that means that I can't work with GAE any more and it means that I need to build my big server all over again and lose my existing DB. Is there something I can do with the existing tools? Thanks.

  Read the article

• Any frameworks or library allow me to run large amount of concurrent jobs schedully?

  - by Yoga

  Are there any high level programming frameworks that allow me to run large amount of concurrent jobs schedully? e.g. I have 100K of urls need to check their uptime every 5 minutes Definitely I can write a program to handle this, but then I need to handle concurrency, queuing, error handling, system throttling, job distribution etc. Will there be a framework that I only focus on a particular job (i.e. the ping task) and the system will take care of the scaling and error handling for me? I am open to any language.

  Read the article

• How to rewrite a TCP MMOG server designed to run in a single machine, in a distributed way?

  - by Dokkat

  I have a MMOG server running on C++, using winsockets. My server won't support more than 200 players. I had the idea of redesigning it so it will use multiple servers instead of one, so, maybe, for example, each server could take care of a number of players, and, if it was too laggy, it could transfer the responsability of that player to other server. I'm not sure of how to program a consistent game logic like that, though. Are there techniques for this?

  Read the article

• Use a custom value object or a Guid as an entity identifier in a distributed system?

  - by Kazark

  tl;dr I've been told that in domain-driven design, an identifier for an entity could be a custom value object, i.e. something other than Guid, string, int, etc. Can this really be advisable in a distributed system? Long version I will invent an situation analogous to the one I am currently facing. Say I have a distributed system in which a central concept is an egg. The system allows you to order eggs and see spending reports and inventory-centric data such as quantity on hand, usage, valuation and what have you. There area variety of services backing these behaviors. And say there is also another app which allows you to compose recipes that link to a particular egg type. Now egg type is broken down by the species—ostrich, goose, duck, chicken, quail. This is fine and dandy because it means that users don't end up with ostrich eggs when they wanted quail eggs and whatnot. However, we've been getting complaints because jumbo chicken eggs are not even close to equivalent to small ones. The price is different, and they really aren't substitutable in recipes. And here we thought we were doing users a favor by not overwhelming them with too many options. Currently each of the services (say, OrderSubmitter, EggTypeDefiner, SpendingReportsGenerator, InventoryTracker, RecipeCreator, RecipeTracker, or whatever) are identifying egg types with an industry-standard integer representation the species (let's call it speciesCode). We realize we've goofed up because this change could effect every service. There are two basic proposed solutions: Use a predefined identifier type like Guid as the eggTypeID throughout all the services, but make EggTypeDefiner the only service that knows that this maps to a speciesCode and eggSizeCode (and potentially to an isOrganic flag in the future, or whatever). Use an EggTypeID value object which is a combination of speciesCode and eggSizeCode in every service. I've proposed the first solution because I'm hoping it better encapsulates the definition of what an egg type is in the EggTypeDefiner and will be more resilient to changes, say if some people now want to differentiate eggs by whether or not they are "organic". The second solution is being suggested by some people who understand DDD better than I do in the hopes that less enrichment and lookup will be necessary that way, with the justification that in DDD using a value object as an ID is fine. Also, they are saying that EggTypeDefiner is not a domain and EggType is not an entity and as such should not have a Guid for an ID. However, I'm not sure the second solution is viable. This "value object" is going to have to be serialized into JSON and URLs for GET requests and used with a variety of technologies (C#, JavaScript...) which breaks encapsulation and thus removes any behavior of the identifier value object (is either of the fields optional? etc.) Is this a case where we want to avoid something that would normally be fine in DDD because we are trying to do DDD in a distributed fashion? Summary Can it be a good idea to use a custom value object as an identifier in a distributed system (solution #2)?

  Read the article

• What is lightweight lock in distributed shared memory systems?

  - by Kutluhan Metin

  I started reading Tanenbaum's Distributed Systems book a while ago. I read about two phase locking and timestamp reordering in transactions chapter. While having a deeper look from google I heard of lightweight transactions/lightweight transactional memory. But I couldn't find any good explanation and implementation. So what is lightweight memory? What are the benefits of lightweight locks? And how can I implement them?

  Read the article

< Previous Page | 41 42 43 44 45 46 47 48 49 50 51 52  | Next Page >