Search Results

Search found 1781 results on 72 pages for 'cluster'.

Page 56/72 | < Previous Page | 52 53 54 55 56 57 58 59 60 61 62 63  | Next Page >

• ZooKeeper and RabbitMQ/Qpid together - overkill or a good combination?

  - by Chris Sears

  Greetings, I'm evaluating some components for a multi-data center distributed system. We're going to be using message queues (via either RabbitMQ or Qpid) so agents can make asynchronous requests to other agents without worrying about addressing, routing, load balancing or retransmission. In many cases, the agents will be interacting with components that were not designed for highly concurrent access, so locking and cross-agent coordination will be needed to avoid race conditions. Also, we'd like the system to automatically respond to agent or data center failures. With the above use cases in mind, ZooKeeper seemed like it might be a good fit. But I'm wondering if trying to use both ZK and message queuing is overkill. It seems like what Zookeeper does could be accomplished by my own cluster manager using AMQP messaging, but that would be hard to get really right. On the other hand, I've seen some examples where ZooKeeper was used to implement message queuing, but I think RabbitMQ/Qpid are a more natural fit for that. Has anyone out there used a combination like this? Thanks in advance, -Chris

  Read the article

• NLB and Host Header Value

  - by Hafeez

  Background: We are using MOSS 2007 in farm configuration, 2 WFE, 1 Indexer and SQL Server. MS NLB is used for load balancing. Host header value mapped to Virtual IP of Cluster in DNS, is used while creating the web applications in MOSS and all are sharing port 80. Problem: When client tries to access the web application that are configured with host header values. Both of WFEs Hangs for 5 minutes, they stop responding to ping and browser shows 'Page not found'. In the Application Log on the WFE, this error is registered "provider: TCP Provider, error: 0 - The semaphore timeout period has expired". Interestingly, the web application with no host header value and hosted on different ports is working correctly. Any clue to solve this problem will be helpful. Thks. Hafeez

  Read the article

• Hadoop on windows server

  - by Luca Martinetti

  Hello, I'm thinking about using hadoop to process large text files on my existing windows 2003 servers (about 10 quad core machines with 16gb of RAM) The questions are: Is there any good tutorial on how to configure an hadoop cluster on windows? What are the requirements? java + cygwin + sshd ? Anything else? HDFS, does it play nice on windows? I'd like to use hadoop in streaming mode. Any advice, tool or trick to develop my own mapper / reducers in c#? What do you use for submitting and monitoring the jobs? Thanks

  Read the article

• MYSQL KEY-VALUE PAIR Viability

  - by Amit

  Hi, I am new to mysql and I am looking for some answers to the follwoing questions: a) Can mysql community server can be leveraged for a key-value pair type database.?? b) Which mysql engine is best suited for a key-value pair type database ?? c) Is Mysql cluster a must for horizontal scaling of key-value based datastore or can it be acheived using MySQL replication?? d) Are there any docs or whitepapers for best practices when implementiing a kv datastore on mysql?? e) Are there any known big implementations other that friendfeed doing kv pair using MYSQL?? Would really appreciate some advise from all you Mysql gurus out there !! Thanks In Advance, Amit

  Read the article

• Out of memory error while using clusterdata in MATLAB

  - by Hossein

  Hi, I am trying to cluster a Matrix (size: 20057x2).: T = clusterdata(X,cutoff); but I get this error: ??? Error using == pdistmex Out of memory. Type HELP MEMORY for your options. Error in == pdist at 211 Y = pdistmex(X',dist,additionalArg); Error in == linkage at 139 Z = linkagemex(Y,method,pdistArg); Error in == clusterdata at 88 Z = linkage(X,linkageargs{1},pdistargs); Error in == kmeansTest at 2 T = clusterdata(X,1); can someone help me. I have 4GB of ram, but think that the problem is from somewhere else..

  Read the article

• Getting clusters of rows close together in time

  - by Mike

  I have a table basically like so ID | ItemID | Start | End | --------------------------------------------------------------- 1 234 10/20/09 8:34:22 10/20/09 8:35:10 2 274 10/20/09 8:35:30 10/20/09 8:36:27 3 272 10/21/09 12:15:00 10/21/09 12:17:00 4 112 10/21/09 12:20:14 10/21/09 12:21:21 5 15 10/21/09 12:22:39 10/21/09 12:24:15 There are two "clusters" of entries here, 1-2 and 3-5 separated by a gap in time, specifically 30 minutes is what I'm interested in. What I would like is the first and last rows of the cluster of entries. This is fairly easy to achieve by retrieving all the rows and looping through them in order of start time, but I'd like to have it in SQL if possible. I'm using SQL Server 2008, thanks.

  Read the article

• Essential skills of a Data Scientist

  - by harshsinghal

  I would like to know more about the relevant skills in the arsenal of a Data Scientist, and with new technologies coming in every day, how one picks and chooses the essentials. A few ideas germane to this discussion: Knowing SQL and the use of a DB such as MySQL, PostgreSQL was great till the advent of NoSql and non-relational databases. MongoDB, CouchDB etc. are becoming popular to work with web-scale data. Knowing a stats tool like R is enough for analysis, but to create applications one may need to add Java, Python, and such others to the list. Data now comes in the form of text, urls, multi-media to name a few, and there are different paradigms associated with their manipulation. What about cluster computing, parallel computing, the cloud, Amazon EC2, Hadoop ? OLS Regression now has Artificial Neural Networks, Random Forests and other relatively exotic machine learning/data mining algos. for company Thoughts?

  Read the article

• Unable to run OpenMPI across more than two machines

  - by rcollyer

  When attempting to run the first example in the boost::mpi tutorial, I was unable to run across more than two machines. Specifically, this seemed to run fine: mpirun -hostfile hostnames -np 4 boost1 with each hostname in hostnames as <node_name> slots=2 max_slots=2. But, when I increase the number of processes to 5, it just hangs. I have decreased the number of slots/max_slots to 1 with the same result when I exceed 2 machines. On the nodes, this shows up in the job list: <user> Ss orted --daemonize -mca ess env -mca orte_ess_jobid 388497408 \ -mca orte_ess_vpid 2 -mca orte_ess_num_procs 3 -hnp-uri \ 388497408.0;tcp://<node_ip>:48823 Additionally, when I kill it, I get this message: node2- daemon did not report back when launched node3- daemon did not report back when launched The cluster is set up with the mpi and boost libs accessible on an NFS mounted drive. Am I running into a deadlock with NFS? Or, is something else going on?

  Read the article

• Estimate serialization size of objects?

  - by Stefan K.

  In my thesis, I woud like to enhance messaging in a cluster. It's important to log runtime information about how big a message is (should I prefer processing local or remote). I could just find frameoworks about estimating the object memory size based on java instrumentation. I've tested classmexer, which didn't come close to the serialization size and sourceforge SizeOf. In a small testcase, SizeOf was around 10% wrong and 10x faster than serialization. (Still transient breaks the estimation completely and since e.g. ArrayList is transient but is serialized as an Array, it's not easy to patch SizeOf. But I could live with that) On the other hand, 10x faster with 10% error doesn't seem very good. Any ideas how I could do better?

  Read the article

• infoWindow on MarkerClusterer

  - by vishwanath

  I need infoWindow to be opened instead of zooming in map, when clicking on the ClusterMarker. I am using Gmaps util library MarkerClusterer for creating cluster of markers. I tried changing following line in markerclusterer.js ClusterMarker_.prototype = new GOverlay(); with ClusterMarker_.prototype = new GMarker(); so that I can get the openInfoWindow() function in the clustermarker, but that didnt worked out. Got some error. If possible, Please suggest solution so that this can be done with MarkerClusterer. Or else any other library which will be able to do this. Any help will be appreciated.

  Read the article

• Does anyone know a better alternative to MS Excel's Solver?

  - by tundal45

  My company has to crunch a lot of data and part of the process involves running the solver and plotting a graph through resulting data points. Obviously there is a lot of copy and paste involved and the whole process is shaky, error prone and all round cluster-fudge. I was wondering if there was an alternative to the solver that can be used so that even if we have to use excel to plot the final graph, there will be a lot less data that needs to be copied and pasted back and forth. It would be great especially if the tool could be easily integrated into a .NET application but I am open to suggestions that may require a little bit of code-fu to get this to work. Thanks!

  Read the article

• Parallelism in Python

  - by fmark

  What are the options for achieving parallelism in Python? I want to perform a bunch of CPU bound calculations over some very large rasters, and would like to parallelise them. Coming from a C background, I am familiar with three approaches to parallelism: Message passing processes, possibly distributed across a cluster, e.g. MPI. Explicit shared memory parallelism, either using pthreads or fork(), pipe(), et. al Implicit shared memory parallelism, using OpenMP. Deciding on an approach to use is an exercise in trade-offs. In Python, what approaches are available and what are their characteristics? Is there a clusterable MPI clone? What are the preferred ways of achieving shared memory parallelism? I have heard reference to problems with the GIL, as well as references to tasklets. In short, what do I need to know about the different parallelization strategies in Python before choosing between them?

  Read the article

• Windows/.NET Load Distribution & Balancing

  - by andrewbadera

  Hi all, Is there a vetted Windows-friendly, or even .NET-native, load-distributing/load-balancing utility out there along the lines of HA Proxy? We have a .NET stack product, and the one piece that we step out of the stack is for load-balancing. We need something with configurable rules for distribution -- perhaps subdomain-driven -- that NLB alone doesn't seem to offer. If it integrates directly with .NET, or offers an exposed API callable by webservices, so much the better! Thanks in advance! Clarification: we need to logically part over boxes. This is not just a cluster/failover/replication scenario.

  Read the article

• Hadoop on Amazon EC2 : Job tracker not starting properly

  - by Algorist

  Hi, We are running Hadoop on Amazon EC2 cluster. We start the master, slaves and attach the ebs volumes and finally waiting for hadoop jobtracker, tasktracker etc to start and we have timeout of 3600 seconds. We are noticing 50% of the time that job tracker is not able to start before the timeout. Reason being, hdfs is not initialized properly and still in safemode and job tracker is unable to start. I noticed few connectivity issues between nodes on EC2 as I tried manually pinging slaves. Did anyone face similar issue and know how to solve this? Thank you Bala

  Read the article

• Clojure / HBase: How to Import HBaseTestingUtility in v0.94.6.1

  - by David Williams

  In Clojure, if I want to start a test cluster using the hbase testing utility, I have to annotate my dependencies with: [org.apache.hbase/hbase "0.92.2" :classifier "tests" :scope "test"] First of all, I have no idea what this means. According to leiningens sample project.clj ;; Dependencies are listed as [group-id/name version]; in addition ;; to keywords supported by Pomegranate, you can use :native-prefix ;; to specify a prefix. This prefix is used to extract natives in ;; jars that don't adhere to the default "<os>/<arch>/" layout that ;; Leiningen expects. Question 1: What does that mean? Question 2: If I upgrade the version: [org.apache.hbase/hbase "0.94.6.1" :classifier "tests" :scope "test"] Then I receive a ClassNotFoundException Exception in thread "main" java.lang.ClassNotFoundException: org.apache.hadoop.hbase.HBaseConfiguration Whats going on here and how do I fix it?

  Read the article

• MPAPI vs MPI.NET vs ?

  - by Olexandr

  I'm working on college project. I have to develop distributed computing system. And i decided to do some research to make this task fun :) I've found MPAPI and MPI.NET libraries. Yes, they are .NET libraries(Mono, in my case). Why .NET ? I'm choosing between Ada, C++ and C# so to i've choosed C# because of lower development time. I have two goals: Simplicity; Performance; Cluster computing. So, what to choose - MPAPI or MPI.NET or something else ?

  Read the article

• java.io.IOException: Invalid argument

  - by Luixv

  Hi I have a web application running in cluster mode with a load balancer. It consists in two tomcats (T1, and T2) addressing only one DB. T2 is nfs mounted to T1. This is the only dofference between both nodes. I have a java method generating some files. If the request runs on T1 there is no problem but if the request is running on node 2 I get an exception as follows: java.io.IOException: Invalid argument at java.io.FileOutputStream.close0(Native Method) at java.io.FileOutputStream.close(FileOutputStream.java:279) The corresponding code is as follows: for (int i = 0; i < dataFileList.size(); i++) { outputFileName = outputFolder + fileNameList.get(i); FileOutputStream fileOut = new FileOutputStream(outputFileName); fileOut.write(dataFileList.get(i), 0, dataFileList.get(i).length); fileOut.flush(); fileOut.close(); } The exception appears at the fileOut.close() Any hint? Luis

  Read the article

• DynaCache invalidation in clustered environment

  - by Ravi

  We are using Horizontal cluster in our PROD with WAS 6.1 as the Application server. We have enabled dynacache service for some of the JSP fragments using SHARED-PUSH in cachespec.xml file. Now we want to do cache invalidation programmatically..ie. whenever something changes in DB related to cache the cache should get invalidated. so can you please let me know what steps are involved in it to achieve this? any configuration settings at server side or any development changes.

  Read the article

• Efficiency of while(true) ServerSocket Listen

  - by Submerged

  I am wondering if a typical while(true) ServerSocket listen loop takes an entire core to wait and accept a client connection (Even when implementing runnable and using Thread .start()) I am implementing a type of distributed computing cluster and each computer needs every core it has for computation. A Master node needs to communicate with these computers (invoking static methods that modify the algorithm's functioning). The reason I need to use sockets is due to the cross platform / cross language capabilities. In some cases, PHP will be invoking these java static methods. I used a java profiler (YourKit) and I can see my running ServerSocket listen thread and it never sleeps and it's always running. Is there a better approach to do what I want? Or, will the performance hit be negligible? Please, feel free to offer any suggestion if you can think of a better way (I've tried RMI, but it isn't supported cross-language. Thanks everyone

  Read the article

• ssh script gives "key_read" error

  - by lugte098

  I'm using a script that connects to a cluster through ssh and sends some commands, then quits the connection. This script basically connects once using ssh, then executes a script in this session. This script loops through a list of commands a few times and after it is finished, the connection is terminated. So this script works fine, except for the fact that after a few loops it gives me the following error at loop 22. And then again at loop 32. The loops do exactly the same thing, so i cannot grasp the problem the script is facing. This is the error: key_read: uudecode AAAAB3NzaC1yc2EAAAABIwAAAQEAxmNx2hcXLpTjuaa3yKC3B9gbF7KprP2/ CH8fBgMbCyIcOB+ZMQDmEnbVTqedBwV/mxjZzorEpHTM8MX2WsTjFsxwzDgcpuxm+3cwfb0WSy9Y4Kb F8crAsRDbBIpUZ2n/iSdRcds9nTjk6PA61kTS24RLACHpqF18vudlO5WcbCOnAwa+DdUs0Raw29UiQc BaC6M4YPnApq9Ayy7a6qFI2uK6efkwfLTZIDivWlIdLpRLEyuBEpozQQhEd0mrGhR/ Gl1GevRvFMms14130xQ4A5UpJSn6CmrRIWBkcgp1TilqDGQ1F5xZOinnc4C00gFrbT3hkkQqY5A9p node023,10.141.0.31 ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAxmNx2hcXLpTjuaa3yKC3 B9gbF7KprP2/CH8fBgMbCyIcOB+ZMQDmEnbVTqedBwV/mxjZzorEpHTM8MX2WsTjFsxwzDgcpuxm+ 3cwfb0WSy9Y4KbF8crAsRDbBIpUZ2n/iSdRcds9nTjk6PA61kTS24RLACHpqF18vudlO5WcbCOnAw a+DdUs0Raw29UiQcBaC6M4YPnApq9Ayy7a6qFI2uK6efkwfLTZIDivWlIdLpRLEyuBEpozQQhEd0m rGhR/Gl1GevRvFMms14130xQ4A5UpJSn6CmrRIWBkcgp1TilqDGQ1F5xZOinnc4C00gFrbT3hkkQqY5 A9pa0lQHFkSw==

  Read the article

• Is AMQP suitable as both an intra and inter-machine software bus?

  - by Bwooce

  I'm trying to get my head around AMQP. It looks great for inter-machine (cluster, LAN, WAN) communication between applications but I'm not sure if it is suitable (in architectural, and current implementation terms) for use as a software bus within one machine. Would it be worth pulling out a current high performance message passing framework to replace it with AMQP, or is this falling into the same trap as RPC by blurring the distinction between local and non-local communication? I'm also wary of the performance impacts of using a WAN technology for intra-machine communications, although this may be more of an implementation concern than architecture. War stories would be appreciated.

  Read the article

• SQL Server 2008 BULK INSERT causes more reads than writes. Why?

  - by sh1ng

  I've huge a table (a few billion rows) with a clustered index and two non-clustered indices. A BULK INSERT operation produces 112000 reads and only 383 writes (duration 19948ms). It's very confusing to me. Why do reads exceed writes? How can I reduce it? update query insert bulk DenormalizedPrice4 ([DP_ID] BigInt, [DP_CountryID] Int, [DP_OperatorID] SmallInt, [DP_OperatorPriceID] BigInt, [DP_SpoID] Int, [DP_TourTypeID] Int, [DP_CheckinDate] Date, [DP_CurrencyID] SmallInt, [DP_Cost] Decimal(9,2), [DP_FirstCityID] Int, [DP_FirstHotelID] Int, [DP_FirstBuildingID] Int, [DP_FirstHotelGlobalStarID] Int, [DP_FirstHotelGlobalMealID] Int, [DP_FirstHotelAccommodationTypeID] Int, [DP_FirstHotelRoomCategoryID] Int, [DP_FirstHotelRoomTypeID] Int, [DP_Days] TinyInt, [DP_Nights] TinyInt, [DP_ChildrenCount] TinyInt, [DP_AdultsCount] TinyInt, [DP_TariffID] Int, [DP_DepartureCityID] Int, [DP_DateCreated] SmallDateTime, [DP_DateDenormalized] SmallDateTime, [DP_IsHide] Bit, [DP_FirstHotelAccommodationID] Int) with (CHECK_CONSTRAINTS) No triggers & foreign keys Cluster Index by DP_ID and two non-unique indexes(with fillfactor=90%) And one more thing DB stored on RAID50 with stripe size 256K

  Read the article

• EJB3.1 Remote invocation - is it distributed automatically? is it expensive?

  - by Hank

  I'm building a JEE6 application with performance and scalability in the forefront of my mind. Business logic and JPA2-facade is held in stateless session beans (EJB3.1). As of right now, the SLSBs implement only @Remote-interfaces. When a bean needs to access another bean, it does so via RMI. My reasoning behind this is the assumption that, once the application runs on a bunch of clustered application servers, the RMI-part allows the execution to be distributed across the whole cluster automagically. Is that a correct assumption? I'm fine with dealing with the downsides of that (objects lose entityManager session, pass-by-value), at least I think so. But I am wondering if constant remote invocation isn't adding more load then necessary.

  Read the article

• Orbital equations, and power required to run them

  - by Adam Davis

  Due to a discussion on the SO IRC today, I'm curious about orbital mechanics, and The equations needed to solve orbital problems The computing power required to solve complex problems The question in particular is calculating when the Earth will plow into the Sun (or vice versa, depending on the frame of reference). I suspect that all the gravitational pulls within our solar system may need to be calculated, which makes me wonder what type of computer cluster is required, or can this be done on a single box? I don't have the experience to do a back of the napkin test here, but perhaps you do? Also, much thx to Gortok for the original inspiration (see comments).

  Read the article

• Genetic programming in c++, library suggestions?

  - by shuttle87

  I'm looking to add some genetic algorithms to an Operations research project I have been involved in. Currently we have a program that aids in optimizing some scheduling and we want to add in some heuristics in the form of genetic algorithms. Are there any good libraries for generic genetic programming/algorithms in c++? Or would you recommend I just code my own? I should add that while I am not new to c++ I am fairly new to doing this sort of mathematical optimization work in c++ as the group I worked with previously had tended to use a proprietary optimization package. We have a fitness function that is fairly computationally intensive to evaluate and we have a cluster to run this on so parallelized code is highly desirable. So is c++ a good language for this? If not please recommend some other ones as I am willing to learn another language if it makes life easier. thanks!

  Read the article

< Previous Page | 52 53 54 55 56 57 58 59 60 61 62 63  | Next Page >