Search Results

Search found 12189 results on 488 pages for 'ds store'.

Page 134/488 | < Previous Page | 130 131 132 133 134 135 136 137 138 139 140 141  | Next Page >

• HDFS some datanodes of cluster are suddenly disconnected while reducers are running

  - by user1429825

  I have 8 slave computers and 1 master computer for running Hadoop (ver 0.21) some datanodes of cluster are suddenly disconnected while I was running MapReduce code on 10GB data After all mappers finished and around 80% of reducers was processed, randomly one or more datanode disconned from network. and then the other datanodes start to disappear from network even if I killed the MapReduce job when I found some datanode was disconnected. I've tried to change dfs.datanode.max.xcievers to 4096, turned off fire-walls of all computing node, disabled selinux and increased the number of file open limit to 20000 but they didn't work at all... anyone have a idea to solve this problem? followings are error log from mapreduce 12/06/01 12:31:29 INFO mapreduce.Job: Task Id : attempt_201206011227_0001_r_000006_0, Status : FAILED java.io.IOException: Bad connect ack with firstBadLink as ***.***.***.148:20010 at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:889) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:820) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427) and followings are logs from datanode 2012-06-01 13:01:01,118 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_-5549263231281364844_3453 src: /*.*.*.147:56205 dest: /*.*.*.142:20010 2012-06-01 13:01:01,136 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(*.*.*.142:20010, storageID=DS-1534489105-*.*.*.142-20010-1337757934836, infoPort=20075, ipcPort=20020) Starting thread to transfer block blk_-3849519151985279385_5906 to *.*.*.147:20010 2012-06-01 13:01:19,135 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(*.*.*.142:20010, storageID=DS-1534489105-*.*.*.142-20010-1337757934836, infoPort=20075, ipcPort=20020):Failed to transfer blk_-5797481564121417802_3453 to *.*.*.146:20010 got java.net.ConnectException: > Connection timed out at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:373) at org.apache.hadoop.hdfs.server.datanode.DataNode$DataTransfer.run(DataNode.java:1257) at java.lang.Thread.run(Thread.java:722) 2012-06-01 13:06:20,342 INFO org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification succeeded for blk_6674438989226364081_3453 2012-06-01 13:09:01,781 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(*.*.*.142:20010, storageID=DS-1534489105-*.*.*.142-20010-1337757934836, infoPort=20075, ipcPort=20020):Failed to transfer blk_-3849519151985279385_5906 to *.*.*.147:20010 got java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/*.*.*.142:60057 remote=/*.*.*.147:20010] at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246) at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:164) at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:203) at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks(BlockSender.java:388) at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:476) at org.apache.hadoop.hdfs.server.datanode.DataNode$DataTransfer.run(DataNode.java:1284) at java.lang.Thread.run(Thread.java:722) hdfs-site.xml <configuration> <property> <name>dfs.name.dir</name> <value>/home/hadoop/data/name</value> </property> <property> <name>dfs.data.dir</name> <value>/home/hadoop/data/hdfs1,/home/hadoop/data/hdfs2,/home/hadoop/data/hdfs3,/home/hadoop/data/hdfs4,/home/hadoop/data/hdfs5</value> </property> <property> <name>dfs.replication</name> <value>3</value> </property> <property> <name>dfs.datanode.max.xcievers</name> <value>4096</value> </property> <property> <name>dfs.http.address</name> <value>0.0.0.0:20070</value> <description>50070 The address and the base port where the dfs namenode web ui will listen on. If the port is 0 then the server will start on a free port. </description> </property> <property> <name>dfs.datanode.http.address</name> <value>0.0.0.0:20075</value> <description>50075 The datanode http server address and port. If the port is 0 then the server will start on a free port. </description> </property> <property> <name>dfs.secondary.http.address</name> <value>0.0.0.0:20090</value> <description>50090 The secondary namenode http server address and port. If the port is 0 then the server will start on a free port. </description> </property> <property> <name>dfs.datanode.address</name> <value>0.0.0.0:20010</value> <description>50010 The address where the datanode server will listen to. If the port is 0 then the server will start on a free port. </description> <property> <name>dfs.datanode.ipc.address</name> <value>0.0.0.0:20020</value> <description>50020 The datanode ipc server address and port. If the port is 0 then the server will start on a free port. </description> </property> <property> <name>dfs.datanode.https.address</name> <value>0.0.0.0:20475</value> </property> <property> <name>dfs.https.address</name> <value>0.0.0.0:20470</value> </property> </configuration> mapred-site.xml <configuration> <property> <name>mapred.job.tracker</name> <value>masternode:29001</value> </property> <property> <name>mapred.system.dir</name> <value>/home/hadoop/data/mapreduce/system</value> </property> <property> <name>mapred.local.dir</name> <value>/home/hadoop/data/mapreduce/local</value> </property> <property> <name>mapred.map.tasks</name> <value>32</value> <description> default number of map tasks per job.</description> </property> <property> <name>mapred.tasktracker.map.tasks.maximum</name> <value>4</value> </property> <property> <name>mapred.reduce.tasks</name> <value>8</value> <description> default number of reduce tasks per job.</description> </property> <property> <name>mapred.map.child.java.opts</name> <value>-Xmx2048M</value> </property> <property> <name>io.sort.mb</name> <value>500</value> </property> <property> <name>mapred.task.timeout</name> <value>1800000</value> <!-- 30 minutes --> </property> <property> <name>mapred.job.tracker.http.address</name> <value>0.0.0.0:20030</value> <description> 50030 The job tracker http server address and port the server will listen on. If the port is 0 then the server will start on a free port. </description> </property> <property> <name>mapred.task.tracker.http.address</name> <value>0.0.0.0:20060</value> <description> 50060 </property> </configuration>

  Read the article

• MySQL: Storage of multiple text fields for a record

  - by Tom

  An inexperienced question: I need to store about 10 unknown-length text fields per record into a MySQL table. I expect no more than 50K rows in total for this table but speed is important. The database actions will be solely SELECTs for all practical purposes. I'm using InnoDB. In other words: id | text1 | text2 | text3 | .... | text10 As I understand that MySQL will store the text elsewhere and use its own indicators on the table itself, I'm wondering whether there's any fundamental performance implications that I should be worrying about given the way the data is stored? (i.e. several "sub-fetches" from the table). Thank you.

  Read the article

• Linq to SQL gives NotSupportedException when using local variables

  - by zwanz0r

  It appears to me that it matters whether you use a variable to temporary store an IQueryable or not. See the simplified example below: This works: List<string> jobNames = new List<string> { "ICT" }; var ictPeops = from p in dataContext.Persons where ( from j in dataContext.Jobs where jobNames.Contains(j.Name) select j.ID).Contains(p.JobID) select p; But when I use a variable to temporary store the subquery I get an exception: List<string> jobNames = new List<string> { "ICT" }; var jobs = from j in dataContext.Jobs where jobNames.Contains(j.Name) select j.ID; var ictPeops = from p in dataContext.Persons where jobs.Contains(p.JobID) select p; "System.NotSupportedException: Queries with local collections are not supported" I don't see what the problem is. Isn't this logic that is supposed to work in LINQ?

  Read the article

• Saving highscores coupled to a facebook account

  - by Eiko

  I want to offer a "highscore" list for friends in my app (at this point iPhone/iPad), so that if the user connected to facebook, he will get a list with his/her friends scores. Connecting is easy, retrieving friends is easy, but figuring out the best way to store the scores is not. As it seems I need to the store the scores on my own server server, no big deal. But what is considered a reasonable safe way to transfer the data? When communicating with facebook, authentication is clear - but communicating with my server basically anyone could post scores for another user if I send user id and score. Obfuscation might help a bit, but is there any better way to make sure that the data comes from the fb-logged-in person? Scores can also go down, so changing scores for other persons won't necessary help them. Thanks :)

  Read the article

• Are bit operations quick?

  - by flashnik

  I'm dealing with a problem which needs to work with a lot of data. Currently its' values are represented as unsigned int. I know that real values do not exceed some limit, say 1000. That means that I can use unsigned short to store it. One profit is that it'll use less space. Do I have to pay for it by loosing in performance? Another assumption. I decided to store data as short but all calling functions use int, so I need to convert between these datatypes when storing/extracting values. Wiil the performance lost be dramatic? Third assumption. Due to great wish to econom memory I decided to use not short but just 10 bits packed into array of unsigned int. What will happen in this case comparing with previous ones?

  Read the article

• Storing multiple checkbox values in database

  - by Madjokr

  Hi, I want to store multiple column values in table.Lets take a example .. What are your favourite colors? the choices can be red,blue,green, orange. So lets assume, the user selects atleast 2 values. Is there any way to store the multiple values in table. I have implemented by concatinating choices of users in a column in the table. I later found that it is a bad practise. Currently i can think of using Bitwise operator and habtm. What are the different ways for storing multiple choices values in table? If I am implementing in rails, What is the best way to implement this with OOP concepts? Is there any builtin options in rails?

  Read the article

• Compare rows between 2 tables

  - by arthur

  I am new to SQL and I need to build a database for a grocery store(not real, just a course assignment) i have two fields from two different tables - supplied price - the price that the store buys from the supplier and price that is given to the customers How can I make a constraint that insures that supplied price is lower then the price that is given to the customers? The relevant tables that I have are: CREATE TABLE Supplied_Products( [Supplier ID] Int NOT NULL Foreign Key References Suppliers, [Product ID] Int NOT NULL Foreign Key References Products, Price Float NOT NULL, CHECK (Price0), Constraint PK_Supplied_Products PRIMARY KEY([Supplier ID] ,[Product ID]) ) CREATE TABLE Products( [Product-ID] Int NOT NULL PRIMARY KEY, [Product Name] Varchar(20) NOT NULL, Price Float NOT NULL, [Category-Name] Varchar(20) NOT NULL Foreign Key References Categories, [Weight] Float NOT NULL, [Is Refrigirated] Varchar(1) DEFAULT 'N' CHECK ([Is Refrigirated] in('Y','N')),/* Is Refrigirated can be only Y-yes or N-no*/ CHECK (Price 0) )

  Read the article

• Distributed, persistent cache using EHCache

  - by Richard

  I currently have a distributed cache using EHCache via RMI that works just fine. I was wondering if you can include persistence with the caches to create a distributed, persistent cache. Alongside this, if the cache was persistent, would it load from the file store, then bootstrap from the cache cluster? Basically, what I want is: Cache starts Cache loads persistent objects from the file store Cache joins the distruted cluster and bootstraps as normal The usecase behind this is having 2 identical components running on independent machines, distributing the cache to avoid losing data in the event that one of the components fails. The persistence would guard against losing all data on the rare occasion that both components fail. Would moving to another distribution method (such as Terracotta) support this?

  Read the article

• I'm working on a website that sells different artwork, what's the best way to handle different image

  - by ThinkingInBits

  I'm working on a website that will allow users to upload and sell their artwork in different sizes. I was wondering what the best way would be to handle the different file sizes automatically. A few points I was curious on: How to define different size categories (small, medium, large) in such a way that I'll be able to dynamically re-size images with proportional dimensions. Should I store actual jpegs of the different sizes for download? Or would it be easier to generate these different sizes for download on the fly My thumbnails will be somewhat larger than your average thumbnails, should I store a second 'thumbnail image' with the sites watermark overlaying it? Or once again, generate this on the fly? All opinions, advice are greatly appreciated!

  Read the article

• Zend Framework Router Getting /module/VALUE/controller/action

  - by cappuccino

  I've been googling around and I can't seem to find anything which explains the use of ZF router well. I've read the documentation on the site, which seems to only talk about re-routing. I am trying to make the format: /module/value/controller/action give /module/controller/action passing on value as a parameter e.g. /store/johnsmithbigsale/home/newstuff would route to /store/home/newstuff passing on johnsmithbigsale as the value to a parameter with a hidden namespace e.g. storeName. Some help would be greatful!

  Read the article

• List of private iPhone APIs?

  - by diego nunes

  . . Hi there, everybody. . . I need to do an app to be distributed ad hoc (it doesn't need to go to the store) but I need to get the information about the "data usage" (gprs/3g traffic). It is available on the system, but there is no official API call to get that info. One app made it through Apple testing (it's called "Download Meter"), though, and I emailed the guys to see if they would share the call, but they were not in that mood. . . Is there any list of private APIs or anything like that? Does anyone have any ideas of how could I get that info? Again: the app doesn't need to go to the store, but I need to install it on stock iPhone (ad hoc will do). . Thanks.

  Read the article

• C#/ASP.NET MVC 4 Instantiate Object Derived From Interface In Factory Method

  - by Chris

  Currently have a Factory class that features a GetSelector function, which returns a concrete implementation of ISelector. I have several different classes that implement ISelector and based on a setting I would like to receive the appropriate ISelector back. public interface ISelector { string GetValue(string Params); } public class XmlSelector : ISelector { public string GetValue(string Params) { // open XML file and get value } } public static class SelectorFactory { public static ISelector GetSelector() { return new XmlSelector(); // Needs changing to look at settings } } My question is what is the best way to store the setting? I am aware of using AppSettings etc. but I'm not sure whether I want to have to store strings in the web.config and perform a switch on it - just seems to be really tightly coupled in that if a new implementation of ISelector is made, then the Factory would need to be changed. Is there any way of perhaps storing an assembly name and instantiating based on that? Thanks, Chris

  Read the article

• don't wanna lose data on Android after uninstalling

  - by soclose

  Hi, Now I make a trial application. I'd like to store IMEI and other info in Android permanently. And I don't want to lose them after uninstalling it. I tested with shared preference but it deletes after un-installation. SharedPreferences settings = getSharedPreferences(PREFS_NAME, 0); SharedPreferences.Editor editor = settings.edit(); editor.putBoolean("silentMode", true); // Commit the edits! editor.commit(); let me know where to store.

  Read the article

• Is SQL server the best DB for Storing and comparing images in database for a small ecommerce applica

  - by iecut

  I have been trying to create a small e-commerce web based application using MS Dot Net framework. The application will let the user allow to store the image of their product that they want to sell or purchase, then they will have the option to upload the image of a particular product and compare that image with the similar images in the database. So my two main concerns are: - Is MS SQL a good option to store and compare the images. - Is the any other better database that can do the same work with less complexity of work and that is also easy to integrate with MS dot net framework.

  Read the article

• MySQL storing negative and positive decimals

  - by Shishant

  Hello, I want to be able to store -11.99 and +11.99 kind of values in mysql db I am thinking of decimals instead of varchar. But reading mysql site I found out that its incompatible with older versions of mysql As a result of the change from string to numeric format for DECIMAL storage, DECIMAL columns no longer store a leading + or - character or leading 0 digits. Before MySQL 5.0.3, if you inserted +0003.1 into a DECIMAL(5,1) column, it was stored as +0003.1. As of MySQL 5.0.3, it is stored as 3.1. For negative numbers, a literal - character is no longer stored. Applications that rely on the older behavior must be modified to account for this change. So what should be the data type, If I have to give up varchar and make it compatible with older versions too?

  Read the article

• I would like to make a field part of a many to many relationship

  - by jona

  I have the following table called stores id store zip 1 Market 1 01569 2 Market 2 01551 3 Market 3 10468 4 Market 4 10435 Zip code table INSERT INTO `zip_codes` (`restaurants_locations_id`, `zip`, `state`, `latitude`, `longitude`, `city`, `full_state`) VALUES (1, '06001', 'CT', ' 41.789698', ' -72.86431', 'Avon', 'Connecticut'), (2, '06002', 'CT', ' 41.832798', ' -72.72642', 'Bloomfield', 'Connecticut'), (3, '06010', 'CT', ' 41.682249', ' -72.93365', 'Bristol', 'Connecticut'), (4, '06013', 'CT', ' 41.758415', ' -72.94642', 'Burlington', 'Connecticut'), (5, '06016', 'CT', ' 41.909097', ' -72.54393', 'Windsorville', 'Connecticut'), (6, '06018', 'CT', ' 42.023510', ' -73.31103', 'Canaan', 'Connecticut'), (7, '06019', 'CT', ' 41.834247', ' -72.89174', 'Canton', 'Connecticut'), etc.... what I want is to create a table which will connect the zip field in the stores tables where one store will be found in several zicodes... How would that table would look like? Thanks

  Read the article

• Serializing ActiveRecord objects without storing their attributes?

  - by Allan Grant

  I'm working on a problem where I need to store serialized hierarchies of Ruby objects in the database. Many of the objects that will need to be saved are ActiveRecord objects with a lot of attributes. Instead of saving the entire objects and then refreshing their attributes from the DB when I load them (in case they changed, which is likely), it would be easier to just store the references (class and database id) for these objects. Does anyone know if there's already a way to do this in Rails, or if there's an existing gem for it? Wanted to check if something existed before spending a ton of time hacking on it.

  Read the article

• How can I dynamically inject code to event handlers in Delphi?

  - by mjustin

  For debugging / performance tests I would like to dynamically add logging code to all event handlers of components of a given type. For example, for all Dataset components located ona TDatamodule, I would like to add some code for the BeforeOpen and the AfterOpen event to store the start and end time and send a line to a logger with the elapsed time in the AfterOpen event. I would prefer to do this dynamically (no component subclassing), so that I can add this to all existing datamodules and forms with minimal effort only when needed. Iterating all components and filtering by their type is easy, but for the components which already have event handlers assigned, I need a way to store the existing event handlers, and assign a new modified event handler which first does the logging and then will invoke the original code which was already present. Is there a design pattern which can be applied, or even some example code which shows how to implement this in Delphi?

  Read the article

• Why is it bad to use boolean flags in databases? And what should be used instead?

  - by David Chanin

  I've been reading through some of guides on database optimization and best practices and a lot of them suggest not using boolean flags at all in the DB schema (ex http://forge.mysql.com/wiki/Top10SQLPerformanceTips). However, they never provide any reason as to why this is bad. Is it a peformance issue? is it hard to index or query properly? Furthermore, if boolean flags are bad, what should you use to store boolean values in a database? Is it better to store boolean flags as an integer and use a bitmask? This seems like it would be less readable.

  Read the article

• Multiplying char and int together in C part 2

  - by teehoo

  If I do the following: int c0 = CHAR_MAX; //8 bit int c1 = CHAR_MAX; //8-bit int i = c0*c1; //store in 32-bit variable printf("%d\n", i); //prints 16129 We can see that there is no problem with to 8-bit numbers being multiplied together, and producing a 32-bit output. However, if I do int i0 = INT_MAX; //32-bit int i1 = INT_MAX; //32 bit variable long long int ll = i0*i1; //store in 64-bit variable printf("%lld\n", ll); //prints 1..overflow!! In this case, two 32-bit variables were multiplied together, overflowed, and then were assigned to the 64-bit variable. So why did this overflow happen when multiplying the ints, but not the chars? Is it dependent on the default word-size of my machine? (32-bits)

  Read the article

• How do I set default values on new properties for existing entities after light weight core data migration?

  - by Moritz

  I've successfully completed light weight migration on my core data model. My custom entity Vehicle received a new property 'tirePressure' which is an optional property of type double with the default value 0.00. When 'old' Vehicles are fetched from the store (Vehicles that were created before the migration took place) the value for their 'tirePressure' property is nil. (Is that expected behavior?) So I thought: "No problem, I'll just do this in the Vehicle class:" - (void)awakeFromFetch { [super awakeFromFetch]; if (nil == self.tirePressure) { [self willChangeValueForKey:@"tirePressure"]; self.tirePressure = [NSNumber numberWithDouble:0.0]; [self didChangeValueForKey:@"tirePressure"]; } } Since "change processing is explicitly disabled around" awakeFromFetch I thought the calls to willChangeValueForKey and didChangeValueForKey would mark 'tirePresure' as dirty. But they don't. Every time these Vehicles are fetched from the store 'tirePressure' continues to be nil despite having saved the context.

  Read the article

• Extjs - Getting more from the server

  - by fatnjazzy

  Hi, Is it possible to get every data from the server? for example, i want to get the columns items from the server Via Ajax/Proxy by sending json string? thanks var grid = new Ext.grid.GridPanel({ store: store, columns: [ {id:'company',header: 'Company', width: 160, sortable: true, dataIndex: 'company'}, {header: 'Price', width: 75, sortable: true, renderer: 'usMoney', dataIndex: 'price'}, {header: 'Change', width: 75, sortable: true, renderer: change, dataIndex: 'change'}, {header: '% Change', width: 75, sortable: true, renderer: pctChange, dataIndex: 'pctChange'}, {header: 'Last Updated', width: 85, sortable: true, renderer: Ext.util.Format.dateRenderer('m/d/Y'), dataIndex: 'lastChange'} ], stripeRows: true, autoExpandColumn: 'company', height: 350, width: 600, title: 'Array Grid', stateful: true, stateId: 'grid' });

  Read the article

• XNA Music mixing real-time

  - by Adam L. S.

  I've created a "format" to store segments of music (prelude part, repeated part, ending part) and time information for these segments (offset, scored length) so I can mix it up in real-time as if it were one piece of music, while repeating the repeated part (optionally) indefinitely. This way, the segments can store decay where the next segment is played, while the previous one is finished. (I've created a player for this in Java, and used the Clip class.) I wanted this format, so I can provide a finite length music (for a jukebox feature), while I play infinite length music in-games. However, when I wanted to code a class in XNA that manages this "format" I've noticed, that there is no obvious way to play "Songs" simultaneously/overlapped. How can I do this/what is the best practice, not leaving the XNA framework? (I don't want to create infinite play-lists.)

  Read the article

• Converting "fr_FR" into "French (France)" in both directions. (Objective-C iPhone)

  - by Janice

  I can easily change a short region-code (en_US) into a longer string... but it there an easy way to also move in the other direction? [displayInEnglish displayNameForKey:NSLocaleIdentifier value:regionCountryCode]; "en_US" becomes "English (United States)". "English (United States)" becomes "en_US". I currently store the short region-code in a database.... but when I show some aggregate results... I need to display the longer strings to the user. Or should I just store the longer strings right in the database... and not even worry about "converting" them later? I'm trying to show a "dollars total" for each country. If you were a user... which would you more likely wish to see (for a currency-total list)? A "French" total A "France" total A "French (France)" total A "fr_Fr" total?

  Read the article

• Most Efficient Alternative Method of Storing Settings for iPhone Apps

  - by JPK

  I am not using the Settings bundle to store the settings for my app, as I prefer to allow the user to access the settings within the app (they may be changed fairly often). I do realize that there is the option to do both, but for now, I am trying to find the most optimal place to store the settings within the app. I have a good number of settings (from what I have read, probably too many for NSUserDefaults), and the two main options I am considering are: 1) storing the settings in a dictionary in the plist, loading the settings into a NSDictionary property in the app delegate and accessing them via the sharedDelegate 2) storing the settings in a Core Data entity (1 row on Settings entity), loading the settings into a Settings object in the app delegate and accessing them via the sharedDelegate Of these two, which would be the optimal method, performance wise?

  Read the article

< Previous Page | 130 131 132 133 134 135 136 137 138 139 140 141  | Next Page >