Search Results

Search found 215 results on 9 pages for 'dfs'.

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

Page 5/9 | < Previous Page | 1 2 3 4 5 6 7 8 9  | Next Page >

  • HDFS some datanodes of cluster are suddenly disconnected while reducers are running

    - by user1429825
    I have 8 slave computers and 1 master computer for running Hadoop (ver 0.21) some datanodes of cluster are suddenly disconnected while I was running MapReduce code on 10GB data After all mappers finished and around 80% of reducers was processed, randomly one or more datanode disconned from network. and then the other datanodes start to disappear from network even if I killed the MapReduce job when I found some datanode was disconnected. I've tried to change dfs.datanode.max.xcievers to 4096, turned off fire-walls of all computing node, disabled selinux and increased the number of file open limit to 20000 but they didn't work at all... anyone have a idea to solve this problem? followings are error log from mapreduce 12/06/01 12:31:29 INFO mapreduce.Job: Task Id : attempt_201206011227_0001_r_000006_0, Status : FAILED java.io.IOException: Bad connect ack with firstBadLink as ***.***.***.148:20010 at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:889) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:820) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:427) and followings are logs from datanode 2012-06-01 13:01:01,118 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block blk_-5549263231281364844_3453 src: /*.*.*.147:56205 dest: /*.*.*.142:20010 2012-06-01 13:01:01,136 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(*.*.*.142:20010, storageID=DS-1534489105-*.*.*.142-20010-1337757934836, infoPort=20075, ipcPort=20020) Starting thread to transfer block blk_-3849519151985279385_5906 to *.*.*.147:20010 2012-06-01 13:01:19,135 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(*.*.*.142:20010, storageID=DS-1534489105-*.*.*.142-20010-1337757934836, infoPort=20075, ipcPort=20020):Failed to transfer blk_-5797481564121417802_3453 to *.*.*.146:20010 got java.net.ConnectException: > Connection timed out at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:373) at org.apache.hadoop.hdfs.server.datanode.DataNode$DataTransfer.run(DataNode.java:1257) at java.lang.Thread.run(Thread.java:722) 2012-06-01 13:06:20,342 INFO org.apache.hadoop.hdfs.server.datanode.DataBlockScanner: Verification succeeded for blk_6674438989226364081_3453 2012-06-01 13:09:01,781 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(*.*.*.142:20010, storageID=DS-1534489105-*.*.*.142-20010-1337757934836, infoPort=20075, ipcPort=20020):Failed to transfer blk_-3849519151985279385_5906 to *.*.*.147:20010 got java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel to be ready for write. ch : java.nio.channels.SocketChannel[connected local=/*.*.*.142:60057 remote=/*.*.*.147:20010] at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:246) at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:164) at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:203) at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendChunks(BlockSender.java:388) at org.apache.hadoop.hdfs.server.datanode.BlockSender.sendBlock(BlockSender.java:476) at org.apache.hadoop.hdfs.server.datanode.DataNode$DataTransfer.run(DataNode.java:1284) at java.lang.Thread.run(Thread.java:722) hdfs-site.xml <configuration> <property> <name>dfs.name.dir</name> <value>/home/hadoop/data/name</value> </property> <property> <name>dfs.data.dir</name> <value>/home/hadoop/data/hdfs1,/home/hadoop/data/hdfs2,/home/hadoop/data/hdfs3,/home/hadoop/data/hdfs4,/home/hadoop/data/hdfs5</value> </property> <property> <name>dfs.replication</name> <value>3</value> </property> <property> <name>dfs.datanode.max.xcievers</name> <value>4096</value> </property> <property> <name>dfs.http.address</name> <value>0.0.0.0:20070</value> <description>50070 The address and the base port where the dfs namenode web ui will listen on. If the port is 0 then the server will start on a free port. </description> </property> <property> <name>dfs.datanode.http.address</name> <value>0.0.0.0:20075</value> <description>50075 The datanode http server address and port. If the port is 0 then the server will start on a free port. </description> </property> <property> <name>dfs.secondary.http.address</name> <value>0.0.0.0:20090</value> <description>50090 The secondary namenode http server address and port. If the port is 0 then the server will start on a free port. </description> </property> <property> <name>dfs.datanode.address</name> <value>0.0.0.0:20010</value> <description>50010 The address where the datanode server will listen to. If the port is 0 then the server will start on a free port. </description> <property> <name>dfs.datanode.ipc.address</name> <value>0.0.0.0:20020</value> <description>50020 The datanode ipc server address and port. If the port is 0 then the server will start on a free port. </description> </property> <property> <name>dfs.datanode.https.address</name> <value>0.0.0.0:20475</value> </property> <property> <name>dfs.https.address</name> <value>0.0.0.0:20470</value> </property> </configuration> mapred-site.xml <configuration> <property> <name>mapred.job.tracker</name> <value>masternode:29001</value> </property> <property> <name>mapred.system.dir</name> <value>/home/hadoop/data/mapreduce/system</value> </property> <property> <name>mapred.local.dir</name> <value>/home/hadoop/data/mapreduce/local</value> </property> <property> <name>mapred.map.tasks</name> <value>32</value> <description> default number of map tasks per job.</description> </property> <property> <name>mapred.tasktracker.map.tasks.maximum</name> <value>4</value> </property> <property> <name>mapred.reduce.tasks</name> <value>8</value> <description> default number of reduce tasks per job.</description> </property> <property> <name>mapred.map.child.java.opts</name> <value>-Xmx2048M</value> </property> <property> <name>io.sort.mb</name> <value>500</value> </property> <property> <name>mapred.task.timeout</name> <value>1800000</value> <!-- 30 minutes --> </property> <property> <name>mapred.job.tracker.http.address</name> <value>0.0.0.0:20030</value> <description> 50030 The job tracker http server address and port the server will listen on. If the port is 0 then the server will start on a free port. </description> </property> <property> <name>mapred.task.tracker.http.address</name> <value>0.0.0.0:20060</value> <description> 50060 </property> </configuration>

    Read the article

  • DFS function, can you guys tell me what is the wrong with this code?

    - by danielDhobbs
    can you guys tell me what is the wrong with this code? it is not working with 1 2 1 3 1 4 2 5 2 6 2 7 2 8 3 8 3 9 4 10 1 - 4 - 10 and stop DFS function void Is_Connected(graphType* g, int v){ //function to define the graph is connected or not int i=0; g_node* w; top = NULL; g -> visited[v] = TRUE; push(v); printf("%d", v); while (top != NULL) { w = g -> adjList[v]; while (w) { if (!g -> visited[w -> vertex]) { push(w -> vertex); g -> visited[w -> vertex] = TRUE; printf("->%d", w->vertex); v = w -> vertex; w = g -> adjList[v]; } else { w = w -> link; } } i++; v = pop(); } if (i == (g -> x)-1){ //number of vertices = number of vertetices pass through puts("\nIs_Connected() result : yes"); } else{ puts("\nIs_Connected() result : no"); } }

    Read the article

  • Bubble shooter search alghoritm

    - by Fofole
    So I have a Matrix of NxM. At a given position (for ex. [2][5]) I have a value which represents a color. If there is nothing at that point the value is -1. What I need to do is after I add a new point, to check all his neighbours with the same color value and if there are more than 2, set them all to -1. If what I said doesn't make sense what I'm trying to do is an alghoritm which I use to destroy all the same color bubbles from my screen, where the bubbles are memorized in a matrix where -1 means no bubble and {0,1,2,...} represent that there is a bubble with a specific color. This is what I tried and failed: public class Testing { static private int[][] gameMatrix= {{3, 3, 4, 1, 1, 2, 2, 2, 0, 0}, {1, 4, 1, 4, 2, 2, 1, 3, 0, 0}, {2, 2, 4, 4, 3, 1, 2, 4, 0, 0}, {0, 1, 2, 3, 4, 1, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, {0, 0, 0, 0, 0, 0, 0, 0, 0, 0}, }; static int Rows=6; static int Cols=10; static int count; static boolean[][] visited=new boolean[15][15]; static int NOCOLOR = -1; static int color = 1; public static void dfs(int r, int c, int color, boolean set) { for(int dr = -1; dr <= 1; dr++) for(int dc = -1; dc <= 1; dc++) if(!(dr == 0 && dc == 0) && ok(r+dr, c+dc)) { int nr = r+dr; int nc = c+dc; // if it is the same color and we haven't visited this location before if(gameMatrix[nr][nc] == color && !visited[nr][nc]) { visited[nr][nc] = true; count++; dfs(nr, nc, color, set); if(set) { gameMatrix[nr][nc] = NOCOLOR; } } } } static boolean ok(int r, int c) { return r >= 0 && r < Rows && c >= 0 && c < Cols; } static void showMatrix(){ for(int i = 0; i < gameMatrix.length; i++) { System.out.print("["); for(int j = 0; j < gameMatrix[0].length; j++) { System.out.print(" " + gameMatrix[i][j]); } System.out.println(" ]"); } System.out.println(); } static void putValue(int value,int row,int col){ gameMatrix[row][col]=value; } public static void main(String[] args){ System.out.println("Initial Matrix:"); putValue(1, 4, 1); putValue(1, 5, 1); showMatrix(); for(int n = 0; n < 15; n++) for(int m = 0; m < 15; m++) visited[n][m] = false; //reset count count = 0; //dfs(bubbles.get(i).getRow(), bubbles.get(i).getCol(), color, false); // get the contiguous count dfs(5,1,color,false); //if there are more than 2 set the color to NOCOLOR for(int n = 0; n < 15; n++) for(int m = 0; m < 15; m++) visited[n][m] = false; if(count > 2) { //dfs(bubbles.get(i).getRow(), bubbles.get(i).getCol(), color, true); dfs(5,1,color,true); } System.out.println("Matrix after dfs:"); showMatrix(); } }

    Read the article

  • Network access lags for Win7 when server network utilization is high

    - by Jeff Miles
    We have a Dell PE2950 file server running Windows 2008, hosting a DFS namespace of ~1.2 TB. This server has two Broadcom 1Gbps NICs teamed together. When there is high traffic going to the server across the network (greater than 200 Mbps), any Windows 7 client accessing a DFS share at the time experiences severe performance problems. For example: Computer A has an AutoCAD drawing opened directly from the DFS share. Performance is normal, not causing any issues. Computer B begins a file transfer, putting a 11GB file onto a different DFS namespace, on the same server Computer A immediately notices lag while using AutoCAD. The cursor momentarily freezes within AutoCAD every 10 seconds or so, and any browsing of the DFS share is extremely slow. Computer B completes file transfer, and performance resumes to normal for Computer A. This is only affecting Windows 7 clients, using a variety of hardware (desktop + laptop). All of our Windows XP clients see no performance impact during the file transfer. Things I have tried with no change: Had Computer A work from an entirely different RAID array from the file transfer destination Updated NIC drivers on clients and server Enabled TCP offload and receive side scaling on the server NIC (previously disabled when the issue began) Antivirus disabled during file transfer I am currently having a user test applications other than AutoCAD when the file transfer occurs, and will update the question with that result. Does anyone have any recommendations for resolution or additional troubleshooting steps?

    Read the article

  • How exactly are Distributed File Systems used in cloud environment?

    - by vaab
    How exactly are Distributed File Systems used in cloud environment ? More precisely: Are live VMs images (or their filesystem) usually located in the DFS ? Are VMs usually used to run the backbone (actual code) of DFS structure ? Precise example citing DFS (ceph, Gluster, GFS, GPFS, Lustre) or cloud environment (Openstack , CloudStack, ...) would be appreciated, even if I'm more interessted by ceph on OpenStack for now.

    Read the article

  • no namenode error in pseudo-mode

    - by Anshu Basia
    I'm new to hadoop and is in learning phase. As per Hadoop Definitve guide, i have set up my hadoop in pseudo distributed mode and everything was working fine. I was even able to execute all the examples from chapter 3 yesterday. Today, when i rebooted my unix and tried to run start-dfs.sh and then tried http://localhost/50070....it is showing error and when i try to stop dfs (stop-dfs.sh) it says no namenode to stop. I have been googling the issue but no result. Also, when i format my namenode again...everything starts working fine and i'm able to connect to the localhost/50070 and even replicate files and directories in hdfs but as soon as i restart my linux and try to connect to hdfs the same problem comes up. Below is the error log: ************************************************************/ 2011-06-22 15:45:55,249 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = ubuntu/127.0.1.1 STARTUP_MSG: args = [] STARTUP_MSG: version = 0.20.203.0 STARTUP_MSG: build = http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security-203 -r 1099333; compiled by 'oom' on Wed May 4 07:57:50 PDT 2011 ************************************************************/ 2011-06-22 15:45:56,383 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from hadoop-metrics2.properties 2011-06-22 15:45:56,455 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source MetricsSystem,sub=Stats registered. 2011-06-22 15:45:56,494 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot period at 10 second(s). 2011-06-22 15:45:56,494 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: NameNode metrics system started 2011-06-22 15:45:57,007 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi registered. 2011-06-22 15:45:57,031 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already exists! 2011-06-22 15:45:57,059 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source jvm registered. 2011-06-22 15:45:57,070 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source NameNode registered. 2011-06-22 15:45:57,374 INFO org.apache.hadoop.hdfs.util.GSet: VM type = 32-bit 2011-06-22 15:45:57,374 INFO org.apache.hadoop.hdfs.util.GSet: 2% max memory = 19.33375 MB 2011-06-22 15:45:57,374 INFO org.apache.hadoop.hdfs.util.GSet: capacity = 2^22 = 4194304 entries 2011-06-22 15:45:57,374 INFO org.apache.hadoop.hdfs.util.GSet: recommended=4194304, actual=4194304 2011-06-22 15:45:57,854 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: fsOwner=anshu 2011-06-22 15:45:57,854 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: supergroup=supergroup 2011-06-22 15:45:57,854 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isPermissionEnabled=true 2011-06-22 15:45:57,868 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: dfs.block.invalidate.limit=100 2011-06-22 15:45:57,869 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s) 2011-06-22 15:45:58,769 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Registered FSNamesystemStateMBean and NameNodeMXBean 2011-06-22 15:45:58,809 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Caching file names occuring more than 10 times **2011-06-22 15:45:58,825 INFO org.apache.hadoop.hdfs.server.common.Storage: Storage directory /tmp/hadoop-anshu/dfs/name does not exist. 2011-06-22 15:45:58,827 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed. org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-anshu/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible. at org.apache.h**adoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:291) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:97) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:379) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:353) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:254) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:434) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1153) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1162) 2011-06-22 15:45:58,828 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop-anshu/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible. at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:291) at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:97) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:379) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:353) at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:254) at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:434) at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1153) at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1162) 2011-06-22 15:45:58,829 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1 ************************************************************/ Any help is appreciated Thank-you

    Read the article

  • WCF- "The underlying connection was closed: The connection was closed unexpectedly"

    - by SumGuy
    Hi there. I'm recieving that wonderfuly ambiguous error message when using one of my webmethods on my WCF webservice. As that error message doesn't provide any explanation whatsoever allow me to post my theory. I believe it may have something to do with the return type I'm using I have a Types DLL which is refrenced in both the webservice and the client. In this DLL is the base class ExceptionMessages. There is a child of this class called DrawingExcepions. Here is some code: public class ExceptionMessages { public object[] ReturnValue { get; set; } } public class DrawingExceptions : ExceptionMessages { private List<DrawingException> des = new List<DrawingException>(); } public class DrawingException { public Exception ExceptionMsg { get; set; } public List<object> Errors { get; set; } } The using code: [OperationContract] ExceptionMessages createNewBom(Bom bom, DrawingFiles dfs); public ExceptionMessages createNewBOM(Bom bom, DrawingFiles dfs) { return insertAssembly(bom, dfs); } public DrawingExceptions insertAssembly(Bom bom, DrawingFiles dfs) { DrawingExceptions des = new DrawingExceptions(); foreach (DrawingFile d in dfs.drawingFiles) { DrawingException temp = insertNewDrawing(bom, d); if (temp != null) des.addDrawingException(temp); if (d.Child != null) des.addDrawingException(insertAssembly(bom, d.Child)); } return des; } Returns to: ExceptionMessages ems = client.createNewBom(bom, currentDFS); if (ems is DrawingExceptions) { } Basically the return type from the webmethod is ExceptionMessages however I would usually be sending the child class back instead. My only idea is that it's the child that's causing the error but as far as I've read, this should have no effect. Has anyone got any ideas what could be going wrong here? If any more info is required, just ask :) Thanks.

    Read the article

  • Why does Samba/CIFS suck so badly. [closed]

    - by sean
    Seriously, machines refusing to save data because files THEY HAVE OPEN are locked BY THEMSELVES. Getting 200+ connections simultaneously takes it out despite a plethora of available disk and network bandwidth. You can't turn off CUPS you have to COMPILE WITHOUT IT. DFS support is completely broken and pretty much useless in the current state (as in DFS for load balancing, not replication). We should just move to NFS and find a DFS like namespace aggregator.

    Read the article

  • Hive performance increase

    - by Sagar Nikam
    I am dealing with a database (2.5 GB) having some tables only 40 row to some having 9 million rows data. when I am doing any query for large table it takes more time. I want results in less time small query on table which have 90 rows only-- hive> select count(*) from cidade; Time taken: 50.172 seconds hdfs-site.xml <configuration> <property> <name>dfs.replication</name> <value>3</value> <description>Default block replication. The actual number of replications can be specified when the file is created. The default is used if replication is not specified in create time. </description> </property> <property> <name>dfs.block.size</name> <value>131072</value> <description>Default block replication. The actual number of replications can be specified when the file is created. The default is used if replication is not specified in create time. </description> </property> </configuration> does these setting affects performance of hive? dfs.replication=3 dfs.block.size=131072 can i set it from hive prompt as hive>set dfs.replication=5 Is this value remains for a perticular session only ? or Is it better to change it in .xml file ?

    Read the article

  • Gave a talk at SoCal Code Camp at USC today titled “Linq to Objects A-Z”

    - by dotneteer
    I gave a talk at SoCal Code Camp on Linq to Objects. With careful categorization of Linq functions, I was able to cover the entire set of Linq functions in only 35 minutes. I was able to spend the rest time on demos. In my first demo, I show I was able to write a top 20 URL type of query using 4 lines of library code and 9 line of Linq code without tools like Log Parser. I also demonstrated that I only need to change 2 lines of code from querying a single log file to a whole directory of log files. It would be as simple to run the query against multiple servers in parallel. In my second demo, I discussed how to turn into graph depth-first-search (DFS) and breath-first-search (BFS) in the a Linq queryable problem. The class LingToGraph contains the only DFS and BFS code I ever have to write; the rest could be done the the lambda passed to the DFS or BFS calls. In future blogs, I will provide more details explanation of code. Links: Link to Powerpoint slides. Link to demos.

    Read the article

  • How to store visited states in iterative deepening / depth limited search?

    - by colinfang
    Update: Search for the first solution. for a normal Depth First Search it is simple, just use a hashset bool DFS (currentState) = { if (myHashSet.Contains(currentState)) { return; } else { myHashSet.Add(currentState); } if (IsSolution(currentState) return true; else { for (var nextState in GetNextStates(currentState)) if (DFS(nextState)) return true; } return false; } However, when it becomes depth limited, i cannot simply do this bool DFS (currentState, maxDepth) = { if (maxDepth = 0) return false; if (myHashSet.Contains(currentState)) { return; } else { myHashSet.Add(currentState); } if (IsSolution(currentState) return true; else { for (var nextState in GetNextStates(currentState)) if (DFS(nextState, maxDepth - 1)) return true; } return false; } Because then it is not going to do a complete search (in a sense of always be able to find a solution if there is any) before maxdepth How should I fix it? Would it add more space complexity to the algorithm? Or it just doesn't require to memoize the state at all. Update: for example, a decision tree is the following: A - B - C - D - E - A | F - G (Goal) Starting from state A. and G is a goal state. Clearly there is a solution under depth 3. However, using my implementation under depth 4, if the direction of search happens to be A(0) -> B(1) -> C(2) -> D(3) -> E(4) -> F(5) exceeds depth, then it would do back track to A, however E is visited, it would ignore the check direction A - E - F - G

    Read the article

  • Hadoop hdfs namenode is throwing an error

    - by KarmicDice
    Full list of error: hb@localhost:/etc/hadoop/conf$ sudo service hadoop-hdfs-namenode start * Starting Hadoop namenode: starting namenode, logging to /var/log/hadoop-hdfs/hadoop-hdfs-namenode-localhost.out 12/09/10 14:41:09 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = localhost/127.0.0.1 STARTUP_MSG: args = [] STARTUP_MSG: version = 2.0.0-cdh4.0.1 STARTUP_MSG: classpath = /etc/hadoop/conf:/usr/lib/hadoop/lib/xmlenc-0.52.jar:/usr/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop/lib/kfs-0.3.jar:/usr/lib/hadoop/lib/asm-3.2.jar:/usr/lib/hadoop/lib/commons-logging-api-1.1.jar:/usr/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/usr/lib/hadoop/lib/stax-api-1.0.1.jar:/usr/lib/hadoop/lib/commons-configuration-1.6.jar:/usr/lib/hadoop/lib/jets3t-0.6.1.jar:/usr/lib/hadoop/lib/jersey-server-1.8.jar:/usr/lib/hadoop/lib/oro-2.0.8.jar:/usr/lib/hadoop/lib/aspectjrt-1.6.5.jar:/usr/lib/hadoop/lib/json-simple-1.1.jar:/usr/lib/hadoop/lib/snappy-java-1.0.3.2.jar:/usr/lib/hadoop/lib/commons-httpclient-3.1.jar:/usr/lib/hadoop/lib/log4j-1.2.15.jar:/usr/lib/hadoop/lib/servlet-api-2.5.jar:/usr/lib/hadoop/lib/jackson-xc-1.8.8.jar:/usr/lib/hadoop/lib/jersey-json-1.8.jar:/usr/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop/lib/commons-el-1.0.jar:/usr/lib/hadoop/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop/lib/commons-collections-3.2.1.jar:/usr/lib/hadoop/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop/lib/jersey-core-1.8.jar:/usr/lib/hadoop/lib/commons-codec-1.4.jar:/usr/lib/hadoop/lib/jsr305-1.3.9.jar:/usr/lib/hadoop/lib/commons-cli-1.2.jar:/usr/lib/hadoop/lib/activation-1.1.jar:/usr/lib/hadoop/lib/jaxb-impl-2.2.3-1.jar:/usr/lib/hadoop/lib/jetty-util-6.1.26.cloudera.1.jar:/usr/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/usr/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/usr/lib/hadoop/lib/commons-lang-2.5.jar:/usr/lib/hadoop/lib/commons-digester-1.8.jar:/usr/lib/hadoop/lib/commons-io-2.1.jar:/usr/lib/hadoop/lib/jsp-api-2.1.jar:/usr/lib/hadoop/lib/guava-11.0.2.jar:/usr/lib/hadoop/lib/jetty-6.1.26.cloudera.1.jar:/usr/lib/hadoop/lib/jsch-0.1.42.jar:/usr/lib/hadoop/lib/zookeeper-3.4.3-cdh4.0.1.jar:/usr/lib/hadoop/lib/avro-1.5.4.jar:/usr/lib/hadoop/lib/core-3.1.1.jar:/usr/lib/hadoop/lib/paranamer-2.3.jar:/usr/lib/hadoop/lib/jettison-1.1.jar:/usr/lib/hadoop/lib/jackson-jaxrs-1.8.8.jar:/usr/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/usr/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/usr/lib/hadoop/lib/commons-net-3.1.jar:/usr/lib/hadoop/lib/jaxb-api-2.2.2.jar:/usr/lib/hadoop/lib/commons-math-2.1.jar:/usr/lib/hadoop/lib/jline-0.9.94.jar:/usr/lib/hadoop/.//hadoop-annotations.jar:/usr/lib/hadoop/.//hadoop-annotations-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop/.//hadoop-common-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop/.//hadoop-auth-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop/.//hadoop-common.jar:/usr/lib/hadoop/.//hadoop-auth.jar:/usr/lib/hadoop/.//hadoop-common-2.0.0-cdh4.0.1-tests.jar:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-hdfs/lib/snappy-java-1.0.3.2.jar:/usr/lib/hadoop-hdfs/lib/log4j-1.2.15.jar:/usr/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/slf4j-api-1.6.1.jar:/usr/lib/hadoop-hdfs/lib/commons-logging-1.1.1.jar:/usr/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-hdfs/lib/commons-daemon-1.0.3.jar:/usr/lib/hadoop-hdfs/lib/zookeeper-3.4.3-cdh4.0.1.jar:/usr/lib/hadoop-hdfs/lib/avro-1.5.4.jar:/usr/lib/hadoop-hdfs/lib/paranamer-2.3.jar:/usr/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-2.0.0-cdh4.0.1-tests.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop-hdfs/.//hadoop-hdfs.jar:/usr/lib/hadoop-yarn/lib/protobuf-java-2.4.0a.jar:/usr/lib/hadoop-yarn/lib/asm-3.2.jar:/usr/lib/hadoop-yarn/lib/netty-3.2.3.Final.jar:/usr/lib/hadoop-yarn/lib/javax.inject-1.jar:/usr/lib/hadoop-yarn/lib/jersey-server-1.8.jar:/usr/lib/hadoop-yarn/lib/jersey-guice-1.8.jar:/usr/lib/hadoop-yarn/lib/snappy-java-1.0.3.2.jar:/usr/lib/hadoop-yarn/lib/log4j-1.2.15.jar:/usr/lib/hadoop-yarn/lib/guice-3.0.jar:/usr/lib/hadoop-yarn/lib/jackson-mapper-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/junit-4.8.2.jar:/usr/lib/hadoop-yarn/lib/jackson-core-asl-1.8.8.jar:/usr/lib/hadoop-yarn/lib/jersey-core-1.8.jar:/usr/lib/hadoop-yarn/lib/jdiff-1.0.9.jar:/usr/lib/hadoop-yarn/lib/guice-servlet-3.0.jar:/usr/lib/hadoop-yarn/lib/aopalliance-1.0.jar:/usr/lib/hadoop-yarn/lib/commons-io-2.1.jar:/usr/lib/hadoop-yarn/lib/avro-1.5.4.jar:/usr/lib/hadoop-yarn/lib/paranamer-2.3.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-web-proxy-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-resourcemanager.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-common-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-server-nodemanager-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-site.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-api-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-common-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-applications-distributedshell.jar:/usr/lib/hadoop-yarn/.//hadoop-yarn-site-2.0.0-cdh4.0.1.jar:/usr/lib/hadoop-mapreduce/.//* STARTUP_MSG: build = file:///var/lib/jenkins/workspace/generic-package-ubuntu64-12-04/CDH4.0.1-Packaging-Hadoop-2012-06-28_17-01-57/hadoop-2.0.0+91-1.cdh4.0.1.p0.1~precise/src/hadoop-common-project/hadoop-common -r 4d98eb718ec0cce78a00f292928c5ab6e1b84695; compiled by 'jenkins' on Thu Jun 28 17:39:19 PDT 2012 ************************************************************/ 12/09/10 14:41:10 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties hdfs-site.xml: hb@localhost:/etc/hadoop/conf$ cat hdfs-site.xml <?xml version="1.0" encoding="UTF-8"?> <!--Autogenerated by Cloudera CM on 2012-09-03T10:13:30.628Z--> <configuration> <property> <name>dfs.https.address</name> <value>localhost:50470</value> </property> <property> <name>dfs.https.port</name> <value>50470</value> </property> <property> <name>dfs.namenode.http-address</name> <value>localhost:50070</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.blocksize</name> <value>134217728</value> </property> <property> <name>dfs.client.use.datanode.hostname</name> <value>false</value> </property> </configuration>

    Read the article

  • .NET Performance: Deep Recursion vs Queue

    - by JeffN825
    I'm writing a component that needs to walk large object graphs, sometimes 20-30 levels deep. What is the most performant way of walking the graph? A. Enqueueing "steps" so as to avoid deep recursion or B. A DFS (depth first search) which may step many levels deep and have a "deep" stack trace at times. I guess the question I'm asking is: Is there a performance hit in .NET for doing a DFS that causes a "deep" stack trace? If so, what is the hit? And would I better better off with some BFS by means of queueing up steps that would have been handled recursively in a DFS? Sorry if I'm being unclear. Thanks.

    Read the article

  • minimax depth first search game tree

    - by Arvind
    Hi I want to build a game tree for nine men's morris game. I want to apply minimax algorithm on the tree for doing node evaluations. Minimax uses DFS to evaluate nodes. So should I build the tree first upto a given depth and then apply minimax or can the process of building the tree and evaluation occur together in recursive minimax DFS? Thank you Arvind

    Read the article

  • Windows 2008 R2 SMB / CIFS Logging to diagnose Brother MFC Network Scanning

    - by Steven Potter
    I am attempting to setup network scanning on a brother MFC-9970CDW printer. According to the Brother documentation, the printer is setup to connect to any CIFS network share. I applied all of the appropriate setting in the printer however I get a "sending error" when I try to scan a document. When I look at the logs of the 2008 R2 server that I am attempting to connect to; I can see in the security log where the printer successfully authenticates, however nothing else is logged. I would assume that immediately after the authentication, the printer is making a CIFS request and some sort of error is occurring, however I can't seem to find any way to log this information to find out what is going on. Is it possible to get Windows 2008 to log SMB/CIFS traffic? Followup: I installed Microsoft netmon and captured the packets associated with the transaction: 510 3:04:28 PM 7/9/2012 34.4277743 System 192.168.1.134 192.168.1.10 SMB SMB:C; Negotiate, Dialect = NT LM 0.12 {SMBOverTCP:30, TCP:29, IPv4:22} 511 3:04:28 PM 7/9/2012 34.4281246 System 192.168.1.10 192.168.1.134 SMB SMB:R; Negotiate, Dialect is NT LM 0.12 (#0), SpnegoToken (1.3.6.1.5.5.2) {SMBOverTCP:30, TCP:29, IPv4:22} 519 3:04:29 PM 7/9/2012 34.8986214 System 192.168.1.134 192.168.1.10 SMB SMB:C; Session Setup Andx, NTLM NEGOTIATE MESSAGE {SMBOverTCP:30, TCP:29, IPv4:22} 520 3:04:29 PM 7/9/2012 34.8989310 System 192.168.1.10 192.168.1.134 SMB SMB:R; Session Setup Andx, NTLM CHALLENGE MESSAGE - NT Status: System - Error, Code = (22) STATUS_MORE_PROCESSING_REQUIRED {SMBOverTCP:30, TCP:29, IPv4:22} 522 3:04:29 PM 7/9/2012 34.9022870 System 192.168.1.134 192.168.1.10 SMB SMB:C; Session Setup Andx, NTLM AUTHENTICATE MESSAGEVersion:v2, Domain: CORP, User: PRINTSUPOFF, Workstation: BRN001BA9AD1FE6 {SMBOverTCP:30, TCP:29, IPv4:22} 523 3:04:29 PM 7/9/2012 34.9032421 System 192.168.1.10 192.168.1.134 SMB SMB:R; Session Setup Andx {SMBOverTCP:30, TCP:29, IPv4:22} 525 3:04:29 PM 7/9/2012 34.9051855 System 192.168.1.134 192.168.1.10 SMB SMB:C; Tree Connect Andx, Path = \\192.168.1.10\IPC$, Service = ????? {SMBOverTCP:30, TCP:29, IPv4:22} 526 3:04:29 PM 7/9/2012 34.9053083 System 192.168.1.10 192.168.1.134 SMB SMB:R; Tree Connect Andx, Service = IPC {SMBOverTCP:30, TCP:29, IPv4:22} 528 3:04:29 PM 7/9/2012 34.9073573 System 192.168.1.134 192.168.1.10 DFSC DFSC:Get DFS Referral Request, FileName: \\192.168.1.10\NSCFILES, MaxReferralLevel: 3 {SMB:33, SMBOverTCP:30, TCP:29, IPv4:22} 529 3:04:29 PM 7/9/2012 34.9152042 System 192.168.1.10 192.168.1.134 SMB SMB:R; Transact2, Get Dfs Referral - NT Status: System - Error, Code = (549) STATUS_NOT_FOUND {SMB:33, SMBOverTCP:30, TCP:29, IPv4:22} 531 3:04:29 PM 7/9/2012 34.9169738 System 192.168.1.134 192.168.1.10 SMB SMB:C; Tree Disconnect {SMBOverTCP:30, TCP:29, IPv4:22} 532 3:04:29 PM 7/9/2012 34.9170688 System 192.168.1.10 192.168.1.134 SMB SMB:R; Tree Disconnect {SMBOverTCP:30, TCP:29, IPv4:22} As you can see, the DFS referral fails and the transaction is shut down. I can't see any reason for the DFS referral to fail. The only reference I can find online is: https://bugzilla.samba.org/show_bug.cgi?id=8003 Anyone have any ideas for a solution?

    Read the article

  • Hopcroft–Karp algorithm in Python

    - by Simon
    I am trying to implement the Hopcroft Karp algorithm in Python using networkx as graph representation. Currently I am as far as this: #Algorithms for bipartite graphs import networkx as nx import collections class HopcroftKarp(object): INFINITY = -1 def __init__(self, G): self.G = G def match(self): self.N1, self.N2 = self.partition() self.pair = {} self.dist = {} self.q = collections.deque() #init for v in self.G: self.pair[v] = None self.dist[v] = HopcroftKarp.INFINITY matching = 0 while self.bfs(): for v in self.N1: if self.pair[v] and self.dfs(v): matching = matching + 1 return matching def dfs(self, v): if v != None: for u in self.G.neighbors_iter(v): if self.dist[ self.pair[u] ] == self.dist[v] + 1 and self.dfs(self.pair[u]): self.pair[u] = v self.pair[v] = u return True self.dist[v] = HopcroftKarp.INFINITY return False return True def bfs(self): for v in self.N1: if self.pair[v] == None: self.dist[v] = 0 self.q.append(v) else: self.dist[v] = HopcroftKarp.INFINITY self.dist[None] = HopcroftKarp.INFINITY while len(self.q) > 0: v = self.q.pop() if v != None: for u in self.G.neighbors_iter(v): if self.dist[ self.pair[u] ] == HopcroftKarp.INFINITY: self.dist[ self.pair[u] ] = self.dist[v] + 1 self.q.append(self.pair[u]) return self.dist[None] != HopcroftKarp.INFINITY def partition(self): return nx.bipartite_sets(self.G) The algorithm is taken from http://en.wikipedia.org/wiki/Hopcroft%E2%80%93Karp_algorithm However it does not work. I use the following test code G = nx.Graph([ (1,"a"), (1,"c"), (2,"a"), (2,"b"), (3,"a"), (3,"c"), (4,"d"), (4,"e"),(4,"f"),(4,"g"), (5,"b"), (5,"c"), (6,"c"), (6,"d") ]) matching = HopcroftKarp(G).match() print matching Unfortunately this does not work, I end up in an endless loop :(. Can someone spot the error, I am out of ideas and I must admit that I have not yet fully understand the algorithm, so it is mostly an implementation of the pseudo code on wikipedia

    Read the article

  • Join Production Server 2008 to 2003 domain

    - by Campo
    I administer a production server for a .com. It is live right now. Server 2008 x64 IIS 7 SQL 2008 PHP MYSQL I have another server which is a DC Server 2003 x86 and a warm standby for the website, sql, DFS, exchange queue. In order to get DFS going to transfer user photos and other content I need it in the domain. My question is, What preparations do I need to do to the production server to allow a smooth transition onto the domain? Things such as permissions for the website. I do not want to be running around resetting all the permissions. The Group Policy on the DC is completely default. Should I add the DNS manually or allow it to add itself? Anything else I left out.

    Read the article

  • Windows AD, bulk user creation, homedrv creation via commandline

    - by Neil
    I am Bulk creating AD users from the commandline (dsadd) and whilst doing so am setting the homedir and homedrv to a DFS location. I observe when I create the user with all these settings via the GUI (dsa.msc) that the homedrv gets created on the DFS share with all the permissions set correctly. But when using dsadd, the folder is not created. How can I replicate this GUI behaviour via the commandline when creating the user? I don't really want to rely on logon scripts to set it up. Do I have to use mkdir and cacls and something else to give the user Ownership? Or maybe I am missing something easy. Any help much appreciated!

    Read the article

  • How best to set up MDT developement and production?

    - by nray
    What's your MDT 2010 test and prod setup? What do you consider best practice? Is it best to use linked deployment shares, and replicate from development to production when testing is complete? What about backing out, if something breaks? Does anyone run MDT shares in DFS, or is there no support in the WinPE boot image for DFS shares? Or what about moving the production share name from one deployment share to another, as you add and test more OS versions, drivers, attributes, etc?

    Read the article

  • `wget` is not recognized or either can't find the file

    - by clankill3r
    if i use cd C:\Program Files (x86)\GnuWin32\bin then i'm able to use wget commands for example: wget http://www.ultralightnews.com/trikes/images/trikes/dfs-singletrike.jpg but i can't find the file back, i looked in c:\ and in the bin folder mentioned above and in GnuWin32\etc If i try wget -O C:\Users\clankill3r\Downloads\wgetfolder wget http://www.ultralightnews.com/trikes/images/trikes/dfs-singletrike.jpg then it says Permision denied, i did allow all permisions possible for every group / user. Some people say it downloads to the current folder your working in (that's why i looked in the bin). But i thought let's try to run the command from another folder so i used: 'cd C:\Users\clankill3r\Downloads\wgetfolderand then the wget comman but then it says thewget` command is not recognized. can someone help?

    Read the article

  • Keeping track of threads when creating them recursively

    - by 66replica
    I'm currently working on some code for my Programming Languages course. I can't post the code but I'm permitted to talk about some high level concepts that I'm struggling with and receive input on them. Basically the code is a recursive DFS on a undirected graph that I'm supposed to convert to a concurrent program. My professor already specified that I should create my threads in the recursive DFS method and then join them in another method. Basically, I'm having trouble thinking of how I should keep track of the threads I'm creating so I can join all of them in the other method. I'm thinking an array of Threads but I'm unsure how to add each new thread to the array or even if that's the right direction.

    Read the article

facebook twitter digg google delicious technorati stumbleupon myspace wordpress linkedin gmail igoogle windows live tumblr viadeo yahoo buzz yahoo mail yahoo bookmarks favorites email print

< Previous Page | 1 2 3 4 5 6 7 8 9  | Next Page >