Search Results

Search found 2515 results on 101 pages for 'distributed filesystems'.

Page 90/101 | < Previous Page | 86 87 88 89 90 91 92 93 94 95 96 97  | Next Page >

• Java: fastest way to do random reads on huge disk file(s)

  - by cocotwo

  I've got a moderately big set of data, about 800 MB or so, that is basically some big precomputed table that I need to speed some computation by several orders of magnitude (creating that file took several mutlicores computers days to produce using an optimized and multi-threaded algo... I do really need that file). Now that it has been computed once, that 800MB of data is read only. I cannot hold it in memory. As of now it is one big huge 800MB file but splitting in into smaller files ain't a problem if it can help. I need to read about 32 bits of data here and there in that file a lot of time. I don't know before hand where I'll need to read these data: the reads are uniformly distributed. What would be the fastest way in Java to do my random reads in such a file or files? Ideally I should be doing these reads from several unrelated threads (but I could queue the reads in a single thread if needed). Is Java NIO the way to go? I'm not familiar with 'memory mapped file': I think I don't want to map the 800 MB in memory. All I want is the fastest random reads I can get to access these 800MB of disk-based data. btw in case people wonder this is not at all the same as the question I asked not long ago: http://stackoverflow.com/questions/2346722/java-fast-disk-based-hash-set

  Read the article

• Mutually beneficial IP/copyright clauses for contract-based freelance work

  - by Nathan de Vries

  I have a copyright section in the contract I give to my clients stating that I retain copyright on any works produced during my work for them as an independent contractor. This is most definitely not intended to place arbitrary restrictions on my clients, but rather to maintain my ability to decide on how the software I create is licensed and distributed. Almost every project I work on results in at least one part of it being released as open source. Every project I work on makes use of third-party software released in the same fashion, so returning the favour is something I would like to continue doing. Unfortunately, the contract is not so clear when it comes to defining the rights of the client in the use of said software. I mention that the code will be licensed to them, but do not mention specifics about exclusivity, ability to produce derivatives etc. As such, a client has raised concerns about the copyright section of my contract, and has suggested that I reword it such that all copyrights are transferred entirely to the client on final payment for the project. This will almost certainly reduce my ability to distribute the software I have created; I would much prefer to find a more mutually beneficial agreement where both our concerns are appeased. Are there any tried and true approaches to licensing software in this kind of situation? To summarise: I want to maintain the ability to license (parts of) the software under my own terms, independently of my relationship with the client; with some guarantee to the client that no trade-secrets or critical business logic will be shared; giving them the ability to re-use my code in their future projects; but not necessarily letting them sell it (I'm not sure about this, though...what happens if they sell their business and the software along with it?) I realise that everyone's feedback is going to be prefixed with "IANAL", however I appreciate any thoughts you might have on the matter.

  Read the article

• The way cores, processes, and threads work exactly?

  - by unknownthreat

  I need a bit of an advice for understanding how this whole procedure work exactly. If I am incorrect in any part described below, please correct me. In a single core CPU, it runs each process in the OS, jumping around from one process to another to utilize the best of itself. A process can also have many threads, in which the CPU core runs through these threads when it is running on the respective process. Now, on a multiple core CPU, Do the cores run in every process together, or can the cores run separately in different processes at one particular point of time? For instance, you have program A running two threads, can a duo core CPU run both threads of this program? I think the answer should be yes if we are using something like OpenMP. But while the cores are running in this OpenMP-embedded process, can one of the core simply switch to other process? For programs that are created for single core, when running at 100%, why the CPU utilization of each core are distributed? (ex. A duo core CPU of 80% and 20%. The utilization percentage of all cores always add up to 100% for this case.) Do the cores try help each other run each thread of each process in some ways? Frankly, I'm not sure how this works exactly. Any advice is appreciated.

  Read the article

• What about parallelism across network using multiple PCs?

  - by MainMa

  Parallel computing is used more and more, and new framework features and shortcuts make it easier to use (for example Parallel extensions which are directly available in .NET 4). Now what about the parallelism across network? I mean, an abstraction of everything related to communications, creation of processes on remote machines, etc. Something like, in C#: NetworkParallel.ForEach(myEnumerable, () => { // Computing and/or access to web ressource or local network database here }); I understand that it is very different from the multi-core parallelism. The two most obvious differences would probably be: The fact that such parallel task will be limited to computing, without being able for example to use files stored locally (but why not a database?), or even to use local variables, because it would be rather two distinct applications than two threads of the same application, The very specific implementation, requiring not just a separate thread (which is quite easy), but spanning a process on different machines, then communicating with them over local network. Despite those differences, such parallelism is quite possible, even without speaking about distributed architecture. Do you think it will be implemented in a few years? Do you agree that it enables developers to easily develop extremely powerfull stuff with much less pain? Example: Think about a business application which extracts data from the database, transforms it, and displays statistics. Let's say this application takes ten seconds to load data, twenty seconds to transform data and ten seconds to build charts on a single machine in a company, using all the CPU, whereas ten other machines are used at 5% of CPU most of the time. In a such case, every action may be done in parallel, resulting in probably six to ten seconds for overall process instead of forty.

  Read the article

• asp.net, wcf authentication and caching

  - by andrew

  I need to place my app business logic into a WCF service. The service shouldn't be dependent on ASP.NET and there is a lot of data regarding the authenticated user which is frequently used in the business logic hence it's supposed to be cached (probably using a distributed cache). As for authentication - I'm going to use two level authentication: Front-End - forms authentication back-end (WCF Service) - message username authentication. For both authentications the same custom membership provider is supposed to be used. To cache the authenticated user data, I'm going to implement two service methods: 1) Authenticate - will retrieve the needed data and place it into the cache(where username will be used as a key) 2) SignOut - will remove the data from the cache Question 1. Is correct to perform authentication that way (in two places) ? Question 2. Is this caching strategy worth using or should I look at using aspnet compatible service and asp.net session ? Maybe, these questions are too general. But, anyway I'd like to get any suggestions or recommendations. Any Idea

  Read the article

• A generic C++ library that provides QtConcurrent functionality?

  - by Lucas

  QtConcurrent is awesome. I'll let the Qt docs speak for themselves: QtConcurrent includes functional programming style APIs for parallel list processing, including a MapReduce and FilterReduce implementation for shared-memory (non-distributed) systems, and classes for managing asynchronous computations in GUI applications. For instance, you give QtConcurrent::map() an iterable sequence and a function that accepts items of the type stored in the sequence, and that function is applied to all the items in the collection. This is done in a multi-threaded manner, with a thread pool equal to the number of logical CPU's on the system. There are plenty of other function in QtConcurrent, like filter(), filteredReduced() etc. The standard CompSci map/reduce functions and the like. I'm totally in love with this, but I'm starting work on an OSS project that will not be using the Qt framework. It's a library, and I don't want to force others to depend on such a large framework like Qt. I'm trying to keep external dependencies to a minimum (it's the decent thing to do). I'm looking for a generic C++ framework that provides me with the same/similar high-level primitives that QtConcurrent does. AFAIK boost has nothing like this (I may be wrong though). boost::thread is very low-level compared to what I'm looking for. I know C# has something very similar with their Parallel Extensions so I know this isn't a Qt-only idea. What do you suggest I use?

  Read the article

• Why is there so much XML in Java these days?

  - by BD at Rivenhill

  This is really more of a philosophy/design issue. I did some work in Java back in the middle 90's and again in the early 2000's and now I'm coming back to it after spending a lot of time in C/C++ and it seems like there was an explosion of XML dependency while I was gone. Major build system tools like ant and maven depend on XML for their configuration, but I'm actually more concerned with all the frameworks, such as Spring, Hibernate, etc. My experience is that powerful supporting libraries like these are where a developer can really get leverage for building programs with lots of features without writing a lot of code, but it really seems like I'm getting one language for the price of two here. I write a bunch of Java classes, but then I also write a bunch of XML files to glue them together. The things that get done in the XML are things that I can see reasonable ways of doing in straight code without the middleman, and they don't really seem to be treated exactly like configuration files: they change rarely and they end up getting committed to source code control like the Java code itself, but they are distributed with the resulting application and need to be unpacked and installed in the classpath in order to get the application to work. I'm working with server applications that are not web-based, so maybe the domain is a bit different from what most people are doing, but I just can't help feeling that I must be doing something wrong here. Can someone point me to a good source of information for why these design choices were made and what problems they are meant to solve so that I can analyze my own experiences in this context?

  Read the article

• How do software projects go over budget and under-deliver?

  - by Carlos

  I've come across this story quite a few times here in the UK: NHS Computer System Summary: We're spunking £12 Billion on some health software with barely anything working. I was sitting the office discussing this with my colleagues, and we had a little think about. From what I can see, all the NHS needs is a database + middle tier of drugs/hospitals/patients/prescriptions objects, and various GUIs for doctors and nurses to look at. You'd also need to think about security and scalability. And you'd need to sit around a hospital/pharmacy/GPs office for a bit to figure out what they need. But, all told, I'd say I could knock together something with that kind of structure in a couple of days, and maybe throw in a month or two to make it work in scale. If I had a few million quid, I could probably hire some really excellent designers to make a maintainable codebase, and also buy appropriate hardware to run the system on. I hate to trivialize something that seems to have caused to much trouble, but to me it looks like just a big distributed CRUD + UI system. So how on earth did this project bloat to £12B without producing much useful software? As I don't think the software sounds so complicated, I can only imagine that something about how it was organised caused this mess. Is it outsourcing that's the problem? Is it not getting the software designers to understand the medical business that caused it? What are your experiences with projects gone over budget, under delivered? What are best practices for large projects? Have you ever worked on such a project?

  Read the article

• Are there any changes in the licensing of Visual Studio 2013 Express editions?

  - by Ramón García-Pérez

  As was going through reading the license.htm file provided as part of the VS2013_RTM_WebExp_ENU.iso offline installation media for the Visual Studio 2013 Express for Web, section 6 reads as follows: 6. PACKAGE MANAGER AND THIRD PARTY SOFTWARE INSTALLATION FEATURES. The software includes the following features (each a “Feature”), each of which enables you to obtain software applications or packages through the Internet from other sources: Extension Manager, New Project Dialog, Web Platform Installer, and Microsoft NuGet-Based Package Manager. Those software applications and packages are offered and distributed in some cases by third parties and in some cases by Microsoft, but each such application or package is under its own license terms. Microsoft is not developing, distributing or licensing any of the third-party applications or packages to you, but instead, as a convenience, enables you to use the Features to access or obtain those applications or packages directly from the third-party application or package providers. By using the Features, you acknowledge and agree that: you are obtaining the applications or packages from such third parties and under separate license terms applicable to each application or package (including, with respect to the package-manager Features, any terms applicable to software dependencies that may be included in the package); MICROSOFT MAKES NO REPRESENTATIONS, WARRANTIES OR GUARANTEES AS TO THE FEED OR GALLERY URL, ANY FEEDS OR GALLERIES FROM SUCH URL, THE INFORMATION CONTAINED THEREIN, OR ANY SOFTWARE APPLICATIONS OR PACKAGES REFERENCED IN OR ACCESSED BY YOU THROUGH SUCH FEEDS OR GALLERIES. MICROSOFT GRANTS YOU NO LICENSE RIGHTS FOR THIRD-PARTY SOFTWARE APPLICATIONS OR PACKAGES THAT ARE OBTAINED USING THE FEATURES. Are there any changes in the licensing of Visual Studio 2013 Express editions? If so, does this means that Visual Studio extensions installation in Express Editions is now allowed? PS: Previous versions of the Express editions did not allow the installation of extensions as per "EULA/TOS" discussed here: Limitations of Visual Studio 2012 Express Desktop

  Read the article

• Fastest reliable way for Clojure (Java) and Ruby apps to communicate

  - by jkndrkn

  Hi There, We have cloud-hosted (RackSpace cloud) Ruby and Java apps that will interact as follows: Ruby app sends a request to Java app. Request consists of map structure containing strings, integers, other maps, and lists (analogous to JSON). Java app analyzes data and sends reply to Ruby App. We are interested in evaluating both messaging formats (JSON, Buffer Protocols, Thrift, etc.) as well as message transmission channels/techniques (sockets, message queues, RPC, REST, SOAP, etc.) Our criteria: Short round-trip time. Low round-trip-time standard deviation. (We understand that garbage collection pauses and network usage spikes can affect this value). High availability. Scalability (we may want to have multiple instances of Ruby and Java app exchanging point-to-point messages in the future). Ease of debugging and profiling. Good documentation and community support. Bonus points for Clojure support. What combination of message format and transmission method would you recommend? Why? I've gathered here some materials we have already collected for review: Comparison of various java serialization options Comparison of Thrift and Protocol Buffers (old) Comparison of various data interchange formats Comparison of Thrift and Protocol Buffers Fallacies of Protocol Buffers RPC features Discussion of RPC in the context of AMQP (Message-Queueing) Comparison of RPC and message-passing in distributed systems (pdf) Criticism of RPC from perspective of message-passing fan Overview of Avro from Ruby programmer perspective

  Read the article

• Web services or shared database for (game) server communication?

  - by jaaronfarr

  We have 2 server clusters: the first is made up of typical web applications backed by SQL databases. The second are highly optimized multiplayer game servers which keep all data in memory. Both clusters communicate with clients via HTTP (Ajax with JSON). There are a few cases in which we need to share data between the two server types, for example, reporting back and storing the results of a game (should ultimately end up in the database). We're considering several approaches for inter-server communication: Just share the MySQL databases between clusters (introduce SQL to the game servers) Sharing data in a distributed key-value store like Memcache, Redis, etc. Use an RPC technology like Google ProtoBufs or Apache Thrift Using RESTful web services (the game server would POST back to the web servers, for example) At the moment, we're leaning towards web services or just sharing the database. Sharing the database seems easy, but we're concerned this adds extra memory and a new dependency into the game servers. Web services provide good separation of concerns and fit with the existing Ajax we use, but add complexity, overhead and many more ways for communication to fail. Are there any other good reasons not to use one or the other approach? Which would be easier to scale?

  Read the article

• In maven2, how do I assemble bits and pieces of different modules to create final distributions?

  - by Carcassi

  I have four maven project: client api jar web service war ui jar web interface war The service war will need to be packaged to include the client api jar, together with javadocs (so that each version is distributed with a matching client and documentation). The web interface war will need the ui jar and all the dependencies (webstart/applet deployment). So I need a 5th project that does all the packaging. How to do this with ant or a script is perfectly clear to me, but not in maven. I tried the following: having the javadocs included as part of the war packaging: this requires the execution of the javadocs goal in project 1 before execution of package in project 2. Haven't found a way to bind plugins/goals across different projects. Using the assembly plugin in project2 had the same problem. create a fifth project and use the assembly plugin. Still the same problems as before, with the problem that since I need different pieces from each sub-project I do not understand how this can be done using the assembly. Is this too hard to do in maven, and should I just give up? Or I am looking at it wrong, in which case, how should I be looking at it? Thanks! Upon further reflection, here is a partial answer: Each project should build all its artifacts. This is done by having the plugins configured to run as per the prepare-resources and package phases. So, in my case, I prepare all that needs to be generated (jar, javadocs, xsd documentation, ...) as different artifacts so that a single "package" goal execution creates all. So, it's not "how project 2 forces project 1 to run different goals", but it's "make project 1 create all of its artifact as part as the normal lifecycle). This seems to simplify things.

  Read the article

• Hadoop File Read

  - by user3684584

  Hadoop Distributed Cache Wordcount example in hadoop 2.2.0. Copied file into hdfs filesystem to be used inside setup of mapper class. protected void setup(Context context) throws IOException,InterruptedException { Path[] uris = DistributedCache.getLocalCacheFiles(context.getConfiguration()); cacheData=new HashMap(); for(Path urifile: uris) { try { BufferedReader readBuffer1 = new BufferedReader(new FileReader(urifile.toString())); String line; while ((line=readBuffer1.readLine())!=null) { System.out.println("**************"+line); cacheData.put(line,line); } readBuffer1.close(); } catch (Exception e) { System.out.println(e.toString()); } } } Inside Driver Main class Configuration conf = new Configuration(); String[] otherArgs = new GenericOptionsParser(conf,args).getRemainingArgs(); if (otherArgs.length != 3) { System.err.println("Usage: wordcount <in> <out>"); System.exit(2); } Job job = new Job(conf, "word_count"); job.setJarByClass(WordCount.class); job.setMapperClass(Map.class); job.setReducerClass(Reduce.class); job.setOutputKeyClass(Text.class); job.setOutputValueClass(IntWritable.class); FileInputFormat.addInputPath(job, new Path(otherArgs[0])); Path outputpath=new Path(otherArgs[1]); outputpath.getFileSystem(conf).delete(outputpath,true); FileOutputFormat.setOutputPath(job,outputpath); System.out.println("CachePath****************"+otherArgs[2]); DistributedCache.addCacheFile(new URI(otherArgs[2]),job.getConfiguration()); System.exit(job.waitForCompletion(true) ? 0 : 1); But getting exception java.io.FileNotFoundException: file:/home/user12/tmp/mapred/local/1408960542382/cache (No such file or directory) So Cache functionality not working properly. Any Idea ?

  Read the article

• Behavior of Struts2 and convention-plugin when there is Index(extends ActionSupport)

  - by hanishi

  We have an Action class named 'Index' immediately under com.example.common.action and is annotated @ParentPackage('default') which is declared in package directive in struts.xml and has "/" for its namespace and extends "struts-default". It also declares @Result so that it responses with jsp files corresponding the string values returned by its execute() method. In our struts.xml, the following struts setting is configured along with other necessary configurations that are needed for convention-plugin. <constant name="struts.action.extension" value=","/> When accessing /my_context/none_existing_path, the request apparently hits this Index class and the contents of the jsp declared in the Index's @Result section gets returned. However, if we provide /my_context/, we receive the following error: HTTP Status 404-There is no Action mapped for namespace[/] and action name [] associated with context path [/my_context]. We want to know the reason why accessing /my_context/none_existing_path, where none_existing_path has no matching action, can fallback to Index class, but error is returned when when the URL requested is just /my_context/. Currently, our convention-plugin settings are declared as follows: <constant name="struts.convention.package.locators.basePackage" value="com.example"/> <constant name="struts.convention.package.locators" value="action"/> Strangely, if we changed the value of the struts.convention.package.locators.basePackage to om.example.common, in which the aforementioned Index file can be immediately found by narrowing the search scope, requesting /my_context/ displays the content of the jsps declared in @Result section of the Index class. However, as our action classes are distributed throughout the com.example.[a-z].action packages, where [a-z] represents the large volume of directories we have in our package structure, we cannot use this trick as a workaround. We have also tried placing index.jsp at the top level of the class path, and have the index.jsp redirect to /my_context/index, which worked but not what we want. Could this be a bug? We appreciate your responses. Thank you in advance. EDIT: JIRA registered, problem solved (from Struts 2.3.12 up)

  Read the article

• when to use Hibernate vs. Simple ResultSets for small application

  - by luke

  I just started working on upgrading a small component in a distributed java application. The main application is a rather complicated applet/servlet combo running on JBoss and it extensively uses Hibernate for its DataAccess. The component i am working on however is very a very straightforward data importing service. Basically the workflow is Listen for a network event Parse the data packet, extract a set of identifiers Map the identifier set to a primary key in our database Parse the rest of the packet and insert items in a related table using the foreign key found in step 3 Repeat in the previous version of this component it used a hibernate based DAL, that is no longer usable for a variety of reasons (in particular it is EOL), so I am in charge of replacing the Data Access layer for this component. So on the one hand I think i should use Hibernate because that's what the rest of the application does, but on the other i think i should just use regular java.sql.* classes because my requirements are really straightforward and aren't expected to change any time soon. So my question is (and i understand it is subjective) at what point do you think that the added complexity of using an ORM tool (in terms of configuration, dependencies...) is worth it? UPDATE due to the way the DataAccesLayer for the main application was written (weird dependencies) i cannot easily use it, i would have to implement it myself.

  Read the article

• Creating multiple instances of a generic database

  - by sagekilla

  Hi all, currently I'm trying to have a setup where a generic database is distributed to students. They would develop an application using this database (Say a shopping cart application), submit their project onto our server, and then it would be graded automatically. These databases are being run in Microsoft SQL Server 2005. We're using user instances to instantiate each database, and multiple requests could be serviced at once. But, the problem is when more than one student submitted a project to be graded, the first database to be instantiated would be the only one and would overwrite all other copies that were currently open. So if stu1 modified his database and stu2 and stu3 had their projects being graded concurrently, at the end of the grading stu1, stu2, and stu3 would have identical DB's at the end. Is there any way I can have multiple independent copies of a generic database, each of which I can load concurrently and modify without having any changes made to any one affecting the others? I did a little reading, and thought it might be possible to do something along the lines of: Student submits project Attach the database with unique db name (specified by student) Do all necessary operations Detach the database I'm unsure if this would fix our problem or be possible, so any help would be much appreciated!

  Read the article

• Mysql - wondering about scaling a twitter-like application ?

  - by user246114

  Hi, I'm developing an app that is vaguely similar to twitter, in that it allows users to follow one another. I wanted to do this using google app engine, for its scalability promises, but it's proving kind of difficult to get running for a few different reasons. I'd basically like to have a _users table, and a _followers table. Users go into the users table, follower relationships go into _followers. The problem is that each row in the users table will probably have like 100 corresponding records in the _followers table as users start following one another. So the number of rows is going to explode quickly. Using app engine, the volume [shouldn't] be a problem. If I go with mysql, and I do actually start to get some traction, how do I scale this up? Am I going to just end up moving to a distributed database in the end anyway? Should I fight it out with google app engine? I read that Twitter was using mysql, and they've run into this problem, and are now switching to cassandra. Thanks

  Read the article

• In sync query calls, one query causing other query to run slower. Why?

  - by Irchi

  Sorry for the long question, but I think this is an interesting situation and I couldn't find any explanations for it: I was involved in optimization of an application that performed a large number of sequential SELECT and INSERT statements on a single dedicated SQL Server database. The process needs to INSERT a large number of records into a table, but for each of them there should be some value mappings, which performed using SELECT statements on another table in the same database. For a specific execution, it took 90 minutes to run. I used a profiler (JProfiler - the application is Java-based) to determine how much time does each part of the application take. It yields that 60% of the time was spent on INSERT method calls, and almost 20% on SELECT calls (the rest distributed in other parts). After some trials, I came to this situation: I commented out the INSERT query that took 60% of the time. I was expecting for the total run time to be around 35 minutes, as I have removed 60% of the 90 minutes. But the whole process took the same 90 minutes (doing only SELECTs and nothing else), but each SELECT took longer this time! Everything was running sync, there were no async calls. And there was only one single thread of execution. SELECT and INSERT queries are very simple, and don't have anything special, and they are on different tables, but on the same DB. I tested with both the DB on the application machine, and on a remote network machine. I can't think of any explanation for this, as the Profiler (Application profiler, not SQL Profiler) reported the changes in the method call times, and by removing INSERT statements SELECT statements took longer to run. Can anyone give me some kind of explanation of what could have happened? (there can't be cache / query optimization stuff, because the queries were run in sync, and in a single thread, and it was far from affecting the cache this much) I should note that the bottleneck of the speed was in SQL server, using most of the CPU time.

  Read the article

• MATLAB: What is an appropriate Data Structure for a Matrix with Random Variable Entries?

  - by user568249

  I'm currently working in an area that is related to simulation and trying to design a data structure that can include random variables within matrices. To motivate this let me say I have the following matrix: [a b; c d] I want to find a data structure that will allow for a, b, c, d to either be real numbers or random variables. As an example, let's say that a = 1, b = -1, c = 2 but let d be a normally distributed random variable with mean 0 and SD 1. The data structure that I have in mind will give no value to d. However, I also want to be able to design a function that can take in the structure, simulate an uniform(0,1), obtain a value for d using an inverse CDF and then spit out an actual matrix. I have several ideas to do this (all related to the MATLAB icdf function) but would like to know how more experienced programmers would do this. In this application, it's important that the structure is as "lean" as possible since I will be working with very very large matrices and memory will be an issue.

  Read the article

• How to mount a blu-ray drive?

  - by Stephan Schielke

  Maybe it is for the best to close this question. This has nothing to do with a bluray drive in general anymore. Probably a hardware defect. I will try to test it with a windows system and different cables again... Thx so far. I have a bluray/dvd/cdrom drive with SATA. Ubuntu wont find it under /dev/sd wodim --devices wodim: Overview of accessible drives (1 found) : ------------------------------------------------------------------------- 0 dev='/dev/sg2' rwrw-- : 'HL-DT-ST' 'BDDVDRW CH08LS10' ------------------------------------------------------------------------- cdrecord -scanbus scsibus2: 2,0,0 200) 'HL-DT-ST' 'BDDVDRW CH08LS10' '2.00' Removable CD-ROM fdisk dont even lists it. Ubuntu only automounts blank DVDs, but neither CDROM nor Blurays. I also changed the sata slot, sata cable and the power cable. The drive works with a windows system. This happens when I try to mount: sudo mount -t auto /dev/scd0 /media/bluray mount: you must specify the filesystem type I tried all filesystems there are. I also installed makemkv. It finds the drive but not the disc. Here is my /dev ls -al /dev total 12 drwxr-xr-x 17 root root 4420 2011-11-25 19:36 . drwxr-xr-x 28 root root 4096 2011-11-25 07:12 .. crw------- 1 root root 10, 235 2011-11-25 19:28 autofs -rw-r--r-- 1 root root 630 2011-11-25 19:28 .blkid.tab -rw-r--r-- 1 root root 630 2011-11-25 19:28 .blkid.tab.old drwxr-xr-x 2 root root 700 2011-11-25 19:27 block drwxr-xr-x 2 root root 100 2011-11-25 19:27 bsg crw------- 1 root root 10, 234 2011-11-25 19:28 btrfs-control drwxr-xr-x 3 root root 60 2011-11-25 19:27 bus drwxr-xr-x 2 root root 3820 2011-11-25 19:28 char crw------- 1 root root 5, 1 2011-11-25 19:28 console lrwxrwxrwx 1 root root 11 2011-11-25 19:28 core -> /proc/kcore drwxr-xr-x 2 root root 60 2011-11-25 19:28 cpu crw------- 1 root root 10, 60 2011-11-25 19:28 cpu_dma_latency drwxr-xr-x 7 root root 140 2011-11-25 19:27 disk crw------- 1 root root 10, 61 2011-11-25 19:28 ecryptfs crw-rw---- 1 root video 29, 0 2011-11-25 19:28 fb0 lrwxrwxrwx 1 root root 13 2011-11-25 19:28 fd -> /proc/self/fd crw-rw-rw- 1 root root 1, 7 2011-11-25 19:28 full crw-rw-rw- 1 root fuse 10, 229 2011-11-25 19:28 fuse crw------- 1 root root 251, 0 2011-11-25 19:28 hidraw0 crw------- 1 root root 251, 1 2011-11-25 19:28 hidraw1 crw------- 1 root root 10, 228 2011-11-25 19:28 hpet lrwxrwxrwx 1 root root 14 2011-11-25 19:27 .initramfs -> /run/initramfs drwxr-xr-x 4 root root 220 2011-11-25 19:28 input crw------- 1 root root 1, 11 2011-11-25 19:28 kmsg srw-rw-rw- 1 root root 0 2011-11-25 19:28 log brw-rw---- 1 root disk 7, 0 2011-11-25 19:28 loop0 brw-rw---- 1 root disk 7, 1 2011-11-25 19:28 loop1 brw-rw---- 1 root disk 7, 2 2011-11-25 19:28 loop2 brw-rw---- 1 root disk 7, 3 2011-11-25 19:28 loop3 brw-rw---- 1 root disk 7, 4 2011-11-25 19:28 loop4 brw-rw---- 1 root disk 7, 5 2011-11-25 19:28 loop5 brw-rw---- 1 root disk 7, 6 2011-11-25 19:28 loop6 brw-rw---- 1 root disk 7, 7 2011-11-25 19:28 loop7 drwxr-xr-x 2 root root 60 2011-11-25 19:27 mapper crw------- 1 root root 10, 227 2011-11-25 19:28 mcelog crw-r----- 1 root kmem 1, 1 2011-11-25 19:28 mem drwxr-xr-x 2 root root 60 2011-11-25 19:27 net crw------- 1 root root 10, 59 2011-11-25 19:28 network_latency crw------- 1 root root 10, 58 2011-11-25 19:28 network_throughput crw-rw-rw- 1 root root 1, 3 2011-11-25 19:28 null crw-rw-rw- 1 root root 195, 0 2011-11-25 19:28 nvidia0 crw-rw-rw- 1 root root 195, 255 2011-11-25 19:28 nvidiactl crw------- 1 root root 1, 12 2011-11-25 19:28 oldmem crw-r----- 1 root kmem 1, 4 2011-11-25 19:28 port crw------- 1 root root 108, 0 2011-11-25 19:28 ppp crw------- 1 root root 10, 1 2011-11-25 19:28 psaux crw-rw-rw- 1 root tty 5, 2 2011-11-25 20:00 ptmx drwxr-xr-x 2 root root 0 2011-11-25 19:27 pts brw-rw---- 1 root disk 1, 0 2011-11-25 19:28 ram0 brw-rw---- 1 root disk 1, 1 2011-11-25 19:28 ram1 brw-rw---- 1 root disk 1, 10 2011-11-25 19:28 ram10 brw-rw---- 1 root disk 1, 11 2011-11-25 19:28 ram11 brw-rw---- 1 root disk 1, 12 2011-11-25 19:28 ram12 brw-rw---- 1 root disk 1, 13 2011-11-25 19:28 ram13 brw-rw---- 1 root disk 1, 14 2011-11-25 19:28 ram14 brw-rw---- 1 root disk 1, 15 2011-11-25 19:28 ram15 brw-rw---- 1 root disk 1, 2 2011-11-25 19:28 ram2 brw-rw---- 1 root disk 1, 3 2011-11-25 19:28 ram3 brw-rw---- 1 root disk 1, 4 2011-11-25 19:28 ram4 brw-rw---- 1 root disk 1, 5 2011-11-25 19:28 ram5 brw-rw---- 1 root disk 1, 6 2011-11-25 19:28 ram6 brw-rw---- 1 root disk 1, 7 2011-11-25 19:28 ram7 brw-rw---- 1 root disk 1, 8 2011-11-25 19:28 ram8 brw-rw---- 1 root disk 1, 9 2011-11-25 19:28 ram9 crw-rw-rw- 1 root root 1, 8 2011-11-25 19:28 random crw-rw-r--+ 1 root root 10, 62 2011-11-25 19:28 rfkill lrwxrwxrwx 1 root root 4 2011-11-25 19:28 rtc -> rtc0 crw------- 1 root root 254, 0 2011-11-25 19:28 rtc0 lrwxrwxrwx 1 root root 3 2011-11-25 19:38 scd0 -> sr0 brw-rw---- 1 root disk 8, 0 2011-11-25 19:28 sda brw-rw---- 1 root disk 8, 1 2011-11-25 19:28 sda1 brw-rw---- 1 root disk 8, 2 2011-11-25 19:28 sda2 brw-rw---- 1 root disk 8, 3 2011-11-25 19:28 sda3 brw-rw---- 1 root disk 8, 5 2011-11-25 19:28 sda5 brw-rw---- 1 root disk 8, 6 2011-11-25 19:28 sda6 brw-rw---- 1 root disk 8, 16 2011-11-25 19:28 sdb brw-rw---- 1 root disk 8, 17 2011-11-25 19:28 sdb1 crw-rw---- 1 root disk 21, 0 2011-11-25 19:28 sg0 crw-rw---- 1 root disk 21, 1 2011-11-25 19:28 sg1 crw-rw----+ 1 root cdrom 21, 2 2011-11-25 19:28 sg2 lrwxrwxrwx 1 root root 8 2011-11-25 19:28 shm -> /run/shm crw------- 1 root root 10, 231 2011-11-25 19:28 snapshot drwxr-xr-x 4 root root 280 2011-11-25 19:28 snd brw-rw----+ 1 root cdrom 11, 0 2011-11-25 19:38 sr0 lrwxrwxrwx 1 root root 15 2011-11-25 19:28 stderr -> /proc/self/fd/2 lrwxrwxrwx 1 root root 15 2011-11-25 19:28 stdin -> /proc/self/fd/0 lrwxrwxrwx 1 root root 15 2011-11-25 19:28 stdout -> /proc/self/fd/1 crw-rw-rw- 1 root tty 5, 0 2011-11-25 19:35 tty crw--w---- 1 root tty 4, 0 2011-11-25 19:28 tty0 crw------- 1 root root 4, 1 2011-11-25 19:28 tty1 crw--w---- 1 root tty 4, 10 2011-11-25 19:28 tty10 crw--w---- 1 root tty 4, 11 2011-11-25 19:28 tty11 crw--w---- 1 root tty 4, 12 2011-11-25 19:28 tty12 crw--w---- 1 root tty 4, 13 2011-11-25 19:28 tty13 crw--w---- 1 root tty 4, 14 2011-11-25 19:28 tty14 crw--w---- 1 root tty 4, 15 2011-11-25 19:28 tty15 crw--w---- 1 root tty 4, 16 2011-11-25 19:28 tty16 crw--w---- 1 root tty 4, 17 2011-11-25 19:28 tty17 crw--w---- 1 root tty 4, 18 2011-11-25 19:28 tty18 crw--w---- 1 root tty 4, 19 2011-11-25 19:28 tty19 crw------- 1 root root 4, 2 2011-11-25 19:28 tty2 crw--w---- 1 root tty 4, 20 2011-11-25 19:28 tty20 crw--w---- 1 root tty 4, 21 2011-11-25 19:28 tty21 crw--w---- 1 root tty 4, 22 2011-11-25 19:28 tty22 crw--w---- 1 root tty 4, 23 2011-11-25 19:28 tty23 crw--w---- 1 root tty 4, 24 2011-11-25 19:28 tty24 crw--w---- 1 root tty 4, 25 2011-11-25 19:28 tty25 crw--w---- 1 root tty 4, 26 2011-11-25 19:28 tty26 crw--w---- 1 root tty 4, 27 2011-11-25 19:28 tty27 crw--w---- 1 root tty 4, 28 2011-11-25 19:28 tty28 crw--w---- 1 root tty 4, 29 2011-11-25 19:28 tty29 crw------- 1 root root 4, 3 2011-11-25 19:28 tty3 crw--w---- 1 root tty 4, 30 2011-11-25 19:28 tty30 crw--w---- 1 root tty 4, 31 2011-11-25 19:28 tty31 crw--w---- 1 root tty 4, 32 2011-11-25 19:28 tty32 crw--w---- 1 root tty 4, 33 2011-11-25 19:28 tty33 crw--w---- 1 root tty 4, 34 2011-11-25 19:28 tty34 crw--w---- 1 root tty 4, 35 2011-11-25 19:28 tty35 crw--w---- 1 root tty 4, 36 2011-11-25 19:28 tty36 crw--w---- 1 root tty 4, 37 2011-11-25 19:28 tty37 crw--w---- 1 root tty 4, 38 2011-11-25 19:28 tty38 crw--w---- 1 root tty 4, 39 2011-11-25 19:28 tty39 crw------- 1 root root 4, 4 2011-11-25 19:28 tty4 crw--w---- 1 root tty 4, 40 2011-11-25 19:28 tty40 crw--w---- 1 root tty 4, 41 2011-11-25 19:28 tty41 crw--w---- 1 root tty 4, 42 2011-11-25 19:28 tty42 crw--w---- 1 root tty 4, 43 2011-11-25 19:28 tty43 crw--w---- 1 root tty 4, 44 2011-11-25 19:28 tty44 crw--w---- 1 root tty 4, 45 2011-11-25 19:28 tty45 crw--w---- 1 root tty 4, 46 2011-11-25 19:28 tty46 crw--w---- 1 root tty 4, 47 2011-11-25 19:28 tty47 crw--w---- 1 root tty 4, 48 2011-11-25 19:28 tty48 crw--w---- 1 root tty 4, 49 2011-11-25 19:28 tty49 crw------- 1 root root 4, 5 2011-11-25 19:28 tty5 crw--w---- 1 root tty 4, 50 2011-11-25 19:28 tty50 crw--w---- 1 root tty 4, 51 2011-11-25 19:28 tty51 crw--w---- 1 root tty 4, 52 2011-11-25 19:28 tty52 crw--w---- 1 root tty 4, 53 2011-11-25 19:28 tty53 crw--w---- 1 root tty 4, 54 2011-11-25 19:28 tty54 crw--w---- 1 root tty 4, 55 2011-11-25 19:28 tty55 crw--w---- 1 root tty 4, 56 2011-11-25 19:28 tty56 crw--w---- 1 root tty 4, 57 2011-11-25 19:28 tty57 crw--w---- 1 root tty 4, 58 2011-11-25 19:28 tty58 crw--w---- 1 root tty 4, 59 2011-11-25 19:28 tty59 crw------- 1 root root 4, 6 2011-11-25 19:28 tty6 crw--w---- 1 root tty 4, 60 2011-11-25 19:28 tty60 crw--w---- 1 root tty 4, 61 2011-11-25 19:28 tty61 crw--w---- 1 root tty 4, 62 2011-11-25 19:28 tty62 crw--w---- 1 root tty 4, 63 2011-11-25 19:28 tty63 crw--w---- 1 root tty 4, 7 2011-11-25 19:28 tty7 crw--w---- 1 root tty 4, 8 2011-11-25 19:28 tty8 crw--w---- 1 root tty 4, 9 2011-11-25 19:28 tty9 crw------- 1 root root 5, 3 2011-11-25 19:28 ttyprintk crw-rw---- 1 root dialout 4, 64 2011-11-25 19:28 ttyS0 crw-rw---- 1 root dialout 4, 65 2011-11-25 19:28 ttyS1 crw-rw---- 1 root dialout 4, 74 2011-11-25 19:28 ttyS10 crw-rw---- 1 root dialout 4, 75 2011-11-25 19:28 ttyS11 crw-rw---- 1 root dialout 4, 76 2011-11-25 19:28 ttyS12 crw-rw---- 1 root dialout 4, 77 2011-11-25 19:28 ttyS13 crw-rw---- 1 root dialout 4, 78 2011-11-25 19:28 ttyS14 crw-rw---- 1 root dialout 4, 79 2011-11-25 19:28 ttyS15 crw-rw---- 1 root dialout 4, 80 2011-11-25 19:28 ttyS16 crw-rw---- 1 root dialout 4, 81 2011-11-25 19:28 ttyS17 crw-rw---- 1 root dialout 4, 82 2011-11-25 19:28 ttyS18 crw-rw---- 1 root dialout 4, 83 2011-11-25 19:28 ttyS19 crw-rw---- 1 root dialout 4, 66 2011-11-25 19:28 ttyS2 crw-rw---- 1 root dialout 4, 84 2011-11-25 19:28 ttyS20 crw-rw---- 1 root dialout 4, 85 2011-11-25 19:28 ttyS21 crw-rw---- 1 root dialout 4, 86 2011-11-25 19:28 ttyS22 crw-rw---- 1 root dialout 4, 87 2011-11-25 19:28 ttyS23 crw-rw---- 1 root dialout 4, 88 2011-11-25 19:28 ttyS24 crw-rw---- 1 root dialout 4, 89 2011-11-25 19:28 ttyS25 crw-rw---- 1 root dialout 4, 90 2011-11-25 19:28 ttyS26 crw-rw---- 1 root dialout 4, 91 2011-11-25 19:28 ttyS27 crw-rw---- 1 root dialout 4, 92 2011-11-25 19:28 ttyS28 crw-rw---- 1 root dialout 4, 93 2011-11-25 19:28 ttyS29 crw-rw---- 1 root dialout 4, 67 2011-11-25 19:28 ttyS3 crw-rw---- 1 root dialout 4, 94 2011-11-25 19:28 ttyS30 crw-rw---- 1 root dialout 4, 95 2011-11-25 19:28 ttyS31 crw-rw---- 1 root dialout 4, 68 2011-11-25 19:28 ttyS4 crw-rw---- 1 root dialout 4, 69 2011-11-25 19:28 ttyS5 crw-rw---- 1 root dialout 4, 70 2011-11-25 19:28 ttyS6 crw-rw---- 1 root dialout 4, 71 2011-11-25 19:28 ttyS7 crw-rw---- 1 root dialout 4, 72 2011-11-25 19:28 ttyS8 crw-rw---- 1 root dialout 4, 73 2011-11-25 19:28 ttyS9 d rwxr-xr-x 3 root root 60 2011-11-25 19:28 .udev crw-r----- 1 root root 10, 223 2011-11-25 19:28 uinput crw-rw-rw- 1 root root 1, 9 2011-11-25 19:28 urandom drwxr-xr-x 2 root root 60 2011-11-25 19:27 usb crw------- 1 root root 252, 0 2011-11-25 19:28 usbmon0 crw------- 1 root root 252, 1 2011-11-25 19:28 usbmon1 crw------- 1 root root 252, 2 2011-11-25 19:28 usbmon2 crw------- 1 root root 252, 3 2011-11-25 19:28 usbmon3 crw------- 1 root root 252, 4 2011-11-25 19:28 usbmon4 crw------- 1 root root 252, 5 2011-11-25 19:28 usbmon5 crw------- 1 root root 252, 6 2011-11-25 19:28 usbmon6 crw------- 1 root root 252, 7 2011-11-25 19:28 usbmon7 crw------- 1 root root 252, 8 2011-11-25 19:28 usbmon8 drwxr-xr-x 4 root root 80 2011-11-25 19:28 v4l crw------- 1 root root 10, 57 2011-11-25 19:28 vboxdrv crw------- 1 root root 10, 56 2011-11-25 19:28 vboxnetctl drwxr-x--- 4 root vboxusers 80 2011-11-25 19:28 vboxusb crw-rw---- 1 root tty 7, 0 2011-11-25 19:28 vcs crw-rw---- 1 root tty 7, 1 2011-11-25 19:28 vcs1 crw-rw---- 1 root tty 7, 2 2011-11-25 19:28 vcs2 crw-rw---- 1 root tty 7, 3 2011-11-25 19:28 vcs3 crw-rw---- 1 root tty 7, 4 2011-11-25 19:28 vcs4 crw-rw---- 1 root tty 7, 5 2011-11-25 19:28 vcs5 crw-rw---- 1 root tty 7, 6 2011-11-25 19:28 vcs6 crw-rw---- 1 root tty 7, 128 2011-11-25 19:28 vcsa crw-rw---- 1 root tty 7, 129 2011-11-25 19:28 vcsa1 crw-rw---- 1 root tty 7, 130 2011-11-25 19:28 vcsa2 crw-rw---- 1 root tty 7, 131 2011-11-25 19:28 vcsa3 crw-rw---- 1 root tty 7, 132 2011-11-25 19:28 vcsa4 crw-rw---- 1 root tty 7, 133 2011-11-25 19:28 vcsa5 crw-rw---- 1 root tty 7, 134 2011-11-25 19:28 vcsa6 crw------- 1 root root 10, 63 2011-11-25 19:28 vga_arbiter crw-rw----+ 1 root video 81, 0 2011-11-25 19:28 video0 crw-rw-rw- 1 root root 1, 5 2011-11-25 19:28 zero sg_scan -i gives me: sudo sg_scan -i /dev/sg0: scsi0 channel=0 id=0 lun=0 [em] ATA ST31000524NS SN12 [rmb=0 cmdq=0 pqual=0 pdev=0x0] /dev/sg1: scsi0 channel=0 id=1 lun=0 [em] ATA WDC WD15EADS-00S 01.0 [rmb=0 cmdq=0 pqual=0 pdev=0x0] /dev/sg2: scsi2 channel=0 id=0 lun=0 [em] HL-DT-ST BDDVDRW CH08LS10 2.00 [rmb=1 cmdq=0 pqual=0 pdev=0x5] sg_map gives me: /dev/sg0 /dev/sda /dev/sg1 /dev/sdb /dev/sg2 /dev/scd0 lsscsi -l gives me: [0:0:0:0] disk ATA ST31000524NS SN12 /dev/sda state=running queue_depth=1 scsi_level=6 type=0 device_blocked=0 timeout=30 [0:0:1:0] disk ATA WDC WD15EADS-00S 01.0 /dev/sdb state=running queue_depth=1 scsi_level=6 type=0 device_blocked=0 timeout=30 [2:0:0:0] cd/dvd HL-DT-ST BDDVDRW CH08LS10 2.00 /dev/sr0 state=running queue_depth=1 scsi_level=6 type=5 device_blocked=0 timeout=30 my udf mod is: filename: /lib/modules/3.0.0-14-generic/kernel/fs/udf/udf.ko license: GPL description: Universal Disk Format Filesystem author: Ben Fennema srcversion: 6ABDE012374D96B9685B8E5 depends: crc-itu-t vermagic: 3.0.0-14-generic SMP mod_unload modversions Do I need special drivers or mods enabled? Do I need to change some BIOS settings? edit: Somehow I am now able to fire the mount command without any filesystem errors, but now I get: mount: no medium found on /dev/sr0

  Read the article

• Clustering Basics and Challenges

  - by Karoly Vegh

  For upcoming posts it seemed to be a good idea to dedicate some time for cluster basic concepts and theory. This post misses a lot of details that would explode the articlesize, should you have questions, do not hesitate to ask them in the comments.  The goal here is to get some concepts straight. I can't promise to give you an overall complete definitions of cluster, cluster agent, quorum, voting, fencing, split brain condition, so the following is more of an explanation. Here we go. -------- Cluster, HA, failover, switchover, scalability -------- An attempted definition of a Cluster: A cluster is a set (2+) server nodes dedicated to keep application services alive, communicating through the cluster software/framework with eachother, test and probe health status of servernodes/services and with quorum based decisions and with switchover/failover techniques keep the application services running on them available. That is, should a node that runs a service unexpectedly lose functionality/connection, the other ones would take over the and run the services, so that availability is guaranteed. To provide availability while strictly sticking to a consistent clusterconfiguration is the main goal of a cluster.  At this point we have to add that this defines a HA-cluster, a High-Availability cluster, where the clusternodes are planned to run the services in an active-standby, or failover fashion. An example could be a single instance database. Some applications can be run in a distributed or scalable fashion. In the latter case instances of the application run actively on separate clusternodes serving servicerequests simultaneously. An example for this version could be a webserver that forwards connection requests to many backend servers in a round-robin way. Or a database running in active-active RAC setup.  -------- Cluster arhitecture, interconnect, topologies -------- Now, what is a cluster made of? Servers, right. These servers (the clusternodes) need to communicate. This of course happens over the network, usually over dedicated network interfaces interconnecting all the clusternodes. These connection are called interconnects.How many clusternodes are in a cluster? There are different cluster topologies. The most simple one is a clustered pair topology, involving only two clusternodes:  There are several more topologies, clicking the image above will take you to the relevant documentation. Also, to answer the question Solaris Cluster allows you to run up to 16 servers in a cluster. Where shall these clusternodes be placed? A very important question. The right answer is: It depends on what you plan to achieve with the cluster. Do you plan to avoid only a server outage? Then you can place them right next to eachother in the datacenter. Do you need to avoid DataCenter outage? In that case of course you should place them at least in different fire zones. Or in two geographically distant DataCenters to avoid disasters like floods, large-scale fires or power outages. We call this a stretched- or campus cluster, the clusternodes being several kilometers away from eachother. To cover really large distances, you probably need to move to a GeoCluster, which is a different kind of animal.  What is a geocluster? A Geographic Cluster in Solaris Cluster terms is actually a metacluster between two, separate (locally-HA) clusters.  -------- Cluster resource types, agents, resources, resource groups -------- So how does the cluster manage my applications? The cluster needs to start, stop and probe your applications. If you application runs, the cluster needs to check regularly if the application state is healthy, does it respond over the network, does it have all the processes running, etc. This is called probing. If the cluster deems the application is in a faulty state, then it can try to restart it locally or decide to switch (stop on node A, start on node B) the service. Starting, stopping and probing are the three actions that a cluster agent does. There are many different kinds of agents included in Solaris Cluster, but you can build your own too. Examples are an agent that manages (mounts, moves) ZFS filesystems, or the Oracle DB HA agent that cares about the database, or an agent that moves a floating IP address between nodes. There are lots of other agents included for Apache, Tomcat, MySQL, Oracle DB, Oracle Weblogic, Zones, LDoms, NFS, DNS, etc.We also need to clarify the difference between a cluster resource and the cluster resource group.A cluster resource is something that is managed by a cluster agent. Cluster resource types are included in Solaris cluster (see above, e.g. HAStoragePlus, HA-Oracle, LogicalHost). You can group cluster resources into cluster resourcegroups, and switch these groups together from one node to another. To stick to the example above, to move an Oracle DB service from one node to another, you have to switch the group between nodes, and the agents of the cluster resources in the group will do the following:  On node A Shut down the DB Unconfigure the LogicalHost IP the DB Listener listens on unmount the filesystem   Then, on node B: mount the FS configure the IP  startup the DB -------- Voting, Quorum, Split Brain Condition, Fencing, Amnesia -------- How do the clusternodes agree upon their action? How do they decide which node runs what services? Another important question. Running a cluster is a strictly democratic thing.Every node has votes, and you need the majority of votes to have the deciding power. Now, this is usually no problem, clusternodes think very much all alike. Still, every action needs to be governed upon in a productive system, and has to be agreed upon. Agreeing is easy as long as the clusternodes all behave and talk to eachother over the interconnect. But if the interconnect is gone/down, this all gets tricky and confusing. Clusternodes think like this: "My job is to run these services. The other node does not answer my interconnect communication, it must be down. I'd better take control and run the services!". The problem is, as I have already mentioned, clusternodes very much think alike. If the interconnect is gone, they all assume the other node is down, and they all want to mount the data backend, enable the IP and run the database. Double IPs, double mounts, double DB instances - now that is trouble. Also, in a 2-node cluster they both have only 50% of the votes, that is, they themselves alone are not allowed to run a cluster.  This is where you need a quorum device. According to Wikipedia, the "requirement for a quorum is protection against totally unrepresentative action in the name of the body by an unduly small number of persons.". They need additional votes to run the cluster. For this requirement a 2-node cluster needs a quorum device or a quorum server. If the interconnect is gone, (this is what we call a split brain condition) both nodes start to race and try to reserve the quorum device to themselves. They do this, because the quorum device bears an additional vote, that could ensure majority (50% +1). The one that manages to lock the quorum device (e.g. if it's an FC LUN, it SCSI reserves it) wins the right to build/run a cluster, the other one - realizing he was late - panics/reboots to ensure the cluster config stays consistent.  Losing the interconnect isn't only endangering the availability of services, but it also endangers the cluster configuration consistence. Just imagine node A being down and during that the cluster configuration changes. Now node B goes down, and node A comes up. It isn't uptodate about the cluster configuration's changes so it will refuse to start a cluster, since that would lead to cluster amnesia, that is the cluster had some changes, but now runs with an older cluster configuration repository state, that is it's like it forgot about the changes.  Also, to ensure application data consistence, the clusternode that wins the race makes sure that a server that isn't part of or can't currently join the cluster can access the devices. This procedure is called fencing. This usually happens to storage LUNs via SCSI reservation.  Now, another important question: Where do I place the quorum disk?  Imagine having two sites, two separate datacenters, one in the north of the city and the other one in the south part of it. You run a stretched cluster in the clustered pair topology. Where do you place the quorum disk/server? If you put it into the north DC, and that gets hit by a meteor, you lose one clusternode, which isn't a problem, but you also lose your quorum, and the south clusternode can't keep the cluster running lacking the votes. This problem can't be solved with two sites and a campus cluster. You will need a third site to either place the quorum server to, or a third clusternode. Otherwise, lacking majority, if you lose the site that had your quorum, you lose the cluster. Okay, we covered the very basics. We haven't talked about virtualization support, CCR, ClusterFilesystems, DID devices, affinities, storage-replication, management tools, upgrade procedures - should those be interesting for you, let me know in the comments, along with any other questions. Given enough demand I'd be glad to write a followup post too. Now I really want to move on to the second part in the series: ClusterInstallation.  Oh, as for additional source of information, I recommend the documentation: http://docs.oracle.com/cd/E23623_01/index.html, and the OTN Oracle Solaris Cluster site: http://www.oracle.com/technetwork/server-storage/solaris-cluster/index.html

  Read the article

• Oracle Solaris 11 ZFS Lab for Openworld 2012

  - by user12626122

  Preface This is the content from the Oracle Openworld 2012 ZFS lab. It was well attended - the feedback was that it was a little short - thats probably because in writing it I bacame very time-concious after the ASM/ACFS on Solaris extravaganza I ran last year which was almost too long for mortal man to finish in the 1 hour session. Enjoy. Table of Contents Exercise Z.1: ZFS Pools Exercise Z.2: ZFS File Systems Exercise Z.3: ZFS Compression Exercise Z.4: ZFS Deduplication Exercise Z.5: ZFS Encryption Exercise Z.6: Solaris 11 Shadow Migration Introduction This set of exercises is designed to briefly demonstrate new features in Solaris 11 ZFS file system: Deduplication, Encryption and Shadow Migration. Also included is the creation of zpools and zfs file systems - the basic building blocks of the technology, and also Compression which is the compliment of Deduplication. The exercises are just introductions - you are referred to the ZFS Adminstration Manual for further information. From Solaris 11 onward the online manual pages consist of zpool(1M) and zfs(1M) with further feature-specific information in zfs_allow(1M), zfs_encrypt(1M) and zfs_share(1M). The lab is easily carried out in a VirtualBox running Solaris 11 with 6 virtual 3 Gb disks to play with. Exercise Z.1: ZFS Pools Task: You have several disks to use for your new file system. Create a new zpool and a file system within it. Lab: You will check the status of existing zpools, create your own pool and expand it. Your Solaris 11 installation already has a root ZFS pool. It contains the root file system. Check this: root@solaris:~# zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT rpool 15.9G 6.62G 9.25G 41% 1.00x ONLINE - root@solaris:~# zpool status pool: rpool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 c3t0d0s0 ONLINE 0 0 0 errors: No known data errors Note the disk device the root pool is on - c3t0d0s0 Now you will create your own ZFS pool. First you will check what disks are available: root@solaris:~# echo | format Searching for disks...done AVAILABLE DISK SELECTIONS: 0. c3t0d0 <ATA-VBOX HARDDISK-1.0 cyl 2085 alt 2 hd 255 sec 63> /pci@0,0/pci8086,2829@d/disk@0,0 1. c3t2d0 <ATA-VBOX HARDDISK-1.0 cyl 1534 alt 2 hd 128 sec 32> /pci@0,0/pci8086,2829@d/disk@2,0 2. c3t3d0 <ATA-VBOX HARDDISK-1.0 cyl 1534 alt 2 hd 128 sec 32> /pci@0,0/pci8086,2829@d/disk@3,0 3. c3t4d0 <ATA-VBOX HARDDISK-1.0 cyl 1534 alt 2 hd 128 sec 32> /pci@0,0/pci8086,2829@d/disk@4,0 4. c3t5d0 <ATA-VBOX HARDDISK-1.0 cyl 1534 alt 2 hd 128 sec 32> /pci@0,0/pci8086,2829@d/disk@5,0 5. c3t6d0 <ATA-VBOX HARDDISK-1.0 cyl 1534 alt 2 hd 128 sec 32> /pci@0,0/pci8086,2829@d/disk@6,0 6. c3t7d0 <ATA-VBOX HARDDISK-1.0 cyl 1534 alt 2 hd 128 sec 32> /pci@0,0/pci8086,2829@d/disk@7,0 Specify disk (enter its number): Specify disk (enter its number): The root disk is numbered 0. The others are free for use. Try creating a simple pool and observe the error message: root@solaris:~# zpool create mypool c3t2d0 c3t3d0 'mypool' successfully created, but with no redundancy; failure of one device will cause loss of the pool So destroy that pool and create a mirrored pool instead: root@solaris:~# zpool destroy mypool root@solaris:~# zpool create mypool mirror c3t2d0 c3t3d0 root@solaris:~# zpool status mypool pool: mypool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM mypool ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 c3t2d0 ONLINE 0 0 0 c3t3d0 ONLINE 0 0 0 errors: No known data errors Back to topExercise Z.2: ZFS File Systems Task: You have to create file systems for later exercises. You can see that when a pool is created, a file system of the same name is created: root@solaris:~# zfs list NAME USED AVAIL REFER MOUNTPOINT mypool 86.5K 2.94G 31K /mypool Create your filesystems and mountpoints as follows: root@solaris:~# zfs create -o mountpoint=/data1 mypool/mydata1 The -o option sets the mount point and automatically creates the necessary directory. root@solaris:~# zfs list mypool/mydata1 NAME USED AVAIL REFER MOUNTPOINT mypool/mydata1 31K 2.94G 31K /data1 Back to top Exercise Z.3: ZFS Compression Task:Try out different forms of compression available in ZFS Lab:Create 2nd filesystem with compression, fill both file systems with the same data, observe results You can see from the zfs(1) manual page that there are several types of compression available to you, set with the property=value syntax: compression=on | off | lzjb | gzip | gzip-N | zle Controls the compression algorithm used for this dataset. The lzjb compression algorithm is optimized for performance while providing decent data compression. Setting compression to on uses the lzjb compression algorithm. The gzip compression algorithm uses the same compression as the gzip(1) command. You can specify the gzip level by using the value gzip-N where N is an integer from 1 (fastest) to 9 (best compression ratio). Currently, gzip is equivalent to gzip-6 (which is also the default for gzip(1)). Create a second filesystem with compression turned on. Note how you set and get your values separately: root@solaris:~# zfs create -o mountpoint=/data2 mypool/mydata2 root@solaris:~# zfs set compression=gzip-9 mypool/mydata2 root@solaris:~# zfs get compression mypool/mydata1 NAME PROPERTY VALUE SOURCE mypool/mydata1 compression off default root@solaris:~# zfs get compression mypool/mydata2 NAME PROPERTY VALUE SOURCE mypool/mydata2 compression gzip-9 local Now you can copy the contents of /usr/lib into both your normal and compressing filesystem and observe the results. Don't forget the dot or period (".") in the find(1) command below: root@solaris:~# cd /usr/lib root@solaris:/usr/lib# find . -print | cpio -pdv /data1 root@solaris:/usr/lib# find . -print | cpio -pdv /data2 The copy into the compressing file system takes longer - as it has to perform the compression but the results show the effect: root@solaris:/usr/lib# zfs list NAME USED AVAIL REFER MOUNTPOINT mypool 1.35G 1.59G 31K /mypool mypool/mydata1 1.01G 1.59G 1.01G /data1 mypool/mydata2 341M 1.59G 341M /data2 Note that the available space in the pool is shared amongst the file systems. This behavior can be modified using quotas and reservations which are not covered in this lab but are covered extensively in the ZFS Administrators Guide. Back to top Exercise Z.4: ZFS Deduplication The deduplication property is used to remove redundant data from a ZFS file system. With the property enabled duplicate data blocks are removed synchronously. The result is that only unique data is stored and common componenents are shared. Task:See how to implement deduplication and its effects Lab: You will create a ZFS file system with deduplication turned on and see if it reduces the amount of physical storage needed when we again fill it with a copy of /usr/lib. root@solaris:/usr/lib# zfs destroy mypool/mydata2 root@solaris:/usr/lib# zfs set dedup=on mypool/mydata1 root@solaris:/usr/lib# rm -rf /data1/* root@solaris:/usr/lib# mkdir /data1/2nd-copy root@solaris:/usr/lib# zfs list NAME USED AVAIL REFER MOUNTPOINT mypool 1.02M 2.94G 31K /mypool mypool/mydata1 43K 2.94G 43K /data1 root@solaris:/usr/lib# find . -print | cpio -pd /data1 2142768 blocks root@solaris:/usr/lib# zfs list NAME USED AVAIL REFER MOUNTPOINT mypool 1.02G 1.99G 31K /mypool mypool/mydata1 1.01G 1.99G 1.01G /data1 root@solaris:/usr/lib# find . -print | cpio -pd /data1/2nd-copy 2142768 blocks root@solaris:/usr/lib#zfs list NAME USED AVAIL REFER MOUNTPOINT mypool 1.99G 1.96G 31K /mypool mypool/mydata1 1.98G 1.96G 1.98G /data1 You could go on creating copies for quite a while...but you get the idea. Note that deduplication and compression can be combined: the compression acts on metadata. Deduplication works across file systems in a pool and there is a zpool-wide property dedupratio: root@solaris:/usr/lib# zpool get dedupratio mypool NAME PROPERTY VALUE SOURCE mypool dedupratio 4.30x - Deduplication can also be checked using "zpool list": root@solaris:/usr/lib# zpool list NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT mypool 2.98G 1001M 2.01G 32% 4.30x ONLINE - rpool 15.9G 6.66G 9.21G 41% 1.00x ONLINE - Before moving on to the next topic, destroy that dataset and free up some space: root@solaris:~# zfs destroy mypool/mydata1 Back to top Exercise Z.5: ZFS Encryption Task: Encrypt sensitive data. Lab: Explore basic ZFS encryption. This lab only covers the basics of ZFS Encryption. In particular it does not cover various aspects of key management. Please see the ZFS Adminastrion Manual and the zfs_encrypt(1M) manual page for more detail on this functionality. Back to top root@solaris:~# zfs create -o encryption=on mypool/data2 Enter passphrase for 'mypool/data2': ******** Enter again: ******** root@solaris:~# Creation of a descendent dataset shows that encryption is inherited from the parent: root@solaris:~# zfs create mypool/data2/data3 root@solaris:~# zfs get -r encryption,keysource,keystatus,checksum mypool/data2 NAME PROPERTY VALUE SOURCE mypool/data2 encryption on local mypool/data2 keysource passphrase,prompt local mypool/data2 keystatus available - mypool/data2 checksum sha256-mac local mypool/data2/data3 encryption on inherited from mypool/data2 mypool/data2/data3 keysource passphrase,prompt inherited from mypool/data2 mypool/data2/data3 keystatus available - mypool/data2/data3 checksum sha256-mac inherited from mypool/data2 You will find the online manual page zfs_encrypt(1M) contains examples. In particular, if time permits during this lab session you may wish to explore the changing of a key using "zfs key -c mypool/data2". Exercise Z.6: Shadow Migration Shadow Migration allows you to migrate data from an old file system to a new file system while simultaneously allowing access and modification to the new file system during the process. You can use Shadow Migration to migrate a local or remote UFS or ZFS file system to a local file system. Task: You wish to migrate data from one file system (UFS, ZFS, VxFS) to ZFS while mainaining access to it. Lab: Create the infrastructure for shadow migration and transfer one file system into another. First create the file system you want to migrate root@solaris:~# zpool create oldstuff c3t4d0 root@solaris:~# zfs create oldstuff/forgotten Then populate it with some files: root@solaris:~# cd /var/adm root@solaris:/var/adm# find . -print | cpio -pdv /oldstuff/forgotten You need the shadow-migration package installed: root@solaris:~# pkg install shadow-migration Packages to install: 1 Create boot environment: No Create backup boot environment: No Services to change: 1 DOWNLOAD PKGS FILES XFER (MB) Completed 1/1 14/14 0.2/0.2 PHASE ACTIONS Install Phase 39/39 PHASE ITEMS Package State Update Phase 1/1 Image State Update Phase 2/2 You then enable the shadowd service: root@solaris:~# svcadm enable shadowd root@solaris:~# svcs shadowd STATE STIME FMRI online 7:16:09 svc:/system/filesystem/shadowd:default Set the filesystem to be migrated to read-only root@solaris:~# zfs set readonly=on oldstuff/forgotten Create a new zfs file system with the shadow property set to the file system to be migrated: root@solaris:~# zfs create -o shadow=file:///oldstuff/forgotten mypool/remembered Use the shadowstat(1M) command to see the progress of the migration: root@solaris:~# shadowstat EST BYTES BYTES ELAPSED DATASET XFRD LEFT ERRORS TIME mypool/remembered 92.5M - - 00:00:59 mypool/remembered 99.1M 302M - 00:01:09 mypool/remembered 109M 260M - 00:01:19 mypool/remembered 133M 304M - 00:01:29 mypool/remembered 149M 339M - 00:01:39 mypool/remembered 156M 86.4M - 00:01:49 mypool/remembered 156M 8E 29 (completed) Note that if you had created /mypool/remembered as encrypted, this would be the preferred method of encrypting existing data. Similarly for compressing or deduplicating existing data. The procedure for migrating a file system over NFS is similar - see the ZFS Administration manual. That concludes this lab session.

  Read the article

• HTML Tidy in NetBeans IDE (Part 2)

  - by Geertjan

  This is what I was aiming for in the previous blog entry: What you can see above (especially if you click to enlarge it) is that I have HTML Tidy integrated into the NetBeans analyzer functionality, which is pluggable from 7.2 onwards. Well, if you set an implementation dependency on "Static Analysis Core", since it's not an official API yet. Also, the scopes of the analyzer functionality are not pluggable. That means you can 'only' set the analyzer's scope to one or more projects, one or more packages, or one or more files. Not one or more folders, which means you can't have a bunch off HTML files in a folder that you access via the Favorites window and then run the analyzer on that folder (or on multiple folders). Thus, to try out my new code, I had to put some HTML files into a package inside a Java application. Then I chose that package as the scope of the analyzer. Then I ran all the analyzers (i.e., standard NetBeans Java hints, FindBugs, as well as my HTML Tidy extension) on that package. The screenshot above is the result. Here's all the code for the above, which is a port of the Action code from the previous blog entry into a new Analyzer implementation: import java.io.IOException; import java.io.PrintWriter; import java.io.StringWriter; import java.util.ArrayList; import java.util.Collections; import java.util.List; import javax.swing.JComponent; import javax.swing.text.Document; import org.netbeans.api.fileinfo.NonRecursiveFolder; import org.netbeans.modules.analysis.spi.Analyzer; import org.netbeans.modules.analysis.spi.Analyzer.AnalyzerFactory; import org.netbeans.modules.analysis.spi.Analyzer.Context; import org.netbeans.modules.analysis.spi.Analyzer.CustomizerProvider; import org.netbeans.modules.analysis.spi.Analyzer.WarningDescription; import org.netbeans.spi.editor.hints.ErrorDescription; import org.netbeans.spi.editor.hints.ErrorDescriptionFactory; import org.netbeans.spi.editor.hints.Severity; import org.openide.cookies.EditorCookie; import org.openide.filesystems.FileObject; import org.openide.loaders.DataObject; import org.openide.util.Exceptions; import org.openide.util.lookup.ServiceProvider; import org.w3c.tidy.Tidy; public class TidyAnalyzer implements Analyzer {     private final Context ctx;     private TidyAnalyzer(Context cntxt) {         this.ctx = cntxt;     }     @Override     public Iterable<? extends ErrorDescription> analyze() {         List<ErrorDescription> result = new ArrayList<ErrorDescription>();         for (NonRecursiveFolder sr : ctx.getScope().getFolders()) {             FileObject folder = sr.getFolder();             for (FileObject fo : folder.getChildren()) {                 for (ErrorDescription ed : doRunHTMLTidy(fo)) {                     if (fo.getMIMEType().equals("text/html")) {                         result.add(ed);                     }                 }             }         }         return result;     }     private List<ErrorDescription> doRunHTMLTidy(FileObject sr) {         final List<ErrorDescription> result = new ArrayList<ErrorDescription>();         Tidy tidy = new Tidy();         StringWriter stringWriter = new StringWriter();         PrintWriter errorWriter = new PrintWriter(stringWriter);         tidy.setErrout(errorWriter);         try {             Document doc = DataObject.find(sr).getLookup().lookup(EditorCookie.class).openDocument();             tidy.parse(sr.getInputStream(), System.out);             String[] split = stringWriter.toString().split("\n");             for (String string : split) {                 //Bit of ugly string parsing coming up:                 if (string.startsWith("line")) {                     final int end = string.indexOf(" c");                     int lineNumber = Integer.parseInt(string.substring(0, end).replace("line ", ""));                     string = string.substring(string.indexOf(": ")).replace(":", "");                     result.add(ErrorDescriptionFactory.createErrorDescription(                             Severity.WARNING,                             string,                             doc,                             lineNumber));                 }             }         } catch (IOException ex) {             Exceptions.printStackTrace(ex);         }         return result;     }     @Override     public boolean cancel() {         return true;     }     @ServiceProvider(service = AnalyzerFactory.class)     public static final class MyAnalyzerFactory extends AnalyzerFactory {         public MyAnalyzerFactory() {             super("htmltidy", "HTML Tidy", "org/jtidy/format_misc.gif");         }         public Iterable<? extends WarningDescription> getWarnings() {             return Collections.EMPTY_LIST;         }         @Override         public <D, C extends JComponent> CustomizerProvider<D, C> getCustomizerProvider() {             return null;         }         @Override         public Analyzer createAnalyzer(Context cntxt) {             return new TidyAnalyzer(cntxt);         }     } } The above only works on packages, not on projects and not on individual files.

  Read the article

• WMI Rights required to read root\MicrosoftIISv2 in IIS7 with IIS6 compatibility mode

  - by JoeBilly

  I need to manage my IIS7 (Windows Server 2008) remotely with a WMI IIS6 API. So I added the IIS6 WMI Compatibility and IIS6 Metabase Compatibility roles to access the root\MicrosoftIIsv2 namespace. I have a domain account which is not administrator on the remote machine ; with this right, everything is ok. I configured these rights for my domain account to access the root\MicrosoftIIsv2 WMI namespace remotely ; note that these rights work perfectly on a IIS6 and Windows Server 2003 : DCOM : Account in Distributed COM Users Remote & local access to DCOM WMI : Root\CIMV2 (I need access here too) Execute methods, Enable Account, Remote Enable Root\Default (I need access here too) Execute methods, Enable Account, Remote Enable Root\MicrosoftIISv2 Execute methods, Enable Account, Provider Write, Remote Enable IIS Metabase (Metabase Explorer) : LM Full Control (W3SVC inherits these permissions) I tried to give some access on C:\Windows\System32\inetsrv too ; don't know if needed. My issue is : I can't list the IIS WebSites (\root\MicrosoftIISv2:IIsWebServerSetting.Name="W3SVC/*"). I don't get an 'access denied' but nothing is returned. My API and powershell tests can connect and execute queries in the root\MicrosoftIISv2 namespace I can read the IIsComputer class ex: Get-WmiObject IIsComputer -namespace "ROOT\MicrosoftIISv2" -authentication PacketPrivacy | SELECT * I can't read the IIsWebServerSetting, IIsWebServer ... to list the WebSites : the query returns an empty collection ex: Get-WmiObject IIsWebServerSetting -namespace "ROOT\MicrosoftIISv2" -authentication PacketPrivacy | SELECT ServerComment All queries work perfectly if the account is administrator as already said I am using PacketPrivacy authentication FI: I got a Warning Event 5605 with the Administrator right or not, that does not seem to have an impact : The root\MicrosoftIISv2 namespace is marked with the RequiresEncryption flag. Access to this namespace might be denied if the script or application does not have the appropriate authentication level. Change the authentication level to Pkt_Privacy and run the script or application again Ok, I have some more informations, when I use IIS 6 Metabase Explorer with my administrator account I can see the rights are correctly inherited for my non-administrator account. But when I try to connect using my non-administrator account, I can list the LM node, but get an "access denied, failed to get a key's data" when I try to browse the child nodes. I'll check further. I tried to Trace the WMI Activity, and everything seems OK ; this tends to confirm that the problem lies in IIS Rights.

  Read the article

• Windows Server 2012 Branchcache vs. DFS-R

  - by TheCleaner

  Warning, subjective question ahead! But hopefully a good one that won't get closed. SCENARIO: I have a branch office that currently has no on-premise server. They access everything including a DC across a 12Mbps WAN link (MPLS). The link isn't saturated, averaging around 20% utilization. The circuit is very stable and has a high SLA and excellent uptime. However, large file transfers (mainly reads, not writes) from the file server across the WAN can be slow. We don't currently utilize DFS. RESEARCH DONE: I'm aware of WAN acceleration, using either dedicated hardware (Riverbed) or a dedicated software VM (Silver Peak) for example. But the pricing is outside of our current budget and the need isn't quite there yet from our perspective (since the issue is mainly in a "pull" scenario not necessarily push/pull). I'm mainly looking at deploying a Windows server at this branch office and either utilizing DFS-R or BranchCache. Looking at a table comparison and assuming we are looking at a "hosted branchcache server" and not simply distributed: It would appear there are benefits to both, even if both are "hosted" on a server. QUESTIONS I ACTUALLY HAVE: In what scenarios do each of these techs shine and where do you choose one over the other? Looking at a hosted Branchcache server, can you set "pre-fetching" of certain folders/files on the central file server so that they are immediately accessible locally at the branch? Do you have to do this on a schedule (if it is possible)? Looking at DFS-R my concern (and apparently solved with 3rd party apps) is file locking and making sure the file gets updated properly during a write operation (ie, making sure if both copies are accessed and both are written to, which file takes precedence and what happens to the changes?). Ideal it would seem would be to lock any alternate replicas of the data, but is it really that big of an issue? Does Branchcache lock the central file for editing? Does branchcache only transmit the deltas back to the central file of what has changed? Would either technology be ill advised if the branch office server was going to be utilized as a domain controller as well?

  Read the article

< Previous Page | 86 87 88 89 90 91 92 93 94 95 96 97  | Next Page >