Search Results

Search found 61393 results on 2456 pages for 'data integration'.

Page 13/2456 | < Previous Page | 9 10 11 12 13 14 15 16 17 18 19 20  | Next Page >

• New Feature in ODI 11.1.1.6: Enterprise Data Quality Integration

  - by Julien Testut

  Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:"Times New Roman"; mso-fareast-theme-font:minor-fareast; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;} Oracle Data Integrator 11.1.1.6.0 introduces a new Open Tool called EnterpriseDataQuality which allows ODI users to invoke an Oracle Enterprise Data Quality Job from a Package. This post will give you an overview of this new feature. Oracle Enterprise Data Quality (OEDQ) provides organizations with an integrated suite of data quality tools that offer an end-to-end solution to measure, improve, and manage the quality of data from any domain, including customer and product data. The addition of the EnterpriseDataQuality Open Tool extends the inline Data Quality capabilities of Oracle Data Integrator with Oracle Enterprise Data Quality powerful data profiling, cleansing, matching, and monitoring capabilities. The EnterpriseDataQuality Open Tool can invoke any OEDQ Job stored in a Project. This Open Tool connects to an OEDQ server using a JMX (Java Management Extensions) interface. Once installed, this Open Tool will be found under Plugins in the Package Toolbox area: This EnterpriseDataQuality Open Tool takes a couple of parameters as inputs such as the Enterprise Data Quality Job and Project names, the OEDQ hostname and JMX port etc. With the EnterpriseDataQuality Open Tool, ODI customers can now incorporate their Oracle Enterprise Data Quality processes within their Data Integration workflows. You will find instructions about how to use the Enterprise Data Quality Open Tool in the Oracle Data Integrator documentation at: Using the EnterpriseDataQuality Open Tool.You can find an overview of all the new features introduced in ODI 11.1.1.6 in the following document: ODI 11.1.1.6 New Features Overview.

  Read the article

• Data Auditor by Example

  - by Jinjin.Wang

  OWB has a node Data Auditors under Oracle Module in Projects Navigator. What is data auditor and how to use it? I will give an introduction to data auditor and show its usage by examples. Data auditor is an important tool in ensuring that data quality levels meet business requirements. Data auditor validates data against a set of data rules to determine which records comply and which do not. It gathers statistical metrics on how well the data in a system complies with a rule by auditing and marking how many errors are occurring against the audited table. Data auditors are typically scheduled for regular execution as part of a process flow, to monitor the quality of the data in an operational environment such as a data warehouse or ERP system, either immediately after updates like data loads, or at regular intervals. How to use data auditor to monitor data quality? Only objects with data rules can be monitored, so the first step is to define data rules according to business requirements and apply them to the objects you want to monitor. The objects can be tables, views, materialized views, and external tables. Secondly create a data auditor containing the objects. You can configure the data auditor and set physical deployment parameters for it as optional, which will be used while running the data auditor. Then deploy and run the data auditor either manually or as part of the process flow. After execution, the data auditor sets several output values, and records that are identified as not complying with the defined data rules contained in the data auditor are written to error tables. Here is an example. We have two tables DEPARTMENTS and EMPLOYEES (see pic-1 and pic-2. Click here for DDL and data) imported into OWB. We want to gather statistical metrics on how well data in these two tables satisfies the following requirements: a. Values of the EMPLOYEES.EMPLOYEE_ID attribute are three-digit numbers. b. Valid values for EMPLOYEES.JOB_ID are IT_PROG, SA_REP, SH_CLERK, PU_CLERK, and ST_CLERK. c. EMPLOYEES.EMPLOYEE_ID is related to DEPARTMENTS.MANAGER_ID. Pic-1 EMPLOYEES Pic-2 DEPARTMENTS 1. To determine legal data within EMPLOYEES or legal relationships between data in different columns of the two tables, firstly we define data rules based on the three requirements and apply them to tables. a. The first requirement is about patterns that an attribute is allowed to conform to. We create a Domain Pattern List data rule EMPLOYEE_PATTERN_RULE here. The pattern is defined in the Oracle Database regular expression syntax as ^([0-9]{3})$ Apply data rule EMPLOYEE_PATTERN_RULE to table EMPLOYEES.

  Read the article

• Big Data – Buzz Words: What is Hadoop – Day 6 of 21

  - by Pinal Dave

  In yesterday’s blog post we learned what is NoSQL. In this article we will take a quick look at one of the four most important buzz words which goes around Big Data – Hadoop. What is Hadoop? Apache Hadoop is an open-source, free and Java based software framework offers a powerful distributed platform to store and manage Big Data. It is licensed under an Apache V2 license. It runs applications on large clusters of commodity hardware and it processes thousands of terabytes of data on thousands of the nodes. Hadoop is inspired from Google’s MapReduce and Google File System (GFS) papers. The major advantage of Hadoop framework is that it provides reliability and high availability. What are the core components of Hadoop? There are two major components of the Hadoop framework and both fo them does two of the important task for it. Hadoop MapReduce is the method to split a larger data problem into smaller chunk and distribute it to many different commodity servers. Each server have their own set of resources and they have processed them locally. Once the commodity server has processed the data they send it back collectively to main server. This is effectively a process where we process large data effectively and efficiently. (We will understand this in tomorrow’s blog post). Hadoop Distributed File System (HDFS) is a virtual file system. There is a big difference between any other file system and Hadoop. When we move a file on HDFS, it is automatically split into many small pieces. These small chunks of the file are replicated and stored on other servers (usually 3) for the fault tolerance or high availability. (We will understand this in the day after tomorrow’s blog post). Besides above two core components Hadoop project also contains following modules as well. Hadoop Common: Common utilities for the other Hadoop modules Hadoop Yarn: A framework for job scheduling and cluster resource management There are a few other projects (like Pig, Hive) related to above Hadoop as well which we will gradually explore in later blog posts. A Multi-node Hadoop Cluster Architecture Now let us quickly see the architecture of the a multi-node Hadoop cluster. A small Hadoop cluster includes a single master node and multiple worker or slave node. As discussed earlier, the entire cluster contains two layers. One of the layer of MapReduce Layer and another is of HDFC Layer. Each of these layer have its own relevant component. The master node consists of a JobTracker, TaskTracker, NameNode and DataNode. A slave or worker node consists of a DataNode and TaskTracker. It is also possible that slave node or worker node is only data or compute node. The matter of the fact that is the key feature of the Hadoop. In this introductory blog post we will stop here while describing the architecture of Hadoop. In a future blog post of this 31 day series we will explore various components of Hadoop Architecture in Detail. Why Use Hadoop? There are many advantages of using Hadoop. Let me quickly list them over here: Robust and Scalable – We can add new nodes as needed as well modify them. Affordable and Cost Effective – We do not need any special hardware for running Hadoop. We can just use commodity server. Adaptive and Flexible – Hadoop is built keeping in mind that it will handle structured and unstructured data. Highly Available and Fault Tolerant – When a node fails, the Hadoop framework automatically fails over to another node. Why Hadoop is named as Hadoop? In year 2005 Hadoop was created by Doug Cutting and Mike Cafarella while working at Yahoo. Doug Cutting named Hadoop after his son’s toy elephant. Tomorrow In tomorrow’s blog post we will discuss Buzz Word – MapReduce. Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: Big Data, PostADay, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, T SQL

  Read the article

• Master Data Management for Location Data - Oracle Site Hub

  - by david.butler(at)oracle.com

  Most MDM discussions cover key domains such as customer, supplier, product, service, and reference data. It is usually understood that these domains have complex structures and hundreds if not thousands of attributes that need governing. Location, on the other hand, strikes most people as address data. How hard can that be? But for many industries, locations are complex, and site information is critical to efficient operations and relevant analytics. Retail stores and malls, bank branches, construction sites come to mind. But one of the best industries for illustrating the power of a site mastering application is Oil & Gas.   Oracle's Master Data Management solution for location data is the Oracle Site Hub. It is a location mastering solution that enables organizations to centralize site and location specific information from heterogeneous systems, creating a single view of site information that can be leveraged across all functional departments and analytical systems.   Let's take a look at the location entities the Oracle Site Hub can manage for the Oil & Gas industry: organizations, property, land, buildings, roads, oilfield, service center, inventory site, real estate, facilities, refineries, storage tanks, vendor locations, businesses, assets; project site, area, well, basin, pipelines, critical infrastructure, offshore platform, compressor station, gas station, etc. Any site can be classified into multiple hierarchies, like organizational hierarchy, operational hierarchy, geographic hierarchy, divisional hierarchies and so on. Any site can also be associated to multiple clusters, i.e. collections of sites, and these can be used as a foundation for driving reporting, analysis, organize daily work, etc. Hierarchies can also be used to model entities which are structured or non-structured collections of nodes, like for example routes, pipelines and more. The User Defined Attribute Framework provides the needed infrastructure to add single row attributes groups like well base attributes (well IDs, well type, well structure and key characterizing measures, and more) and well geometry, and multi row attribute groups like well applications, permits, production data, activities, operations, logs, treatments, tests, drills, treatments, and KPIs. Site Hub can also model areas, lands, fields, basins, pools, platforms, eco-zones, and stratigraphic layers as specific sites, tracking their base attributes, aliases, descriptions, subcomponents and more. Midstream entities (pipelines, logistic sites, pump stations) and downstream entities (cylinders, tanks, inventories, meters, partner's sites, routes, facilities, gas stations, and competitor sites) can also be easily modeled, together with their specific attributes and relationships. Site Hub can store any type of unstructured data associated to a site. This could be stored directly or on an external content management solution, like Oracle Universal Content Management. Considering a well, for example, Site Hub can store any relevant associated multimedia file such as: CAD drawings of the well profile, structure and/or parts, engineering documents, contracts, applications, permits, logs, pictures, photos, videos and more. For any site entity, Site Hub can associate all the related assets and equipments at the site, as well as all relationships between sites, between a site and multiple parties, and between a site and any purchasable or sellable item, over time. Items can be equipment, instruments, facilities, services, products, production entities, production facilities (pipelines, batteries, compressor stations, gas plants, meters, separators, etc.), support facilities (rigs, roads, transmission or radio towers, airstrips, etc.), supplier products and services, catalogs, and more. Items can just be associated to sites using standard Site Hub features, or they can be fully mastered by implementing Oracle Product Hub. Site locations (addresses or geographical coordinates) are also managed with out-of-the-box address geo-coding capabilities coupled with Google Maps integration to deliver powerful mapping capabilities and spatial data analysis. Locations can be shared between different sites. Centered on the site location, any site can also have associated areas. Site Hub can master any site location specific information, like for example cadastral, ownership, jurisdictional, geological, seismic and more, and any site-centric area specific information, like for example economical, political, risk, weather, logistic, traffic information and more. Now if anyone ever asks you why locations need MDM, think about how all these Oil & Gas entities and attributes would translate into your business locations. To learn more about Oracle's full MDM solution for the digital oil field, here is a link to Roberto Negro's outstanding whitepaper: Oracle Site Master Data Management for mastering wells and other PPDM entities in a digital oilfield context  

  Read the article

• Optimal Data Structure for our own API

  - by vermiculus

  I'm in the early stages of writing an Emacs major mode for the Stack Exchange network; if you use Emacs regularly, this will benefit you in the end. In order to minimize the number of calls made to Stack Exchange's API (capped at 10000 per IP per day) and to just be a generally responsible citizen, I want to cache the information I receive from the network and store it in memory, waiting to be accessed again. I'm really stuck as to what data structure to store this information in. Obviously, it is going to be a list. However, as with any data structure, the choice must be determined by what data is being stored and what how it will be accessed. What, I would like to be able to store all of this information in a single symbol such as stack-api/cache. So, without further ado, stack-api/cache is a list of conses keyed by last update: `(<csite> <csite> <csite>) where <csite> would be (1362501715 . <site>) At this point, all we've done is define a simple association list. Of course, we must go deeper. Each <site> is a list of the API parameter (unique) followed by a list questions: `("codereview" <cquestion> <cquestion> <cquestion>) Each <cquestion> is, you guessed it, a cons of questions with their last update time: `(1362501715 <question>) (1362501720 . <question>) <question> is a cons of a question structure and a list of answers (again, consed with their last update time): `(<question-structure> <canswer> <canswer> <canswer> and ` `(1362501715 . <answer-structure>) This data structure is likely most accurately described as a tree, but I don't know if there's a better way to do this considering the language, Emacs Lisp (which isn't all that different from the Lisp you know and love at all). The explicit conses are likely unnecessary, but it helps my brain wrap around it better. I'm pretty sure a <csite>, for example, would just turn into (<epoch-time> <api-param> <cquestion> <cquestion> ...) Concerns: Does storing data in a potentially huge structure like this have any performance trade-offs for the system? I would like to avoid storing extraneous data, but I've done what I could and I don't think the dataset is that large in the first place (for normal use) since it's all just human-readable text in reasonable proportion. (I'm planning on culling old data using the times at the head of the list; each inherits its last-update time from its children and so-on down the tree. To what extent this cull should take place: I'm not sure.) Does storing data like this have any performance trade-offs for that which must use it? That is, will set and retrieve operations suffer from the size of the list? Do you have any other suggestions as to what a better structure might look like?

  Read the article

• How do you handle the fetchxml result data?

  - by Luke Baulch

  I have avoided working with fetchxml as I have been unsure the best way to handle the result data after calling crmService.Fetch(fetchXml). In a couple of situations, I have used an XDocument with LINQ to retrieve the data from this data structure, such as: XDocument resultset = XDocument.Parse(_service.Fetch(fetchXml)); if (resultset.Root == null || !resultset.Root.Elements("result").Any()) { return; } foreach (var displayItem in resultset.Root.Elements("result").Select(item => item.Element(displayAttributeName)).Distinct()) { if (displayItem!= null && displayItem.Value != null) { dropDownList.Items.Add(displayItem.Value); } } What is the best way to handle fetchxml result data, so that it can be easily used. Applications such as passing these records into an ASP.NET datagrid would be quite useful.

  Read the article

• Generate Entity Data Model from Data Contract

  - by CSmooth.net

  I would like to find a fast way to convert a Data Contract to a Entity Data Model. Consider the following Data Contract: [DataContract] class PigeonHouse { [DataMember] public string housename; [DataMember] public List<Pigeon> pigeons; } [DataContract] class Pigeon { [DataMember] public string name; [DataMember] public int numberOfWings; [DataMember] public int age; } Is there an easy way to create an ADO.NET Entity Data Model from this code?

  Read the article

• Detecting a Lightweight Core Data Migration

  - by hadronzoo

  I'm using Core Data's automatic lightweight migration successfully. However, when a particular entity gets created during a migration, I'd like to populate it with some data. Of course I could check if the entity is empty every time the application starts, but this seems inefficient when Core Data has a migration framework. Is it possible to detect when a lightweight migration occurs (possibly using KVO or notifications), or does this require implementing standard migrations? I've tried using the NSPersistentStoreCoordinatorStoresDidChangeNotification, but it doesn't fire when migrations occur.

  Read the article

• Getting started with massive data

  - by Max

  I'm a math guy and occasionally do some statistics/machine learning analysis consulting projects on the side. The data I have access to are usually on the smaller side, at most a couple hundred of megabytes (and almost always far less), but I want to learn more about handling and analyzing data on the gigabyte/terabyte scale. What do I need to know and what are some good resources to learn from? Hadoop/MapReduce is one obvious start. Is there a particular programming language I should pick up? (I primarily work now in Python, Ruby, R, and occasionally Java, but it seems like C and Clojure are often used for large-scale data analysis?) I'm not really familiar with the whole NoSQL movement, except that it's associated with big data. What's a good place to learn about it, and is there a particular implementation (Cassandra, CouchDB, etc.) I should get familiar with? Where can I learn about applying machine learning algorithms to huge amounts of data? My math background is mostly on the theory side, definitely not on the numerical or approximation side, and I'm guessing most of the standard ML algorithms don't really scale. Any other suggestions on things to learn would be great!

  Read the article

• what is best way to store long term data in iphone Core Data or SQLLite?

  - by AmitSri

  Hi all, I am working on i-Phone app targeting 3.1.3 and later SDK. I want to know the best way to store user's long term data on i-phone without losing performance, consistency and security. I know, that i can use Core Data, PList and SQL-Lite for storing user specific data in custom formats.But, want to know which one is good to use without compromising app performance and scalability in near future. Thanks

  Read the article

• How to view existing data in Core Data?

  - by mshsayem

  Well, may be this question is silly, but I couldn't find a way (except programmatically). I built a project (for iPhone OS 3.0) which uses Core Data. The xcdatamodel file shows the schema description, but I want to see the data in tabular form (like the management studio for mssql server or phpmyadmin for mysql). Is there any way (except coding)? What is that? Also, which file (in disk/device) those data are stored into? [ I built the tutorial (from apple) on Core Data, named Locations. They used this line somewhere in the code: NSURL *storeUrl = [NSURL fileURLWithPath: [[self applicationDocumentsDirectory] stringByAppendingPathComponent: @"Locations.sqlite"]]; But, I did not see any "xxxxx.sqlite" file in project location (nor in the disk).]

  Read the article

• Efficient alternatives to merge for larger data.frames R

  - by Etienne Low-Décarie

  I am looking for an efficient (both computer resource wise and learning/implementation wise) method to merge two larger (size1 million / 300 KB RData file) data frames. "merge" in base R and "join" in plyr appear to use up all my memory effectively crashing my system. Example load test data frame and try test.merged<-merge(test, test) or test.merged<-join(test, test, type="all") - The following post provides a list of merge and alternatives: How to join data frames in R (inner, outer, left, right)? The following allows object size inspection: https://heuristically.wordpress.com/2010/01/04/r-memory-usage-statistics-variable/ Data produced by anonym

  Read the article

• find the top K most frequent numbers in a data stream

  - by Jin

  This is more of a data structure question rather than a coding question. If I am fetching a data stream, i.e, I keep receiving float numbers once at a time, how should I keep track of the top K frequent numbers? Here my memory is 4G and I prefer to have less communication with hard drive unless necessary. I think heap is good for updating the max and min. How should I design the data structure? Thanks

  Read the article

• Druapl & Regular PHP Integration

  - by user333128

  I'm building a new website which has one core application and many content pages. Content pages are mostly dynamic and I require a way to manage this dynamic content on a regular basis. The core application's main functionality is a 3 step process or reading user data (input page), reading data from MySQL (product page) and submitting an application to an email address (application page). Ideally I would like to build the core application in regular PHP and leverage Drupal for its content management capabilities. Can Drupal and regular PHP be integrated as I suggest easily? My feeling is that coding the core application as a Drupal module(s) will add layers of complexity that could be difficult to code from the outset and maintain later on as the system matures - so I would really like to just use regular PHP. Let me explain where dynamic content (managed by the CMS) intersects with the core application: Dynamic content such as FAQ data is used both on the 'normal' help pages and also within a mini-feed displayed within core application pages down a right hand side column. In this column, 3 random questions are pulled from the database and displayed as a feed. When users click on FAQ question they are not taken away from the core application product page but are instead shown data in a pop-up window displaying the question and answer. In addition, users can browse other questions and answers through a simple navigation menu within this popup. There are 3 such like feeds as I describe above that I require on the core application product page. So, what is the ideal solution here in terms of 'keeping things simple' for both the management of dynamic content and the ease of coding the core application? Can 'regular PHP' and Drupal co-exist 'peacefully'? If so, how is this technically possible? Because there is some content managed by Drupal contained within core application pages, can the core application still be coded in regular PHP? Any advice / suggestions? Thank you! Jim.

  Read the article

• About best practices of JMS integration using TIBCO .Net client

  - by Waheed Sayed

  I'm working on an integration project where I'm talking to JMS framework using TIBCO .Net client. A colleague is recommdending a design decision based on a fear of receiving too many messages suddenly in a way that our application can't handle it. We are using Asynchronous mode while receiving messages. I'm new to that topic so I couldn't find quick best practices to support his clam or mine. Any suggestions? How can I test this subject?

  Read the article

• Continuous Integration - with what to start: CruiseControl.NET vs TeamCity vs Visual Studio Team Sys

  - by Mendy

  I'm new to Continuous Integration. I want an advice with what tool should I start deal with. I see that this is the biggest tools right now: CruiseControl.NET, TeamCity and Visual Studio Team System. I'm using this tools: Visual Studio 2010, Mercurial, NAnt, NUnit.

  Read the article

• Numerical Integration of a vector in matlab?

  - by clowny

  Hello all! Well i have an issue. I have a vector of 358 numbers. I'd like to make a numerical integration of this vector, but i don't know the function of this one. I found that we can use trapz or quad, but i don't really understand how to integrate without the function. Thanx for any help!

  Read the article

• How rspec works with rails3 for integration-tests?

  - by makevoid

  What I'm trying to ahieve is to do integration tests with webrat in rails3 like Yehuda does with test-unit in http://pivotallabs.com/talks/76-extending-rails-3 minute 34. an example: describe SomeApp it "should show the index page" visit "/" body.should =~ /hello world/ end end Does someone knows a way to do it?

  Read the article

• Can you Profile Ms test integration tests?

  - by Maslow

  I have never done any profiling, and have come across the first place that I really want to do it. Is it possible to profile my MS integration tests? What tools do I need? is there something built-in to VS 2008 professional?

  Read the article

• Continous integration with Reporting Services

  - by SDReyes

  I'm implementing a continuous integration environment with SVN and reporting services. The reports are stored in the SVN repository. when a change occurs, they are automatically downloaded from the repository, and any file changed should be uploaded to the reporting services server. How could you automate the upload/update process for .rdl files?

  Read the article

• What’s the ROI of Continuous Integration?

  - by Liggy

  Currently, our organization does not practice Continuous Integration. In order for us to get an CI server up and running, I will need to produce a document demonstrating the return on the investment. Aside from cost savings by finding and fixing bugs early, I'm curious about other benefits/savings that I could stick into this document.

  Read the article

• What's the workflow of Continuous Integration With Hudson?

  - by Satoru.Logic

  Hi, all. I am referred to Hudson today. I have heard about continuous integration before, but I have no idea what the heck is a ci-server. Hudson is really easy to install in Ubuntu and in several minutes I managed to set up an instance of it. But I don't quite understand the workflow of a ci-server, or how am I supposed to use it? Please tell me if you have experience about ci, thanks in advance.

  Read the article

• Use of GORM methods in Integration test

  - by canotto90

  I'm trying to use gorm find method on my domain class, inside of an Spock Integration Spec. My code: class myDomainClassSpec extends IntegrationSpec{ ... def 'my test'() { when: ... then: MyDomainClass.find { id == 1 } } ... } This fails, throwing: groovy.lang.MissingPropertyException: No such property: id for class: grails.gorm.DetachedCriteria If instead I code: MyDomainClass.findAll().find { id == 1 } it works. Any ideas??

  Read the article

• Paypal integration with php codeigniter...

  - by chandru_cp

  How to get started with Paypal integration with php codeigniter? Where should i start? Any suggestion?

  Read the article

• Continuous build integration tool for Rails and PHP?

  - by user261231

  Most of "multi-platform" CI tools I've seen are in the Java/.NET world, like TeamCity or Hudson. Is there a continuous integration tool that can handle both PHP and Rails projects?

  Read the article

< Previous Page | 9 10 11 12 13 14 15 16 17 18 19 20  | Next Page >