Search Results

Search found 61369 results on 2455 pages for 'data validation'.

Page 9/2455 | < Previous Page | 5 6 7 8 9 10 11 12 13 14 15 16  | Next Page >

  • How to Assure an Effective Data Model

    As a general rule in my opinion the effectiveness of a data model can be directly related to the accuracy and complexity of a project’s requirements. For example there is no need to work on very detailed data models when the details surrounding a specific data model have not been defined or even clarified. Developing data models when the clarity of project requirements is limited tends to introduce designed issues because the proper details to create an effective data model are not even known. One way to avoid this issue is to create data models that correspond to the complexity of the existing project requirements so that when requirements are updated then new data models can be created based any new discoveries regarding requirements on a fine grain level.  This allows for data models to be composed of general entities to be created initially when a project’s requirements are very vague and then the entities are refined as new and more substantial requirements are defined or redefined. This promotes communication amongst all stakeholders within a project as they go through the process of defining and finalizing project requirements.In addition, here are some general tips that can be applied to projects in regards to data modeling.Initially model all data generally and slowly reactor the data model as new requirements and business constraints are applied to a project.Ensure that data modelers have the proper tools and training they need to design a data model accurately.Create a common location for all project documents so that everyone will be able to review a project’s data models along with any other project documentation.All data models should follow a clear naming schema that tells readers the intended purpose for the data and how it is going to be applied within a project.

    Read the article

  • SQL SERVER – Why Do We Need Data Quality Services – Importance and Significance of Data Quality Services (DQS)

    - by pinaldave
    Databases are awesome.  I’m sure my readers know my opinion about this – I have made SQL Server my life’s work after all!  I love technology and all things computer-related.  Of course, even with my love for technology, I have to admit that it has its limits.  For example, it takes a human brain to notice that data has been input incorrectly.  Computer “brains” might be faster than humans, but human brains are still better at pattern recognition.  For example, a human brain will notice that “300” is a ridiculous age for a human to be, but to a computer it is just a number.  A human will also notice similarities between “P. Dave” and “Pinal Dave,” but this would stump most computers. In a database, these sorts of anomalies are incredibly important.  Databases are often used by multiple people who rely on this data to be true and accurate, so data quality is key.  That is why the improved SQL Server features Master Data Management talks about Data Quality Services.  This service has the ability to recognize and flag anomalies like out of range numbers and similarities between data.  This allows a human brain with its pattern recognition abilities to double-check and ensure that P. Dave is the same as Pinal Dave. A nice feature of Data Quality Services is that once you set the rules for the program to follow, it will not only keep your data organized in the future, but go to the past and “fix up” any data that has already been entered.  It also allows you do combine data from multiple places and it will apply these rules across the board, so that you don’t have any weird issues that crop up when trying to fit a round peg into a square hole. There are two parts of Data Quality Services that help you accomplish all these neat things.  The first part is DQL Server, which you can think of as the hardware component of the system.  It is installed on the side of (it needs to install separately after SQL Server is installed) SQL Server and runs quietly in the background, performing all its cleanup services. DQS Client is the user interface that you can interact with to set the rules and check over your data.  There are three main aspects of Client: knowledge base management, data quality projects and administration.  Knowledge base management is the part of the system that allows you to set the rules, or program the “knowledge base,” so that your database is clean and consistent. Data Quality projects are what run in the background and clean up the data that is already present.  The administration allows you to check out what DQS Client is doing, change rules, and generally oversee the entire process.  The whole process is user-friendly and a pleasure to use.  I highly recommend implementing Data Quality Services in your database. Here are few of my blog posts which are related to Data Quality Services and I encourage you to try this out. SQL SERVER – Installing Data Quality Services (DQS) on SQL Server 2012 SQL SERVER – Step by Step Guide to Beginning Data Quality Services in SQL Server 2012 – Introduction to DQS SQL SERVER – DQS Error – Cannot connect to server – A .NET Framework error occurred during execution of user-defined routine or aggregate “SetDataQualitySessions” – SetDataQualitySessionPhaseTwo SQL SERVER – Configuring Interactive Cleansing Suggestion Min Score for Suggestions in Data Quality Services (DQS) – Sensitivity of Suggestion SQL SERVER – Unable to DELETE Project in Data Quality Projects (DQS) Reference: Pinal Dave (http://blog.SQLAuthority.com) Filed under: PostADay, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, T SQL, Technology Tagged: Data Quality Services, DQS

    Read the article

  • Welcome Oracle Data Integration 12c: Simplified, Future-Ready Solutions with Extreme Performance

    - by Irem Radzik
    Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 The big day for the Oracle Data Integration team has finally arrived! It is my honor to introduce you to Oracle Data Integration 12c. Today we announced the general availability of 12c release for Oracle’s key data integration products: Oracle Data Integrator 12c and Oracle GoldenGate 12c. The new release delivers extreme performance, increase IT productivity, and simplify deployment, while helping IT organizations to keep pace with new data-oriented technology trends including cloud computing, big data analytics, real-time business intelligence. With the 12c release Oracle becomes the new leader in the data integration and replication technologies as no other vendor offers such a complete set of data integration capabilities for pervasive, continuous access to trusted data across Oracle platforms as well as third-party systems and applications. Oracle Data Integration 12c release addresses data-driven organizations’ critical and evolving data integration requirements under 3 key themes: Future-Ready Solutions Extreme Performance Fast Time-to-Value       There are many new features that support these key differentiators for Oracle Data Integrator 12c and for Oracle GoldenGate 12c. In this first 12c blog post, I will highlight only a few:·Future-Ready Solutions to Support Current and Emerging Initiatives: Oracle Data Integration offer robust and reliable solutions for key technology trends including cloud computing, big data analytics, real-time business intelligence and continuous data availability. Via the tight integration with Oracle’s database, middleware, and application offerings Oracle Data Integration will continue to support the new features and capabilities right away as these products evolve and provide advance features. E    Extreme Performance: Both GoldenGate and Data Integrator are known for their high performance. The new release widens the gap even further against competition. Oracle GoldenGate 12c’s Integrated Delivery feature enables higher throughput via a special application programming interface into Oracle Database. As mentioned in the press release, customers already report up to 5X higher performance compared to earlier versions of GoldenGate. Oracle Data Integrator 12c introduces parallelism that significantly increases its performance as well. Fast Time-to-Value via Higher IT Productivity and Simplified Solutions:  Oracle Data Integrator 12c’s new flow-based declarative UI brings superior developer productivity, ease of use, and ultimately fast time to market for end users.  It also gives the ability to seamlessly reuse mapping logic speeds development.Oracle GoldenGate 12c ‘s Integrated Delivery feature automatically optimally tunes the process, saving time while improving performance. This is just a quick glimpse into Oracle Data Integrator 12c and Oracle GoldenGate 12c. On November 12th we will reveal much more about the new release in our video webcast "Introducing 12c for Oracle Data Integration". Our customer and partner speakers, including SolarWorld, BT, Rittman Mead will join us in launching the new release. Please join us at this free event to learn more from our executives about the 12c release, hear our customers’ perspectives on the new features, and ask your questions to our experts in the live Q&A. Also, please continue to follow our blogs, tweets, and Facebook updates as we unveil more about the new features of the latest release. /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;}

    Read the article

  • OnSelectedIndexChange only fires on second click when using custom page validation script

    - by Kris P
    Okay.. this is hard to explain, so here it goes. I have an update panel which contains a number of controls. The update panel is triggered by the OnSelectedIndexChanged event of a dropdownlist called: ddlUSCitizenshipStatus. It works as expected when I selected a new item. However, if I leave ddlUSCitizenshipStatus with the default value, and then click "submit" button on the page, the requiredfieldvalidators say there is an error on ddlUSCitizenshipStatus (which it should, as I never selected a value). So I then choose a value, the error message goes away on ddlUSCitizenshipStatus, however the updatepanel does not refresh. I've debugged this locally and the OnSelectedIndexChanged event for ddlUSCitizenshipStatus does not fire. If I choose an item in the ddlUSCitizenshipStatus list a second time, the OnSelectedIndexChanged server event fires and the update panel refreshes and works as expected. The issue is, I have to select an item in ddlUSCitizenshipStatus twice, after failed validation, before the updatepanel it's sitting in updates. The submit button on the page looks like this: <asp:LinkButton ID="btnSubmitPage1" runat="server" CssClass="continueButton" OnClick="btnSubmitPage1_Click" CausesValidation="true" OnClientClick="javascript: return ValidatePage();" /> If I remove my custom OnClientClick script, making the submit button look like this: <asp:LinkButton ID="btnSubmitPage1" runat="server" CssClass="continueButton" OnClick="btnSubmitPage1_Click" CausesValidation="true" ValidationGroup="valGrpAdditionalInformation" /> The dropdownlist, update panel, and reguiredfieldvalidator all work as expected. However, I need to run that custom "ValidatePage()" script when the button is clicked. Below is what my ValidatePage script looks like. I've been troubleshooting this for more hours than I can count.... I hope someone is able to help me. Please let me know if you can figure out why ddlUSCitizenshipStatus doesn't update the updatepanel until the second click after a failed validation. function ValidatePage() { var blnDoPostBack = true; if (typeof(Page_ClientValidate) == 'function' ) { //Client side validation can occur, so lets do it. //Validate each validation group. for( var i = 0; i < Page_ValidationSummaries.length; i++ ) Page_ClientValidate( Page_ValidationSummaries[i].validationGroup.toString() ); //Validate every validation control on the page. for (var i = 0; i < Page_Validators.length; i++) ValidatorValidate(Page_Validators[i]); //Figure out which validation groups have errors, store a list of these validation groups in an array. var aryValGrpsWithErrors = []; for( var i = 0; i < Page_Validators.length; i++ ) { if( !Page_Validators[i].isvalid ) { //This particular validator has thrown an error. //Remeber to not do a postback, as we were able to catch this validation error client side. blnDoPostBack = false; //If we haven't already registered the validation group this erroring validation control is a part of, do so now. if( aryValGrpsWithErrors.indexOf( Page_Validators[i].validationGroup.toString() ) == -1 ) aryValGrpsWithErrors[aryValGrpsWithErrors.length++] = Page_Validators[i].validationGroup.toString(); } } //Now display every validation summary that has !isvalid validation controls in it. for( var i = 0; i < Page_ValidationSummaries.length; i++ ) { if( aryValGrpsWithErrors.indexOf( Page_ValidationSummaries[i].validationGroup.toString() ) != -1 ) { Page_ValidationSummaries[i].style.display = ""; document.getElementById( Page_ValidationSummaries[i].id.toString() + "Wrapper" ).style.display = ""; } else { //The current validation summary does not have any error messages in it, so make sure it's hidden. Page_ValidationSummaries[i].style.display = "none"; document.getElementById( Page_ValidationSummaries[i].id.toString() + "Wrapper" ).style.display = "none"; } } } return blnDoPostBack; }

    Read the article

  • Excel 2010 data validation warning (compatibility mode)

    - by Madmanguruman
    We have some legacy worksheets that were created in Excel 2003, which are used by LabVIEW-based test automation software. The current LabVIEW software can only handle the legacy .xls format, so we're forced to keep these worksheets as-is for the time being. We've migrated to Office 2010 and when working with these worksheets, I see this warning: "The following features in this workbook are not supported by earlier versions of Excel. These features may be lost or degraded when you save this workbook in the currently selected file format. Click Continue to save the workbook anyway. To keep all of your features, click Cancel and then save the file in one of the new file formats." "Significant loss of functionality" "One or more cells in this workbook contain data validation rules which refer to values on other worksheets. These data validation rules will not be saved." When I click 'Find', some cells that do indeed have validation rules are highlighted, but those rules are all on the same worksheet! We're using simple list-based validation, with some cells off to the side containing the valid values (for example, cell B4 has a List with Source "=$D$4:$E$4") This makes no sense to me whatsoever. One, the workbook was created in Excel 2003, so obviously we couldn't implement a feature that doesn't exist. Secondly, the modifications we're making don't involve changing the validation rules at all. Thirdly, the complaint that Excel is making is incorrect! All of the rules are on the same worksheet as the target. As if the story wasn't bizarre enough: I went ahead and saved the worksheet with Excel 2010. I then went to an old computer back in the lab and opened the document with Excel 2003. Guess what - the validations were untouched! My questions are: is this a legitimate bug in Excel 2010, or is this some exotic error in the legacy .xls worksheet that is confusing the heck out of Excel 2010? Has anyone else observed this issue working in compatibility mode?

    Read the article

  • Big Data – Final Wrap and What Next – Day 21 of 21

    - by Pinal Dave
    In yesterday’s blog post we explored various resources related to learning Big Data and in this blog post we will wrap up this 21 day series on Big Data. I have been exploring various terms and technology related to Big Data this entire month. It was indeed fun to write about Big Data in 21 days but the subject of Big Data is much bigger and larger than someone can cover it in 21 days. My first goal was to write about the basics and I think we have got that one covered pretty well. During this 21 days I have received many questions and answers related to Big Data. I have covered a few of the questions in this series and a few more I will be covering in the next coming months. Now after understanding Big Data basics. I am personally going to do a list of the things next. I thought I will share the same with you as this will give you a good idea how to continue the journey of the Big Data. Build a schedule to read various Apache documentations Watch all Pluralsight Courses Explore HortonWorks Sandbox Start building presentation about Big Data – this is a great way to learn something new Present in User Groups Meetings on Big Data Topics Write more blog posts about Big Data I am going to continue learning about Big Data – I want you to continue learning Big Data. Please leave a comment how you are going to continue learning about Big Data. I will publish all the informative comments on this blog with due credit. I want to end this series with the infographic by UMUC. Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: Big Data, PostADay, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, T SQL

    Read the article

  • Partner Webcast - Focus on Oracle Data Profiling and Data Quality 11g

    - by lukasz.romaszewski(at)oracle.com
    Normal 0 false false false EN-US X-NONE X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin-top:0cm; mso-para-margin-right:0cm; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0cm; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi; mso-ansi-language:RO;} Partner Webcast Focus on Oracle Data Profiling and Data Quality 11g February 24th, 12am  CET   Oracle offers an integrated suite Data Quality software architected to discover and correct today's data quality problems and establish a platform prepared for tomorrow's yet unknown data challenges. Oracle Data Profiling provides data investigation, discovery, and profiling in support of quality, migration, integration, stewardship, and governance initiatives. It includes a broad range of features that expand upon basic profiling, including automated monitoring, business-rule validation, and trend analysis. Oracle Data Quality for Data Integrator provides cleansing, standardization, matching, address validation, location enrichment, and linking functions for global customer data and operational business data. It ensures that data adheres to established standards that are adaptable to fit each organization's specific needs.  Both single - and double - byte data are processed in local languages to provide a unique and centralized view of customers, products and services.   During this in-person briefing, Data Integration Solution Specialists will be providing a technical overview and a walkthrough.   Agenda ·         Oracle Data Integration Strategy overview ·         A focus on Oracle Data Profiling and Oracle Data Quality for Data Integrator: o   Oracle Data Profiling o   Oracle Data Quality for Data Integrator o   Live demoo   Q&A Delivery Format  This FREE online LIVE eSeminar will be delivered over the Web and Conference Call. Registrations   received less than 24hours  prior to start time may not receive confirmation to attend. To register , click here. For any questions please contact [email protected]

    Read the article

  • Registration form validation not validating

    - by jgray
    I am a noob when it comes to web development. I am trying to validate a registration form and to me it looks right but it will not validate.. This is what i have so far and i am validating through a repository or database. Any help would be greatly appreciated. thanks <?php session_start(); $title = "User Registration"; $keywords = "Name, contact, phone, e-mail, registration"; $description = "user registration becoming a member."; require "partials/_html_header.php"; //require "partials/_header.php"; require "partials/_menu.php"; require "DataRepository.php"; // if all validation passed save user $db = new DataRepository(); // form validation goes here $first_nameErr = $emailErr = $passwordErr = $passwordConfirmErr = ""; $first_name = $last_name = $email = $password = $passwordConfirm = ""; if(isset($_POST['submit'])) { $valid = TRUE; // check if all fields are valid { if ($_SERVER["REQUEST_METHOD"] == "POST") { if (empty($_POST["first_name"])) {$first_nameErr = "Name is required";} else { // $first_name = test_input($_POST["first_name"]); // check if name only contains letters and whitespace if (!preg_match("/^[a-zA-Z ]*$/",$first_name)) { $first_nameErr = "Only letters and white space allowed"; } } if (empty($_POST["email"])) {$emailErr = "Email is required";} else { // $email = test_input($_POST["email"]); // check if e-mail address syntax is valid if (!preg_match("/([\w\-]+\@[\w\-]+\.[\w\-]+)/",$email)) { $emailErr = "Invalid email format"; } } if (!preg_match("/(......)/",$password)) { $passwordErr = "Subject must contain THREE or more characters!"; } if ($_POST['password']!= $_POST['passwordConfirm']) { echo("Oops! Password did not match! Try again. "); } function test_input($data) { $data = trim($data); $data = stripslashes($data); $data = htmlspecialchars($data); return $data; } } } if(!$db->isEmailUnique($_POST['email'])) { $valid = FALSE; //display errors in the correct places } // if still valid save the user if($valid) { $new_user = array( 'first_name' => $_POST['first_name'], 'last_name' => $_POST['last_name'], 'email' => $_POST['email'], 'password' => $_POST['password'] ); $results = $db->saveUser($new_user); if($results == TRUE) { header("Location: login.php"); } else { echo "WTF!"; exit; } } } ?> <head> <style> .error {color: #FF0000;} </style> </head> <h1 class="center"> World Wide Web Creations' User Registration </h1> <p><span class="error"></span><p> <form method="POST" action="<?php echo htmlspecialchars($_SERVER["PHP_SELF"]);?>" onsubmit="return validate_form()" > First Name: <input type="text" name="first_name" id="first_name" value="<?php echo $first_name;?>" /> <span class="error"> <?php echo $first_nameErr;?></span> <br /> <br /> Last Name(Optional): <input type="text" name="last_name" id="last_name" value="<?php echo $last_name;?>" /> <br /> <br /> E-mail: <input type="email" name="email" id="email" value="<?php echo $email;?>" /> <span class="error"> <?php echo $emailErr;?></span> <br /> <br /> Password: <input type="password" name="password" id="password" value="" /> <span class="error"> <?php echo $passwordErr;?></span> <br /> <br /> Confirmation Password: <input type="password" name="passwordConfirm" id="passwordConfirm" value="" /> <span class="error"> <?php echo $passwordConfirmErr;?></span> <br /> <br /> <br /> <br /> <input type="submit" name="submit" id="submit" value="Submit Data" /> <input type="reset" name="reset" id="reset" value="Reset Form" /> </form> </body> </html> <?php require "partials/_footer.php"; require "partials/_html_footer.php"; ?> class DataRepository { // version number private $version = "1.0.3"; // turn on and off debugging private static $debug = FALSE; // flag to (re)initialize db on each call private static $initialize_db = FALSE; // insert test data on initialization private static $load_default_data = TRUE; const DATAFILE = "203data.txt"; private $data = NULL; private $errors = array(); private $user_fields = array( 'id' => array('required' => 0), 'created_at' => array('required' => 0), 'updated_at' => array('required' => 0), 'first_name' => array('required' => 1), 'last_name' => array('required' => 0), 'email' => array('required' => 1), 'password' => array('required' => 1), 'level' => array('required' => 0, 'default' => 2), ); private $post_fields = array( 'id' => array('required' => 0), 'created_at' => array('required' => 0), 'updated_at' => array('required' => 0), 'user_id' => array('required' => 1), 'title' => array('required' => 1), 'message' => array('required' => 1), 'private' => array('required' => 0, 'default' => 0), ); private $default_user = array( 'id' => 1, 'created_at' => '2013-01-01 00:00:00', 'updated_at' => '2013-01-01 00:00:00', 'first_name' => 'Admin Joe', 'last_name' => 'Tester', 'email' => '[email protected]', 'password' => 'a94a8fe5ccb19ba61c4c0873d391e987982fbbd3', 'level' => 1, ); private $default_post = array( 'id' => 1, 'created_at' => '2013-01-01 00:00:00', 'updated_at' => '2013-01-01 00:00:00', 'user_id' => 1, 'title' => 'My First Post', 'message' => 'This is the message of the first post.', 'private' => 0, ); // constructor will load existing data into memory // if it does not exist it will create it and initialize if desired public function __construct() { // check if need to reset if(DataRepository::$initialize_db AND file_exists(DataRepository::DATAFILE)) { unlink(DataRepository::DATAFILE); } // if file doesn't exist, create the initial datafile if(!file_exists(DataRepository::DATAFILE)) { $this->log("Data file does not exist. Attempting to create it... (".__FUNCTION__.":".__LINE__.")"); // create initial file $this->data = array( 'users' => array( ), 'posts' => array() ); // load default data if needed if(DataRepository::$load_default_data) { $this->data['users'][1] = $this->default_user; $this->data['posts'][1] = $this->default_post; } $this->writeTheData(); } // load the data into memory for use $this->loadTheData(); } private function showErrors($break = TRUE, $type = NULL) { if(count($this->errors) > 0) { echo "<div style=\"color:red;font-weight: bold;font-size: 1.3em\":<h3>$type Errors</h3><ol>"; foreach($this->errors AS $error) { echo "<li>$error</li>"; } echo "</ol></div>"; if($break) { "</br></br></br>Exiting because of errors!"; exit; } } } private function writeTheData() { $this->log("Attempting to write the datafile: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); file_put_contents(DataRepository::DATAFILE, json_encode($this->data)); $this->log("Datafile written: ".DataRepository::DATAFILE." (line: ".__LINE__.")"); } private function loadTheData() { $this->log("Attempting to load the datafile: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); $this->data = json_decode(file_get_contents(DataRepository::DATAFILE), true); $this->log("Datafile loaded: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")", $this->data); } private function validateFields(&$info, $fields, $pre_errors = NULL) { // merge in any pre_errors if($pre_errors != NULL) { $this->errors = array_merge($this->errors, $pre_errors); } // check all required fields foreach($fields AS $field => $reqs) { if(isset($reqs['required']) AND $reqs['required'] == 1) { if(!isset($info[$field]) OR strlen($info[$field]) == 0) { $this->errors[] = "$field is a REQUIRED field"; } } // set any default values if not present if(isset($reqs['default']) AND (!isset($info[$field]) OR $info[$field] == "")) { $info[$field] = $reqs['default']; } } $this->showErrors(); if(count($this->errors) == 0) { return TRUE; } else { return FALSE; } } private function validateUser(&$user_info) { // check if the email is already in use $this->log("About to check pre_errors: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")", $user_info); $pre_errors = NULL; if(isset($user_info['email'])) { if(!$this->isEmailUnique($user_info['email'])) { $pre_errors = array('The email: '.$user_info['email'].' is already used in our system'); } } $this->log("After pre_error check: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")", $pre_errors); return $this->validateFields($user_info, $this->user_fields, $pre_errors); } private function validatePost(&$post_info) { // check if the user_id in the post actually exists $this->log("About to check pre_errors: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")", $post_info); $pre_errors = NULL; if(isset($post_info['user_id'])) { if(!isset($this->data['users'][$post_info['user_id']])) { $pre_errors = array('The posts must belong to a valid user. (User '.$post_info['user_id'].' does not exist in the data'); } } $this->log("After pre_error check: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")", $pre_errors); return $this->validateFields($post_info, $this->post_fields, $pre_errors); } private function log($message, $data = NULL) { $style = "background-color: #F8F8F8; border: 1px solid #DDDDDD; border-radius: 3px; font-size: 13px; line-height: 19px; overflow: auto; padding: 6px 10px;"; if(DataRepository::$debug) { if($data != NULL) { $dump = "<div style=\"$style\"><pre>".json_encode($data, JSON_PRETTY_PRINT)."</pre></div>"; } else { $dump = NULL; } echo "<code><b>Debug:</b> $message</code>$dump<br />"; } } public function saveUser($user_info) { $this->log("Entering saveUser: (".__FUNCTION__.":".__LINE__.")", $user_info); $mydata = array(); $update = FALSE; // check for existing data if(isset($user_info['id']) AND $this->data['users'][$user_info['id']]) { $mydata = $this->data['users'][$user_info['id']]; $this->log("Loaded prior user: ".print_r($mydata, TRUE)." (".__FUNCTION__.":".__LINE__.")"); } // copy over existing values $this->log("Before copying over existing values: (".__FUNCTION__.":".__LINE__.")", $mydata); foreach($user_info AS $k => $v) { $mydata[$k] = $user_info[$k]; } $this->log("After copying over existing values: (".__FUNCTION__.":".__LINE__.")", $mydata); // check required fields if($this->validateUser($mydata)) { // hash password if new if(isset($mydata['password'])) { $mydata['password'] = sha1($mydata['password']); } // if no id, add the next available one if(!isset($mydata['id']) OR (int)$mydata['id'] < 1) { $this->log("No id set: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); if(count($this->data['users']) == 0) { $mydata['id'] = 1; $this->log("Setting id to 1: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); } else { $mydata['id'] = max(array_keys($this->data['users']))+1; $this->log("Found max id and added 1 [".$mydata['id']."]: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); } } // set created date if null if(!isset($mydata['created_at'])) { $mydata['created_at'] = date ("Y-m-d H:i:s", time()); } // update modified time $mydata['modified_at'] = date ("Y-m-d H:i:s", time()); // copy into data and save $this->log("Before data save: (".__FUNCTION__.":".__LINE__.")", $this->data); $this->data['users'][$mydata['id']] = $mydata; $this->writeTheData(); } return TRUE; } public function getUserById($id) { if(isset($this->data['users'][$id])) { return $this->data['users'][$id]; } else { return array(); } } public function isEmailUnique($email) { // find the user that has the right username/password foreach($this->data['users'] AS $k => $v) { $this->log("Checking unique email: {$v['email']} == $email (".__FUNCTION__.":".__LINE__.")", NULL); if($v['email'] == $email) { $this->log("FOUND NOT unique email: {$v['email']} == $email (".__FUNCTION__.":".__LINE__.")", NULL); return FALSE; break; } } $this->log("Email IS unique: $email (".__FUNCTION__.":".__LINE__.")", NULL); return TRUE; } public function login($username, $password) { // hash password for validation $password = sha1($password); $this->log("Attempting to login with $username / $password: (".__FUNCTION__.":".__LINE__.")", NULL); $user = NULL; // find the user that has the right username/password foreach($this->data['users'] AS $k => $v) { if($v['email'] == $username AND $v['password'] == $password) { $user = $v; break; } } $this->log("Exiting login: (".__FUNCTION__.":".__LINE__.")", $user); return $user; } public function savePost($post_info) { $this->log("Entering savePost: (".__FUNCTION__.":".__LINE__.")", $post_info); $mydata = array(); // check for existing data if(isset($post_info['id']) AND $this->data['posts'][$post_info['id']]) { $mydata = $this->data['posts'][$post_info['id']]; $this->log("Loaded prior posts: ".print_r($mydata, TRUE)." (".__FUNCTION__.":".__LINE__.")"); } $this->log("Before copying over existing values: (".__FUNCTION__.":".__LINE__.")", $mydata); foreach($post_info AS $k => $v) { $mydata[$k] = $post_info[$k]; } $this->log("After copying over existing values: (".__FUNCTION__.":".__LINE__.")", $mydata); // check required fields if($this->validatePost($mydata)) { // if no id, add the next available one if(!isset($mydata['id']) OR (int)$mydata['id'] < 1) { $this->log("No id set: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); if(count($this->data['posts']) == 0) { $mydata['id'] = 1; $this->log("Setting id to 1: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); } else { $mydata['id'] = max(array_keys($this->data['posts']))+1; $this->log("Found max id and added 1 [".$mydata['id']."]: ".DataRepository::DATAFILE." (".__FUNCTION__.":".__LINE__.")"); } } // set created date if null if(!isset($mydata['created_at'])) { $mydata['created_at'] = date ("Y-m-d H:i:s", time()); } // update modified time $mydata['modified_at'] = date ("Y-m-d H:i:s", time()); // copy into data and save $this->data['posts'][$mydata['id']] = $mydata; $this->log("Before data save: (".__FUNCTION__.":".__LINE__.")", $this->data); $this->writeTheData(); } return TRUE; } public function getAllPosts() { return $this->loadPostsUsers($this->data['posts']); } public function loadPostsUsers($posts) { foreach($posts AS $id => $post) { $posts[$id]['user'] = $this->getUserById($post['user_id']); } return $posts; } public function dump($line_number, $temp = 'NO') { // if(DataRepository::$debug) { if($temp == 'NO') { $temp = $this->data; } echo "<pre>Dumping from line: $line_number\n"; echo json_encode($temp, JSON_PRETTY_PRINT); echo "</pre>"; } } } /* * Change Log * * 1.0.0 * - first version * 1.0.1 * - Added isEmailUnique() function for form validation and precheck on user save * 1.0.2 * - Fixed getAllPosts() to include the post's user info * - Added loadPostsUsers() to load one or more posts with their user info * 1.0.3 * - Added autoload to always add admin Joe. */

    Read the article

  • Performing both client side and server side validation using jQuery and CodeIgniter

    - by Vasu
    What is the right way of doing both client side and server side validation using jQuery and CodeIgniter? I am using the jQuery form plugin for form submit. I would like to use jQuery validation plugin (http://docs.jquery.com/Plugins/Validation) for client side validation and CodeIgniter form validation on the server side. However the two don't seem to gel together (or I am unable to get my head around it). Can someone help please? Whether its a client side validation or server side validation, the user should see consistent UI displaying error messages next to the input fields.

    Read the article

  • Big Data – Buzz Words: What is HDFS – Day 8 of 21

    - by Pinal Dave
    In yesterday’s blog post we learned what is MapReduce. In this article we will take a quick look at one of the four most important buzz words which goes around Big Data – HDFS. What is HDFS ? HDFS stands for Hadoop Distributed File System and it is a primary storage system used by Hadoop. It provides high performance access to data across Hadoop clusters. It is usually deployed on low-cost commodity hardware. In commodity hardware deployment server failures are very common. Due to the same reason HDFS is built to have high fault tolerance. The data transfer rate between compute nodes in HDFS is very high, which leads to reduced risk of failure. HDFS creates smaller pieces of the big data and distributes it on different nodes. It also copies each smaller piece to multiple times on different nodes. Hence when any node with the data crashes the system is automatically able to use the data from a different node and continue the process. This is the key feature of the HDFS system. Architecture of HDFS The architecture of the HDFS is master/slave architecture. An HDFS cluster always consists of single NameNode. This single NameNode is a master server and it manages the file system as well regulates access to various files. In additional to NameNode there are multiple DataNodes. There is always one DataNode for each data server. In HDFS a big file is split into one or more blocks and those blocks are stored in a set of DataNodes. The primary task of the NameNode is to open, close or rename files and directory and regulate access to the file system, whereas the primary task of the DataNode is read and write to the file systems. DataNode is also responsible for the creation, deletion or replication of the data based on the instruction from NameNode. In reality, NameNode and DataNode are software designed to run on commodity machine build in Java language. Visual Representation of HDFS Architecture Let us understand how HDFS works with the help of the diagram. Client APP or HDFS Client connects to NameSpace as well as DataNode. Client App access to the DataNode is regulated by NameSpace Node. NameSpace Node allows Client App to connect to the DataNode based by allowing the connection to the DataNode directly. A big data file is divided into multiple data blocks (let us assume that those data chunks are A,B,C and D. Client App will later on write data blocks directly to the DataNode. Client App does not have to directly write to all the node. It just has to write to any one of the node and NameNode will decide on which other DataNode it will have to replicate the data. In our example Client App directly writes to DataNode 1 and detained 3. However, data chunks are automatically replicated to other nodes. All the information like in which DataNode which data block is placed is written back to NameNode. High Availability During Disaster Now as multiple DataNode have same data blocks in the case of any DataNode which faces the disaster, the entire process will continue as other DataNode will assume the role to serve the specific data block which was on the failed node. This system provides very high tolerance to disaster and provides high availability. If you notice there is only single NameNode in our architecture. If that node fails our entire Hadoop Application will stop performing as it is a single node where we store all the metadata. As this node is very critical, it is usually replicated on another clustered as well as on another data rack. Though, that replicated node is not operational in architecture, it has all the necessary data to perform the task of the NameNode in the case of the NameNode fails. The entire Hadoop architecture is built to function smoothly even there are node failures or hardware malfunction. It is built on the simple concept that data is so big it is impossible to have come up with a single piece of the hardware which can manage it properly. We need lots of commodity (cheap) hardware to manage our big data and hardware failure is part of the commodity servers. To reduce the impact of hardware failure Hadoop architecture is built to overcome the limitation of the non-functioning hardware. Tomorrow In tomorrow’s blog post we will discuss the importance of the relational database in Big Data. Reference: Pinal Dave (http://blog.sqlauthority.com) Filed under: Big Data, PostADay, SQL, SQL Authority, SQL Query, SQL Server, SQL Tips and Tricks, T SQL

    Read the article

  • Oracle Data Integration 12c: Perspectives of Industry Experts, Customers and Partners

    - by Irem Radzik
    Normal 0 false false false EN-US X-NONE X-NONE MicrosoftInternetExplorer4 As you may have seen from our recent blog posts on Oracle Data Integrator 12c and Oracle GoldenGate 12c, we are very excited to share with you the great new features the 12c release brings to Oracle’s data integration solutions. And, fortunately we are not alone in this sentiment. Since the press announcement October 17th, which incorporates our customers' and experts' testimonials, we have seen positive comments in leading technology publications and social media as well. Here are some examples: In CIO and PCWorld you can find Joab Jackson’s article, Oracle Data Integrator 12c ready for real-time analysis, where wrote about the tight integration between Oracle Data Integrator and Oracle GoldenGate . He noted “Heeding the call from enterprise customers who clamor for more immediacy in their data-driven reports, Oracle has updated its data-integration software portfolio so that it can more rapidly deliver data to data warehouses and analysis applications.” Integration Developer News’ Vance McCarthy wrote the article Oracle Ships ‘Future Proofs’ Integration Tools for Traditional, Cloud, Big Data, Real-Time Projects and mentioned that “Oracle Data Integrator 12c and Oracle GoldenGate 12c sport a wide range of improvements to let devs more easily deliver data integration for cloud, analytics, big data and other new projects that leverage multiple datasets for business.“ InformationWeek’s Doug Henschen gave a great overview to several key features including the new flow-based UI in Oracle Data Integrator. Doug said “Oracle Data Integrator 12c introduces a complete makeover of the job-building experience, while real-time oriented GoldenGate 12c introduces performance gains “. In Database Trends and Applications’ article Oracle Strengthens Data Integration with Release of Oracle Data Integrator 12c and Oracle GoldenGate 12c highlighted the productivity aspect of the new solution with his remarks: “tight integration between Oracle Data Integrator 12c and Oracle GoldenGate 12c enables developers to leverage Oracle GoldenGate’s low overhead, real-time change data capture completely within the Oracle Data Integrator Studio without additional training”. We are also thrilled about what our customers and partners have to say about our products and the new release. And we are equally excited to share those perspectives with you in our upcoming launch video webcast on November 12th. SolarWorld Industries America’s Senior Database Manager, Russ Toyama will join our executives in our studio in Redwood Shores to discuss GoldenGate’s core benefits and the new release, while Surren Partharb, CTO of Strategic Technology Services for BT, and Mark Rittman, CTO of Rittman Mead, will provide their comments via the interviews conducted in the UK. This interactive panel discussion in the video webcast will unveil the new release with the expertise of our development executives and the great insight from our customers and partners. In addition, our product experts will be available online to answer chat questions. This is really a great opportunity to learn how Oracle's data integration offering has changed the integration and replication technology space with the new release, and established itself as the new leader. If you have not registered for this free event yet, you can do so via this link. We will run the live event at 8am PT/4pm GMT, followed by a replay of the event with live chat for Q&A  at 10am PT/6pm GMT. The replay will be available on-demand for those who register but cannot attend either session on November 12th. /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-qformat:yes; mso-style-parent:""; mso-padding-alt:0in 5.4pt 0in 5.4pt; mso-para-margin-top:0in; mso-para-margin-right:0in; mso-para-margin-bottom:10.0pt; mso-para-margin-left:0in; line-height:115%; mso-pagination:widow-orphan; font-size:11.0pt; font-family:"Times New Roman","serif"; mso-fareast-font-family:"Times New Roman";}

    Read the article

  • For an ORM supporting data validation, should constraints be enforced in the database as well?

    - by Ramnique Singh
    I have always applied constraints at the database level in addition to my (ActiveRecord) models. But I've been wondering if this is really required? A little background I recently had to unit test a basic automated timestamp generation method for a model. Normally, the test would create an instance of the model and save it without validation. But there are other required fields that aren't nullable at the in the table definition, meaning I cant save the instance even if I skip the ActiveRecord validation. So I'm thinking if I should remove such constraints from the db itself, and let the ORM handle them? Possible advantages if I skip constraints in db, imo - Can modify a validation rule in the model, without having to migrate the database. Can skip validation in testing. Possible disadvantage? If its possible that ORM validation fails or is bypassed, howsoever, the database does not check for constraints. What do you think? EDIT In this case, I'm using the Yii Framework, which generates the model from the database, hence database rules are generated also (though I could always write them post-generation myself too).

    Read the article

  • New Big Data Appliance Security Features

    - by mgubar
    The Oracle Big Data Appliance (BDA) is an engineered system for big data processing.  It greatly simplifies the deployment of an optimized Hadoop Cluster – whether that cluster is used for batch or real-time processing.  The vast majority of BDA customers are integrating the appliance with their Oracle Databases and they have certain expectations – especially around security.  Oracle Database customers have benefited from a rich set of security features:  encryption, redaction, data masking, database firewall, label based access control – and much, much more.  They want similar capabilities with their Hadoop cluster.    Unfortunately, Hadoop wasn’t developed with security in mind.  By default, a Hadoop cluster is insecure – the antithesis of an Oracle Database.  Some critical security features have been implemented – but even those capabilities are arduous to setup and configure.  Oracle believes that a key element of an optimized appliance is that its data should be secure.  Therefore, by default the BDA delivers the “AAA of security”: authentication, authorization and auditing. Security Starts at Authentication A successful security strategy is predicated on strong authentication – for both users and software services.  Consider the default configuration for a newly installed Oracle Database; it’s been a long time since you had a legitimate chance at accessing the database using the credentials “system/manager” or “scott/tiger”.  The default Oracle Database policy is to lock accounts thereby restricting access; administrators must consciously grant access to users. Default Authentication in Hadoop By default, a Hadoop cluster fails the authentication test. For example, it is easy for a malicious user to masquerade as any other user on the system.  Consider the following scenario that illustrates how a user can access any data on a Hadoop cluster by masquerading as a more privileged user.  In our scenario, the Hadoop cluster contains sensitive salary information in the file /user/hrdata/salaries.txt.  When logged in as the hr user, you can see the following files.  Notice, we’re using the Hadoop command line utilities for accessing the data: $ hadoop fs -ls /user/hrdataFound 1 items-rw-r--r--   1 oracle supergroup         70 2013-10-31 10:38 /user/hrdata/salaries.txt$ hadoop fs -cat /user/hrdata/salaries.txtTom Brady,11000000Tom Hanks,5000000Bob Smith,250000Oprah,300000000 User DrEvil has access to the cluster – and can see that there is an interesting folder called “hrdata”.  $ hadoop fs -ls /user Found 1 items drwx------   - hr supergroup          0 2013-10-31 10:38 /user/hrdata However, DrEvil cannot view the contents of the folder due to lack of access privileges: $ hadoop fs -ls /user/hrdata ls: Permission denied: user=drevil, access=READ_EXECUTE, inode="/user/hrdata":oracle:supergroup:drwx------ Accessing this data will not be a problem for DrEvil. He knows that the hr user owns the data by looking at the folder’s ACLs. To overcome this challenge, he will simply masquerade as the hr user. On his local machine, he adds the hr user, assigns that user a password, and then accesses the data on the Hadoop cluster: $ sudo useradd hr $ sudo passwd $ su hr $ hadoop fs -cat /user/hrdata/salaries.txt Tom Brady,11000000 Tom Hanks,5000000 Bob Smith,250000 Oprah,300000000 Hadoop has not authenticated the user; it trusts that the identity that has been presented is indeed the hr user. Therefore, sensitive data has been easily compromised. Clearly, the default security policy is inappropriate and dangerous to many organizations storing critical data in HDFS. Big Data Appliance Provides Secure Authentication The BDA provides secure authentication to the Hadoop cluster by default – preventing the type of masquerading described above. It accomplishes this thru Kerberos integration. Figure 1: Kerberos Integration The Key Distribution Center (KDC) is a server that has two components: an authentication server and a ticket granting service. The authentication server validates the identity of the user and service. Once authenticated, a client must request a ticket from the ticket granting service – allowing it to access the BDA’s NameNode, JobTracker, etc. At installation, you simply point the BDA to an external KDC or automatically install a highly available KDC on the BDA itself. Kerberos will then provide strong authentication for not just the end user – but also for important Hadoop services running on the appliance. You can now guarantee that users are who they claim to be – and rogue services (like fake data nodes) are not added to the system. It is common for organizations to want to leverage existing LDAP servers for common user and group management. Kerberos integrates with LDAP servers – allowing the principals and encryption keys to be stored in the common repository. This simplifies the deployment and administration of the secure environment. Authorize Access to Sensitive Data Kerberos-based authentication ensures secure access to the system and the establishment of a trusted identity – a prerequisite for any authorization scheme. Once this identity is established, you need to authorize access to the data. HDFS will authorize access to files using ACLs with the authorization specification applied using classic Linux-style commands like chmod and chown (e.g. hadoop fs -chown oracle:oracle /user/hrdata changes the ownership of the /user/hrdata folder to oracle). Authorization is applied at the user or group level – utilizing group membership found in the Linux environment (i.e. /etc/group) or in the LDAP server. For SQL-based data stores – like Hive and Impala – finer grained access control is required. Access to databases, tables, columns, etc. must be controlled. And, you want to leverage roles to facilitate administration. Apache Sentry is a new project that delivers fine grained access control; both Cloudera and Oracle are the project’s founding members. Sentry satisfies the following three authorization requirements: Secure Authorization:  the ability to control access to data and/or privileges on data for authenticated users. Fine-Grained Authorization:  the ability to give users access to a subset of the data (e.g. column) in a database Role-Based Authorization:  the ability to create/apply template-based privileges based on functional roles. With Sentry, “all”, “select” or “insert” privileges are granted to an object. The descendants of that object automatically inherit that privilege. A collection of privileges across many objects may be aggregated into a role – and users/groups are then assigned that role. This leads to simplified administration of security across the system. Figure 2: Object Hierarchy – granting a privilege on the database object will be inherited by its tables and views. Sentry is currently used by both Hive and Impala – but it is a framework that other data sources can leverage when offering fine-grained authorization. For example, one can expect Sentry to deliver authorization capabilities to Cloudera Search in the near future. Audit Hadoop Cluster Activity Auditing is a critical component to a secure system and is oftentimes required for SOX, PCI and other regulations. The BDA integrates with Oracle Audit Vault and Database Firewall – tracking different types of activity taking place on the cluster: Figure 3: Monitored Hadoop services. At the lowest level, every operation that accesses data in HDFS is captured. The HDFS audit log identifies the user who accessed the file, the time that file was accessed, the type of access (read, write, delete, list, etc.) and whether or not that file access was successful. The other auditing features include: MapReduce:  correlate the MapReduce job that accessed the file Oozie:  describes who ran what as part of a workflow Hive:  captures changes were made to the Hive metadata The audit data is captured in the Audit Vault Server – which integrates audit activity from a variety of sources, adding databases (Oracle, DB2, SQL Server) and operating systems to activity from the BDA. Figure 4: Consolidated audit data across the enterprise.  Once the data is in the Audit Vault server, you can leverage a rich set of prebuilt and custom reports to monitor all the activity in the enterprise. In addition, alerts may be defined to trigger violations of audit policies. Conclusion Security cannot be considered an afterthought in big data deployments. Across most organizations, Hadoop is managing sensitive data that must be protected; it is not simply crunching publicly available information used for search applications. The BDA provides a strong security foundation – ensuring users are only allowed to view authorized data and that data access is audited in a consolidated framework.

    Read the article

  • How to present a stable data model in a public API that allows internal data structures to be changed without breaking the public view of the data?

    - by Max Palmer
    I am in the process of developing an application that allows users to write C# scripts. These scripts allow users to call selected methods and to access and manipulate data in a document. This works well, however, in the development version, scripts access the document's (internal) data structures directly. This means that if we were to change the internal data model/structure, there is a good chance that someone's script will no longer compile. We obviously want to prevent this breaking change from happening, but still want to allow the user to write sensible C# code (whilst not restricting how we develop our internal data model as a result). We therefore need to decouple our scripting API and its data structures from our internal methods and data structures. We've a few ideas as to how we might allow the user to access a what is effectively a stable public version of the document's internal data*, but I wanted to throw the question out there to someone who might have some real experience of this problem. NB our internal document's data structure is quite complex and it could be quite difficult to wrap. We know we want to expose as little as possible in our public API, especially as once it's out there, it's out there for good. Can anyone help? How do scripting languages / APIs decouple their public API and data structures from their internal data structures? Is there no real alternative to having to write a complex interaction layer? If we need to do this, what's a good approach or pattern for wrapping complex data structures that include nested objects, including collections? I've looked at the API facade pattern, which looks like it's trying to address these kinds of issues, but are there alternatives? *One idea is to build a data facade that is kept stable across versions of our application. The facade exposes a set of facade data objects that are used in the script code. These maintain backwards compatibility and wrap access to our internal document's data model.

    Read the article

  • Play Framework custom validation errors with multiple String parameters

    - by Mark
    I'm trying to set a custom validation error with multiple params in Play!, but it seems like my validation parameters are not rendered correctly. I have defined in messages: validation.customerror=This is first param "%s", and this is the second "%s" The in my code I execute: validation.addError("","validation.customerror", "FIRST", "SECOND"); And I get: This is first param "", and this is the second "FIRST" Thoughts?

    Read the article

  • How to perform duplicate key validation using entlib (or DataAnnotations), MVC, and Repository pattern

    - by olivehour
    I have a set of ASP.NET 4 projects that culminate in an MVC (3 RC2) app. The solution uses Unity and EntLib Validation for cross-cutting dependency injection and validation. Both are working great for injecting repository and service layer implementations. However, I can't figure out how to do duplicate key validation. For example, when a user registers, we want to make sure they don't pick a UserID that someone else is already using. For this type of validation, the validating object must have a repository reference... or some other way to get an IQueryable / IEnumerable reference to check against other rows already in the DB. What I have is a UserMetadata class that has all of the property setters and getters for a user, along with all of the appropriate DataAnnotations and EntLib Validation attributes. There is also a UserEntity class implemented using EF4 POCO Entity Generator templates. The UserEntity depends on UserMetadata, because it has a MetadataTypeAttribute. I also have a UserViewModel class that has the same exact MetadataType attribute. This way, I can apply the same validation rules, via attributes, to both the entity and viewmodel. There are no concrete references to the Repository classes whatsoever. All repositories are injected using Unity. There is also a service layer that gets dependency injection. In the MVC project, service layer implementation classes are injected into the Controller classes (the controller classes only contain service layer interface references). Unity then injects the Repository implementations into the service layer classes (service classes also only contain interface references). I've experimented with the DataAnnotations CustomValidationAttribute in the metadata class. The problem with this is the validation method must be static, and the method cannot instantiate a repository implementation directly. My repository interface is IRepository, and I have only one single repository implementation class defined as EntityRepository for all domain objects. To instantiate a repository explicitly I would need to say new EntityRepository(), which would result in a circular dependency graph: UserMetadata [depends on] DuplicateUserIDValidator [depends on] UserEntity [depends on] UserMetadata. I've also tried creating a custom EntLib Validator along with a custom validation attribute. Here I don't have the same problem with a static method. I think I could get this to work if I could just figure out how to make Unity inject my EntityRepository into the validator class... which I can't. Right now, all of the validation code is in my Metadata class library, since that's where the custom validation attribute would go. Any ideas on how to perform validations that need to check against the current repository state? Can Unity be used to inject a dependency into a lower-layer class library?

    Read the article

  • Effective Data Validation

    - by John Conde
    What's an effective way to handle data validation, say, from a form submission? Originally I had a bunch of if statements that checked each value and collected invalid values in an array for later retrieval (and listing). // Store errors here $errors = array(); // Hypothetical check if a string is alphanumeric if (!preg_match('/^[a-z\d]+$/i', $fieldvalue)) { $errors[$fieldname] = 'Please only use letters and numbers for your street address'; } // etc... What I did next was create a class that handles various data validation scenarios and store the results in an internal array. After data validation was complete I would check to see if any errors occurred and handle accordingly: class Validation { private $errorList = array(); public function isAlphaNumeric($string, $field, $msg = '') { if (!preg_match('/^[a-z\d]+$/i', $string)) { $this->errorList[$field] = $msg; } } // more methods here public function creditCard($cardNumber, $field, $msg = '') { // Validate credit card number } // more methods here public function hasErrors() { return count($this->errorList); } } /* Client code */ $validate = new Validation(); $validate->isAlphaNumeric($fieldvalue1, $fieldname1, 'Please only use letters and numbers for your street address'); $validate->creditCard($fieldvalue2, $fieldname2, 'Please enter a valid credit card number'); if ($validate->hasErrors()) { // Handle as appropriate } Naturally it didn't take long before this class became bloated with the virtually unlimited types of data to be validated. What I'm doing now is using decorators to separate the different types of data into their own classes and call them only when needed leaving generic validations (i.e. isAlphaNumeric()) in the base class: class Validation { private $errorList = array(); public function isAlphaNumeric($string, $field, $msg = '') { if (!preg_match('/^[a-z\d]+$/i', $string)) { $this->errorList[$field] = $msg; } } // more generic methods here public function setError($field, $msg = '') { $this->errorList[$field] = $msg; } public function hasErrors() { return count($this->errorList); } } class ValidationCreditCard { protected $validate; public function __construct(Validation $validate) { $this->validate = $validate; } public function creditCard($cardNumber, $field, $msg = '') { // Do validation // ... // if there is an error $this->validate->setError($field, $msg); } // more methods here } /* Client code */ $validate = new Validation(); $validate->isAlphaNumeric($fieldvalue, $fieldname, 'Please only use letters and numbers for your street address'); $validateCC = new ValidationCreditCard($validate); $validateCC->creditCard($fieldvalue2, $fieldname2, 'Please enter a valid credit card number'); if ($validate->hasErrors()) { // Handle as appropriate } Am I on the right track? Or did I just complicate data validation more then I needed to?

    Read the article

  • Why Cornell University Chose Oracle Data Masking

    - by Troy Kitch
    One of the eight Ivy League schools, Cornell University found itself in the unfortunate position of having to inform over 45,000 University community members that their personal information had been breached when a laptop was stolen. To ensure this wouldn’t happen again, Cornell took steps to ensure that data used for non-production purposes is de-identified with Oracle Data Masking. A recent podcast highlights why organizations like Cornell are choosing Oracle Data Masking to irreversibly de-identify production data for use in non-production environments. Organizations often copy production data, that contains sensitive information, into non-production environments so they can test applications and systems using “real world” information. Data in non-production has increasingly become a target of cyber criminals and can be lost or stolen due to weak security controls and unmonitored access. Similar to production environments, data breaches in non-production environments can cost millions of dollars to remediate and cause irreparable harm to reputation and brand. Cornell’s applications and databases help carry out the administrative and academic mission of the university. They are running Oracle PeopleSoft Campus Solutions that include highly sensitive faculty, student, alumni, and prospective student data. This data is supported and accessed by a diverse set of developers and functional staff distributed across the university. Several years ago, Cornell experienced a data breach when an employee’s laptop was stolen.  Centrally stored backup information indicated there was sensitive data on the laptop. With no way of knowing what the criminal intended, the university had to spend significant resources reviewing data, setting up service centers to handle constituent concerns, and provide free credit checks and identity theft protection services—all of which cost money and took time away from other projects. To avoid this issue in the future Cornell came up with several options; one of which was to sanitize the testing and training environments. “The project management team was brought in and they developed a project plan and implementation schedule; part of which was to evaluate competing products in the market-space and figure out which one would work best for us.  In the end we chose Oracle’s solution based on its architecture and its functionality.” – Tony Damiani, Database Administration and Business Intelligence, Cornell University The key goals of the project were to mask the elements that were identifiable as sensitive in a consistent and efficient manner, but still support all the previous activities in the non-production environments. Tony concludes,  “What we saw was a very minimal impact on performance. The masking process added an additional three hours to our refresh window, but it was well worth that time to secure the environment and remove the sensitive data. I think some other key points you can keep in mind here is that there was zero impact on the production environment. Oracle Data Masking works in non-production environments only. Additionally, the risk of exposure has been significantly reduced and the impact to business was minimal.” With Oracle Data Masking organizations like Cornell can: Make application data securely available in non-production environments Prevent application developers and testers from seeing production data Use an extensible template library and policies for data masking automation Gain the benefits of referential integrity so that applications continue to work Listen to the podcast to hear the complete interview.  Learn more about Oracle Data Masking by registering to watch this SANS Institute Webcast and view this short demo.

    Read the article

  • Removing Barriers to Create Effective Data Models

    After years of creating and maintaining data models, I have started to notice common barriers that decrease the accuracy and usefulness of models. In my opinion, the main causes of these barriers are the lack of knowledge and communication from within a company. The lack of knowledge in regards to data models or data modeling can take many forms. Company Culture Knowledge Whether documented or undocumented, existing business rules of a company can affect how data is modeled. For example, if a company only allows 1 assigned person per customer to be able to manipulate a customer’s record then then a data model that includes an associated table that joins customers and employee’s would be unneeded because that would allow for the possibility of multiple employees to handle a customer because of the potential for a many to many relationship between Customers and Employees. Technical Knowledge Depending on the data modeler’s proficiency in modeling data they can inadvertently cause issues and/or complications with a design without even noticing. It is important that companies share data modeling responsibilities so that the models are developed from multiple perspectives of a system, company and the original problem.  In addition, the tools that a company selects to create data models can also affect the accuracy of the model if designer are not familiar with the tools or the tools are too complex to use for the designer. Existing System Knowledge In order for a data modeler to model data for an existing system so that new changes can be applied to a system then they need to at least know the basic concepts of a system so that they can work within it. This will promote reusability of data and prevent the chance of duplicating data. Project Knowledge This should be pretty obvious, but it is very hard to create an accurate data model without knowing what data needs to be modeled. I have always found it strange that I have been asked to start modeling data prior to a client formalizing any requirements. Usually when this happens I have to make several iterations to a model, and the client still does not know exactly what they want.  In addition additional issues can arise when certain stakeholders of a project are not consulted prior to the design or after the project is over because it can cause miss understandings and confusion by the end user as well as possibly not solving the original problem for which a project is intended to solve. One common thread between each type of knowledge is that they can all be avoided through the use of good communication. For example, if a modeler is new to a company then they should ask older employees about any business specific rules that may be documented or undocumented that must be applied to projects in general. Furthermore, if a modeler is not really familiar with a specific data modeling software then they need to speak up and ask for help form other employees or their manager. This will not only help the modeler in the project, but also help them in future projects that they do for the company. Additionally, if a project is not clearly defined prior to a data modeler being assigned the modeling project then it is their responsibility to communicate with the other stakeholders to clarify any part of a project that is unclear so that the data model that is created is accurately aligned with a project.

    Read the article

  • Is Form validation and Business validation too much?

    - by Robert Cabri
    I've got this question about form validation and business validation. I see a lot of frameworks that use some sort of form validation library. You submit some values and the library validates the values from the form. If not ok it will show some errors on you screen. If all goes to plan the values will be set into domain objects. Here the values will be or, better said, should validated (again). Most likely the same validation in the validation library. I know 2 PHP frameworks having this kind of construction Zend/Kohana. When I look at programming and some principles like Don't Repeat Yourself (DRY) and single responsibility principle (SRP) this isn't a good way. As you can see it validates twice. Why not create domain objects that do the actual validation. Example: Form with username and email form is submitted. Values of the username field and the email field will be populated in 2 different Domain objects: Username and Email class Username {} class Email {} These objects validate their data and if not valid throw an exception. Do you agree? What do you think about this aproach? Is there a better way to implement validations? I'm confused about a lot of frameworks/developers handling this stuff. Are they all wrong or am I missing a point? Edit: I know there should also be client side kind of validation. This is a different ballgame in my Opinion. If You have some comments on this and a way to deal with this kind of stuff, please provide.

    Read the article

  • How often do you use data structures (ie Binary Trees, Linked Lists) in your jobs/side projects?

    - by Chris2021
    It seems to me that, for everyday use, more primitive data structures like arrays get the job done just as well as a binary tree would. My question is how common is to use these structures when writing code for projects at work or projects that you pursue in your free time? I understand the better insertion time/deletion time/sorting time for certain structures but would that really matter that much if you were working with a relatively small amount of data?

    Read the article

  • Why would you use data structures (ie Binary Trees, Linked Lists) in your jobs/side projects? [closed]

    - by Chris2021
    It seems to me that, for everyday use, more primitive data structures like arrays get the job done just as well as a binary tree would. My question is how common is to use these structures when writing code for projects at work or projects that you pursue in your free time? I understand the better insertion time/deletion time/sorting time for certain structures but would that really matter that much if you were working with a relatively small amount of data?

    Read the article

  • DRY Validation with MVC2

    - by Matthew
    Hi All, I'm trying to figure out how I can define validation rules for my domain objects in one single location within my application but have run in to a snag... Some background: My location has several parts: - Database - DAL - Business Logic Layer - SOAP API Layer - MVC website The MVC website accesses the database via the SOAP API, just as third parties would. We are using server and and client side validation on the MVC website as well as in the SOAP API Layer. To avoid having to manually write client side validation we are implementing strongly typed views in conjunction with the Html.TextBoxFor and Html.ValidationMessageFor HTML helpers, as shown in Step 3 here. We also create custom models for each form where one form takes input for multiple domain objects. This is where the problem begins, the HTML helpers read from the model for the data annotation validation attributes. In most cases our forms deal with multiple domain objects and you can't specify more than one type in the <%@Page ... Inherits="System.Web.Mvc.ViewPage" % page directive. So we are forced to create a custom model class, which would mean duplicating validation attributes from the domain objects on to the model class. I've spent quite some time looking for workarounds to this, such has referencing the same MetadataType from both the domain class and the custom MVC models, but that won't work for several reasons: You can only specify one MetadataType attribute per class, so its a problem if a model references multiple domain objects, each with their own metadata type. The data annotation validation code throws an exception if the model class doesn't contain a property that is specified in the referenced MetadataType which is a problem with the model only deals with a subset of the properties for a given domain object. I've looked at other solutions as well but to no avail. If anyone has any ideas on how to achieve a single source for validation logic that would work across MVC client and server side validation functionality and other locations (such as my SOAP API) I would love to hear it! Thanks in advance, Matthew

    Read the article

  • Create combined client side and server side validation in Symfony2

    - by ausi
    I think it would be very useful to create client side form validation up on the symfony2 Form and Validator components. The best way to do this would be to pass the validation constraints to the form view. With that information it would be possible to make a template that renders a form field to something like this: <div> <label for="form_email">E-Mail</label> <input id="form_email" type="text" name="form[email]" value="" data-validation-constraints='["NotBlank":{},"MinLength":{"limit":6}]' /> </div> The JavaScript part then would be to find all <input> elements that have the data-validation-constraints attribute and create the correct validation for them. To pass the validation constraints to the form view i thought the best way would be to create a form type extension. That's the point of my Question: Is this the correct way? And how is this possible? At the Moment my form type extension looks like this: use Symfony\Component\Form\FormInterface; use Symfony\Component\Form\FormView; use Symfony\Component\Form\FormBuilder; class FieldTypeExtension extends \Symfony\Component\Form\AbstractTypeExtension{ public function getExtendedType(){ return 'field'; } public function buildView(FormView $view, FormInterface $form) { // at this point i didn't find a way to get the // validation constraints out of the $form // the `getAllValidationConstraints` here is just an example $view->set('validation_constraints', $form->getAllValidationConstraints()); } } How can i get all validation constraints applied to one form field out of the FormInterface object?

    Read the article

< Previous Page | 5 6 7 8 9 10 11 12 13 14 15 16  | Next Page >