Many companies or organizations do regular data cleansing. When you cleanse
the data,
the data quality goes up to some higher level.
The data quality level is determined by
the amount
of work invested in
the cleansing. As time passes,
the data quality deteriorates, and you need to repeat
the cleansing process. If you spend an equal amount
of effort as you did with
the previous cleansing, you can expect
the same level
of data quality as you had after
the previous cleansing. And then
the data quality deteriorates over time again, and
the cleansing process starts over and over again.
The idea
of Data Quality Services is to mitigate
the cleansing process. While
the amount
of time you need to spend on cleansing decreases, you will achieve higher and higher levels
of data quality. While cleansing, you learn what types
of errors to expect, discover error patterns, find domains
of correct values, etc. You don’t throw away this knowledge. You store it and use it to find and correct
the same issues automatically during your next cleansing process.
The following figure shows this graphically.
The idea
of master data management, which you can perform with Master Data Services (MDS), is to prevent data quality from deteriorating. Once you reach a particular quality level,
the MDS application—together with
the defined policies, people, and master data management processes—allow you to maintain this level permanently. This idea is shown in
the following picture. OK, now you know what DQS and MDS are about. You can imagine
the importance on maintaining
the data quality. Here are some resources that help you preparing and executing
the data quality (DQ) and master data management (MDM) activities. Books Dejan Sarka and Davide Mauri: Data Quality and Master Data Management with Microsoft SQL Server 2008 R2 – a general introduction to MDM, MDS, and data profiling. Matching explained in depth. Dejan Sarka, Matija Lah and Grega Jerkic: MCTS Self-Paced Training Kit (Exam 70-463): Building Data Warehouses with Microsoft SQL Server
2012 – I wrote quite a few chapters about DQ and MDM, and introduced also SQL Server
2012 DQS. Thomas Redman: Data Quality:
The Field Guide – you should start with this book. Thomas Redman is
the father
of DQ and MDM. Tyler Graham: Microsoft SQL Server
2012 Master Data Services – MDS in depth from a product team mate. Arkady Maydanchik: Data Quality Assessment – data profiling in depth. Tamraparni Dasu, Theodore Johnson: Exploratory Data Mining and Data Cleaning – advanced data profiling with data mining. Forthcoming presentations I am presenting a DQS and MDM seminar at PASS SQL Rally Amsterdam 2013: Wednesday, November 6th, 2013: Enterprise Information Management with SQL Server
2012 – a good kick start to your first DQ and / or MDM project. Courses Data Quality and Master Data Management with SQL Server
2012 – I wrote a 2-day course for SolidQ. If you are interested in this course, which I could also deliver in a shorter seminar way, you can contact your closes SolidQ subsidiary, or,
of course, me directly on addresses
[email protected] or
[email protected]. This course could also complement
the existing courseware portfolio
of training providers, which are welcome to contact me as well. Start improving
the quality
of your data now!