ABS Training Course - Making Quality informed Decisions

Just to add some extra information regarding a Data Quality Framework. I recently took a course with the ABS titled 'Making Quality Informed Decisions". It was an excellent foundation and training for applying a Data Quality Framework to your organisation. It was obviously based around the ABS framework, however the principles could be use to help you apply any data quality framework to your organisation. It was a great day, with an informative approach that helped attendants relate the workshop to their organisations, and I couldn't recommend it enough. There are several other courses offered that may also apply to your role to help you gain a basic understanding on statistics, metrics, and how to turn data into usable information.

The ABS Data Quality Framework

One of the first things to take into account when starting on your long road to a governed information quality environment is your Data Quality Framework (DQF). With the ABS being the foremost central data and statistical organisation in the country, most of the frameworks and procedures they adopt tend to turn into a national standard. The DQF is no exception, with a plethora of Australian government organisations adopting this as their own.

It is essential to learn how to adapt this framework to your organisation. Initially I had a few gripes with a how to apply a missing dimension, however depending on the context, just about everything can be applied through sub categories of this framework. The seven dimensions of the ABS framework include Institutional Environment, Accuracy, Relevance, Timeliness, Accessibility, Coherence, and Interpretability. In ensuring your organisation has a common view of each dimensions importance to them, you must build a hierarchy. Traditionally Institutional Environment, Accuracy, and Timeliness are probably the most essential elements to what most people would traditionally associate with 'quality'. This is something to keep in mind.

Frameworks, Strategies, Policies… it seems its easy to get lost in all this documented beaurocracy. No matter how you approach your overall data quality initiative there are a few other essential accompaniments to your framework. At the forefront would be Data Quality Statements or Declarations, a Data Quality Assessment Framework or method, Roles & Definitions, and a Data Quality Maturity model. Sometimes the maturity model can be more associated with a Data Governance Framework, however DQ is a cornerstone in the Data Governance Framework - so it is a case of "Which came first, the chicken or the egg?". Essentially these all make up an overarching strategy, and I'll briefly outline a bit more about each of these components to a good Data Quality Strategy:

Data Quality Statements or Declarations:
Essentially these are a document to accompany any submission or delivery of a dataset internally or externally. They can be outlined by DQ dimensions and include a set of contextual questions to accompany each dimension. This is essential for someone to make a risk assessment around their data need. Does this data cover the right timeframe they need to research? What were the accuracy issues that were encountered during this dataset and how will it effect the outcome of the data need? All sorts of imperative questions can be answered by these quality statements prior to delving into the data and seeing what will arise.

Data Quality Assessment Framework:
This is a framework for how you would like people to approach the assessment of data collections or outputs in their area. The format and questions usually correlate pretty closely with your Data Quality Statements, and your output essentially feeds in to that document. This is where you will uncover all the nitty gritty to allow a dataset user to assess the fit for purpose for their data need.

Roles & Definitions:
There are a thousand different definitions around on the internet and in varying organisations. Essentially what you need to do here is build a common language for roles and responsibilities, as well as any data quality terminology and jargon. It's all about getting everyone onto the same page and promoting a culture right? Part of a culture is language, so you will need to determine how the language will suit your current organisational terminology.

Data Quality Maturity Model:
This part is easily confused. A lot of Executives will want to know what the data quality of a report is like, or how trustworthy it is. They want you to put it into some simple terms… what gets more simple than a number rating of 1-5? Essentially this is what a Data Quality Maturity Model boils down to. It is not a perfect representation of the quality of the data, however the score is associated with how governed and processed the data is. You will have to tailor this to your organisation and their processes. An example range would be level 1 would be data that is straight from the source system with only local processes applied. The range would then extend up to the highest level, where 5 represents the data being governed, modelled, entered in the data dictionary, validated, cleansed, warehoused, and all the other data quality processes of your organisation have been applied. You may feel that a rating of 1-5 is too broad and you need to further define it. This is where tailoring to your organisation is imperative and you must make a process that will work for you. This is also a vital component to building quality through Data Governance.

The ABS are also very good at supplying extra information and helping you apply the framework to your organisation. Their services include many different classes and workshops, as well as Information Management Strategy consultants to help your organisation build and implement a strategy that will suit you. Ultimately not one size will always fit all, and there will be more information on Data Quality Strategy and Framework to come in due course….

For more information on the ABS Data Quality Framework click HERE.

The Journey Begins here...

The Road to Data Quality... it's a rocky, unsealed, dusty, windy, one lane country road. Trying to progress from a smallsville in the middle of nowhere, to make it to the organised highways and freeways of the big smoke is a long journey. More often than not, the information on how to get from the start of your journey to the end are there. There are countless different road maps, start point and end point GPS coordinates, you can see a thousand different satellite views on the area you are to travel through. The aim of this blog is to be like the navigator in your car, describing the twists, turns, and journey as it eventuates.

OK. Without wasting too much time on metaphor and imagery, what I'm providing here is a "how to" guide from beginning to end on Data Quality. Over the last few months I have started a new position with an organisation with no formal Data Governance policy or framework, including the lack of Data Quality policy, framework and procedure. While there is a wealth of information on the internet, it all seems to be for large commercial enterprise and doesn't seem to cover basics enough. Organisational Change. Embedding Corporate Culture. The big scary metamorphasis from an ungoverned wild west into an organised metropolitan centre. Data Quality must start from the ground up, and if an organisation does not embrace the most basic principles - you will be profiling, mining, and cleansing data till the cows come home but wasting your time.

So... to all those interested parties out there - This is your starting point. The blog may be best to follow in a chronological order, from start to finish. Witness the issues, risks, threats, opportunities, cultural shift, and implementation of a data quality project (in vague terms so I don't violate any NDA's) for the less technical to relate to. Technical aspects are essential, it IS data we are talking about, however they come later in the plan. The first step is raising the awareness, building policy & procedure, and laying the foundation towards the future path towards a paved road...