Sunday, February 21, 2010

Data as the Grand Illusion...

I have been lucky in my professional life to have met some really fascinating and knowledgeable people who have guided my data quality journey over the past few years. One of those persons is Jill Dyche (@JillDyche) with Baseline Consulting. After working with them as a customer at Microsoft, I then had the opportunity to sit down with her and her partner at Baseline Evan Levy during some fabulous off hours discussions during the Fall 2008 session of TDWI's World Conference in San Diego. Imagine being able raise a beer (or two) with some of the best in YOUR industry — and just hoping that some of that knowledge somehow would rub off on you and/or that you retain a tenth of that discussion...


Fast forward to 2009 — I continued to follow Jill and Evan via various blogs and web postings, and with the introduction of Twitter I now had another great outlet for getting my data quality info. It was via Jill that I got 'introduced' to Jim Harris and his outstanding @ocdqblog postings. It is really great to read the interactions between Jim, Jill and others of our industry, and to be able to learn from them...

So imagine my surprise and delight to find a tweet from Jill highlighting a recent post by Jim — where the challenge was for Jim to write an article about data quality in the style of a Rush song! TOO COOL! Now, for the record there is really only ONE Rush song I like — Tom Sawyer — but after reading Jim's post he not only captures the essence that IS Rush, but he manages to do a really great job of visualizing some of the challenges we have on selling data quality to our customers and co-workers.

So — I thought I would share my own take on this challenge and discuss data quality in the style of Styx's The Grand Illusion... I think too often we try to explain data quality issues and practices to those we are working with, and they just don't want to see it. I thought about following up with "Come Sail Away" as an encore, but figured that I had pushed this far enough.

Thank you Jim for a great read. Thank you Jill for your inspiration. Thank you Dennis DeYoung and Tommy Shaw for your fabulous music.

The Grand (Data) ILLUSION
Data can be a Grand Illusion
Run that report and see what's happening
Run your scripts, get a profiler just for show…

The staging db’s set, are audits running?
Suddenly your DBA’s heart is pounding
Wishing secretly they had backed up that cube.

But don't be fooled by those analysts
The managers or the CIOs
They’ll show you graphs of how your data should be
But they're just someone else's fantasy.

So if you think your data is complete confusion
Because you never match the source
Just remember that it's a Grand illusion
And without signed specs we’re placing blame…
And it’s all a game...

So if you think your data is complete confusion
Because your data steward said it was…
Just remember that it's a Grand illusion
And yet we can make it match again.
Auto correct that name...

Profiling spells competition, join us in our blind ambition
Get yourself a brand new cloud server
Someday soon we'll stop to ponder what the TSQL meant, I’ll always wonder
Jim’s made the grade and still we wonder if Jill and Evan are buying the next round?

I would love to hear what you think... drop me a note at berry at jbrcc dot com.