Sometimes the universe just screams an idea at you and all you can do is listen to it. As I pondered my initial foray into the blogosphere, I was pointed to Jill Dyche’s blog on designing a competency center (Feb. 8, 2011 http://baseline-consulting.typepad.com/jilldyche/). At home , my wife is currently exploring our many options for redesigning our front entry, and the current book I am reading is “The Design of Everyday Things” by Don Norman (thanks to John Susag for the recommendation!). So I opened myself up to the cosmos and wanted to kick off my blog with what I think is the underpinning issue in data warehousing – Are you designing your solution for end user ability or something else?
Editor’s note: Rob Armstrong is an employee of Teradata. Teradata is a sponsor of The Smart Data Collective.
By way of an introduction, this has been my life longer than it hasn’t been. I used to joke that the longest relationship I had was with my pager (same pager number for 10 years!). Through my 25 years at Teradata, I have been involved in coding database functionality, supporting systems from a deep technical nature, and then participated in the exciting, on-site world of implementing data warehouses in a variety of industries. From IT to Business, and from worker to decision maker, I have learned a lot of lessons and look forward to sharing them with you and I trust I will also learn a thing or two from our discussions.
So what is the point of exterior design? One of the themes of the “everyday things” book is that many products are designed for the management and maintenance of the production processes rather than the usability and effectiveness of it to the ultimate user. I will not spoil the book for you but it is an interesting read. Too many data warehouse implementations are concerned with the process of collecting and managing the data. Think of the infamous “batch window” where the database is taken away from the users for X hours to load detail data and then manage data summaries. In today’s world it is hard to find implementers, let alone business users, that think that batch windows and regularly taking data systems down is an acceptable solution.
This idea goes into a lot of processes. From the data collection and ETL efforts, to the data model and embedding the business rules into the metadata, and to the end user GUI and how user friendly and intuitive it is to ultimately allow the user to “self serve”. If the design and implementation teams keep the end usage in mind then greater acceptance and productiveity will occur.
Now comes the dilemma, or what I like to term a “data warehouse paradox”. How does one design for the business usage when the business usage constantly changes? It is hard enough to get the business to define the initial data and usage, and yet to adequately design and implement the warehouse we need to have them define continuing usage over the years? Good luck.
Fortunately there are some answers that will assist in overcoming this problem. In the interest of “blog time” I will explore each of the main ones over the next few postings. The first answer is that regardless of what happens, the users will constantly want new and different analytics against basically the same data.
This is good as we know that if we have a substantive amount of data and we keep the data relationships intact in the model then any analytics can be run against the data. If you start to enforce the usage and function in the data model then you are going to erect barriers to new analytics. The design may be very easy to load and manage, and may be easy for the initial usage (after all it was designed for that) but it makes new requirements very hard to accommodate and usage will quickly die out. What you are left with is a system that cost a lot but provides very little.
OK, let’s start with that. The question of the week is how flexible is your underlying model? Have you functionalized the model for “easy of use”? How long does it take to accommodate new requests against the same data? Do you need to constantly create new summaries and extracts to provide the next capability? These answers will provide a good understanding if you had the “exterior design” in mind and if you are on the path to long term success.
Next blog I will continue the things that we can know about the business need that allows the IT groups to accelerate the path towards true self service.
