5 Steps to Setting your Big Data Goals

We all have goals, or at least should.

We all have goals, or at least should. I’m reminded of a t-shirt that says, “If you don’t have goals, you’ll never score.” Meaning, if you don’t realize and set goals, you won’t get the desired outcome.In the context of Big Data, over half of projects started are never completed, and can even have very bad results.  

Having done both Big Data as well as BI projects before, we’ve learned not only what to do, but what not to do. If we could help even one person out there with this advice, it would make us all warm and fuzzy.

1) Your initiative needs a sponsor. Likely a C level employee within the organization that knows what value needs to come from the project. This person will take ownership of the project, and will be held accountable for failures and setbacks. He or she will check progress and milestones as well as address potential and actual blockers. It’s important that there is only one person overseeing the project, so that there aren’t instructions coming from various sources, resulting in people being pulled in different directions. On a positive note, the team members involved will know that they have one person to turn to, and that person is there with only one thing in mind, a clear vision of the outcome. Please note that this doesn’t mean there shouldn’t be other managers assisting, but everyone involved should be on the same page.   

2) Define your business questions. Business questions are crucial to discovering what business problems exist, so they can be understood and solved for the betterment of the company. What do you want to know? Maybe you want to know which of your campaigns worked the best based on user acquisition rate by geography and/or time of day. Perhaps you want to know how to reduce product shrinkage or optimize your warehouse layout.  If you’re an ecommerce, it would be good to know the revenue at a product level as well as average customer support calls/chat/tickets for a specific time frame.  

3) Start small. Don’t bite off more than you can chew. Focus on the most important questions first. This is not easy because you probably feel, and rightfully so, that all questions are important. They are, but which is most pertinent to the project is what needs to be targeted at this juncture. Questions will evolve and new ones will be added. Stay focused, and handle them at a later stage.

4) Invest in understanding the data. Where is it? Which data is coming from where? The best way to handle this is the process of data profiling. Also, expect schema changes and plan for your system to be able to handle those changes. If you can identify the problem areas at the beginning, it will be less difficult and take less time to handle them up front as opposed to once the system is built. Lastly with your data, expect data corruption and just bad data in general. Again, plan for this up front, it will save headaches in the long run.  

5) Get an expert or two. You’ll need a technical expert that knows the ins and outs of the platform and how it is to be built. If your technical expert isn’t well versed in the business side of the company, get someone that does.  He or she should know every aspect of the business model, the finances, the products and/or services, and how it is all tied together.  

This process will not be easy, but it will make going forward easier than if you did not undertake it at all. 

original post