By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics in sports industry
    Here’s How Data Analytics In Sports Is Changing The Game
    6 Min Read
    data analytics on nursing career
    Advances in Data Analytics Are Rapidly Transforming Nursing
    8 Min Read
    data analytics reveals the benefits of MBA
    Data Analytics Technology Proves Benefits of an MBA
    9 Min Read
    data-driven image seo
    Data Analytics Helps Marketers Substantially Boost Image SEO
    8 Min Read
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: The Art of Pickling Data
Share
Notification Show More
Latest News
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security
ai in software development
3 AI-Based Strategies to Develop Software in Uncertain Times
Software
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Warehousing > The Art of Pickling Data
Data Warehousing

The Art of Pickling Data

DataQualityEdge
Last updated: 2009/08/24 at 2:23 PM
DataQualityEdge
4 Min Read
SHARE

I have never pickled before, and probably won’t but I do enjoy eating them. The best tasting pickles one can imagine were pulled out of our 69 year-old backpacking mountaineer\pickling savant companion’s backpack last week. Yes, he brought a jar of pickles into the mountains, which we all enjoyed and devoured. So to the man known as ‘Uncle Dave’, I salute you and here’s a little analogy of pickling data. Besides who doesn’t like a good crunchy pickle.

— 1) In pickling we need to sterilize the equipment. Otherwise you may get contaminants that can ruin your pickles.

In datawarehousing, we need a computer or server to store the data electronically. You want to start with a clean server to maximize the amount of data you can store and to ensure no ‘cross-contamination from old tables’. I haven’t heard of this happening other then in mainframe environments; where the back-end data from ‘shadow tables’ can still come back and repopulate the ‘main’ front-end tables. If the bad data was not removed from both the back-end and front-end tables simultaneously contamination will happen.

More Read

What is Data Pipeline A detailed explaination

What is Data Pipeline? A Detailed Explanation

Understanding ETL Tools as a Data-Centric Organization
Differentiating Between Data Lakes and Data Warehouses
How Will The Cloud Impact Data Warehousing Technologies?
Big Data Is More Prevalent in Daily Life Than You Might Think

— 2) Prepare the brine with salt, vinegar, garlic and other spices/ingredients to create your pickling …


I have never pickled before, and probably won’t but I do enjoy eating them. The best tasting pickles one can imagine were pulled out of our 69 year-old backpacking mountaineer\pickling savant companion’s backpack last week. Yes, he brought a jar of pickles into the mountains, which we all enjoyed and devoured. So to the man known as ‘Uncle Dave’, I salute you and here’s a little analogy of pickling data. Besides who doesn’t like a good crunchy pickle.

— 1) In pickling we need to sterilize the equipment. Otherwise you may get contaminants that can ruin your pickles.

In datawarehousing, we need a computer or server to store the data electronically. You want to start with a clean server to maximize the amount of data you can store and to ensure no ‘cross-contamination from old tables’. I haven’t heard of this happening other then in mainframe environments; where the back-end data from ‘shadow tables’ can still come back and repopulate the ‘main’ front-end tables. If the bad data was not removed from both the back-end and front-end tables simultaneously contamination will happen.

— 2) Prepare the brine with salt, vinegar, garlic and other spices/ingredients to create your pickling solution, bring to a boil.

Prepare your scripts, data loading jobs, data models,tables, attributes, your data quality routines and more. I included data quality routines because you want to study the trends determine when they break from the norm. Data quality is the spice that will make it all better.

— 3) Boil vegetables place in jar with pickling solution, and seal.

Prepare your files and run the jobs to load the data in your repository.

— 4) After a few weeks, enjoy the pickles of your labour, the crunchier the better.

Unlike pickling, you can begin to enjoy the crunchy bits of your data and what they are telling you immediately after the data is stored. It might not be tasty but it may very well be interesting. After all, the interpretation of data is information, and information is power.

DataQualityEdge August 24, 2009
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
anti-spoofing tips
Anti-Spoofing is Crucial for Data-Driven Businesses
Security

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

What is Data Pipeline A detailed explaination
Big Data

What is Data Pipeline? A Detailed Explanation

8 Min Read
etl for data-driven businesses
Big Data

Understanding ETL Tools as a Data-Centric Organization

8 Min Read
data lake vs data warehouse
Data Lake

Differentiating Between Data Lakes and Data Warehouses

7 Min Read
moving to the cloud
Big DataCloud ComputingData WarehousingExclusive

How Will The Cloud Impact Data Warehousing Technologies?

6 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

giveaway chatbots
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive
AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?