Cookies help us display personalized product recommendations and ensure you have great shopping experience.

By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    sales and data analytics
    How Data Analytics Improves Lead Management and Sales Results
    9 Min Read
    data analytics and truck accident claims
    How Data Analytics Reduces Truck Accidents and Speeds Up Claims
    7 Min Read
    predictive analytics for interior designers
    Interior Designers Boost Profits with Predictive Analytics
    8 Min Read
    image fx (67)
    Improving LinkedIn Ad Strategies with Data Analytics
    9 Min Read
    big data and remote work
    Data Helps Speech-Language Pathologists Deliver Better Results
    6 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: The Art of Pickling Data
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Warehousing > The Art of Pickling Data
Data Warehousing

The Art of Pickling Data

DataQualityEdge
DataQualityEdge
4 Min Read
SHARE

I have never pickled before, and probably won’t but I do enjoy eating them. The best tasting pickles one can imagine were pulled out of our 69 year-old backpacking mountaineer\pickling savant companion’s backpack last week. Yes, he brought a jar of pickles into the mountains, which we all enjoyed and devoured. So to the man known as ‘Uncle Dave’, I salute you and here’s a little analogy of pickling data. Besides who doesn’t like a good crunchy pickle.

— 1) In pickling we need to sterilize the equipment. Otherwise you may get contaminants that can ruin your pickles.

In datawarehousing, we need a computer or server to store the data electronically. You want to start with a clean server to maximize the amount of data you can store and to ensure no ‘cross-contamination from old tables’. I haven’t heard of this happening other then in mainframe environments; where the back-end data from ‘shadow tables’ can still come back and repopulate the ‘main’ front-end tables. If the bad data was not removed from both the back-end and front-end tables simultaneously contamination will happen.

— 2) Prepare the brine with salt, vinegar, garlic and other spices/ingredients to create your pickling …


I have never pickled before, and probably won’t but I do enjoy eating them. The best tasting pickles one can imagine were pulled out of our 69 year-old backpacking mountaineer\pickling savant companion’s backpack last week. Yes, he brought a jar of pickles into the mountains, which we all enjoyed and devoured. So to the man known as ‘Uncle Dave’, I salute you and here’s a little analogy of pickling data. Besides who doesn’t like a good crunchy pickle.

— 1) In pickling we need to sterilize the equipment. Otherwise you may get contaminants that can ruin your pickles.

In datawarehousing, we need a computer or server to store the data electronically. You want to start with a clean server to maximize the amount of data you can store and to ensure no ‘cross-contamination from old tables’. I haven’t heard of this happening other then in mainframe environments; where the back-end data from ‘shadow tables’ can still come back and repopulate the ‘main’ front-end tables. If the bad data was not removed from both the back-end and front-end tables simultaneously contamination will happen.

— 2) Prepare the brine with salt, vinegar, garlic and other spices/ingredients to create your pickling solution, bring to a boil.

Prepare your scripts, data loading jobs, data models,tables, attributes, your data quality routines and more. I included data quality routines because you want to study the trends determine when they break from the norm. Data quality is the spice that will make it all better.

— 3) Boil vegetables place in jar with pickling solution, and seal.

Prepare your files and run the jobs to load the data in your repository.

— 4) After a few weeks, enjoy the pickles of your labour, the crunchier the better.

Unlike pickling, you can begin to enjoy the crunchy bits of your data and what they are telling you immediately after the data is stored. It might not be tasty but it may very well be interesting. After all, the interpretation of data is information, and information is power.

More Read

Digital Universe Study: The Big Hype
Singularity Hub predicts the cost to sequence an entire…
Unify Brand, Agency, and Technology Efforts with Your Customer Data
The Social Media Evolution is an incredible opportunity: but needs business management’s understanding of why and how!
The amount of digital data created in 2010 will equal…
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

sales and data analytics
How Data Analytics Improves Lead Management and Sales Results
Analytics Big Data Exclusive
ai in marketing
How AI and Smart Platforms Improve Email Marketing
Artificial Intelligence Exclusive Marketing
AI Document Verification for Legal Firms: Importance & Top Tools
AI Document Verification for Legal Firms: Importance & Top Tools
Artificial Intelligence Exclusive
AI supply chain
AI Tools Are Strengthening Global Supply Chains
Artificial Intelligence Exclusive

Stay Connected

1.2kFollowersLike
33.7kFollowersFollow
222FollowersPin

You Might also Like

In this new era, as Saffo puts it, “…the central economic actor…

3 Min Read

Right Time Business Optimization

8 Min Read

InformationWeek’s “10 Most Strategic IT Vendors” Includes Teradata

4 Min Read

Cyberlaw scholar Jonathan Zittrain of Harvard: Ubiquitous human…

1 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai is improving the safety of cars
From Bolts to Bots: How AI Is Fortifying the Automotive Industry
Artificial Intelligence
AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-25 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?