By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    predictive analytics in dropshipping
    Predictive Analytics Helps New Dropshipping Businesses Thrive
    12 Min Read
    data-driven approach in healthcare
    The Importance of Data-Driven Approaches to Improving Healthcare in Rural Areas
    6 Min Read
    analytics for tax compliance
    Analytics Changes the Calculus of Business Tax Compliance
    8 Min Read
    big data analytics in gaming
    The Role of Big Data Analytics in Gaming
    10 Min Read
    analyst,women,looking,at,kpi,data,on,computer,screen
    Promising Benefits of Predictive Analytics in Asset Management
    11 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Budgeting Time on a Modeling Project
Share
Notification Show More
Latest News
ai software development
Key Strategies to Develop AI Software Cost-Effectively
Artificial Intelligence
ai in omnichannel marketing
AI is Driving Huge Changes in Omnichannel Marketing
Artificial Intelligence
ai for small business tax planning
Maximize Tax Deductions as a Business Owner with AI
Artificial Intelligence
ai in marketing with 3D rendering
Marketers Use AI to Take Advantage of 3D Rendering
Artificial Intelligence
How Big Data Is Transforming the Maritime Industry
How Big Data Is Transforming the Maritime Industry
Big Data
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > Modeling > Budgeting Time on a Modeling Project
Modeling

Budgeting Time on a Modeling Project

DeanAbbott
Last updated: 2012/09/13 at 1:36 PM
DeanAbbott
3 Min Read
SHARE

Within the time allotted for any empirical modeling project, the analyst must decide how to allocate time for various aspects of the process.  As is the case with any finite resource, more time spent on this means less time spent on that.  I suspect that many modelers enjoy the actual modeling part of the job most.  It is easy to try “one more” algorithm: Already tried logistic regression and a neural network?  Try CART next.

Within the time allotted for any empirical modeling project, the analyst must decide how to allocate time for various aspects of the process.  As is the case with any finite resource, more time spent on this means less time spent on that.  I suspect that many modelers enjoy the actual modeling part of the job most.  It is easy to try “one more” algorithm: Already tried logistic regression and a neural network?  Try CART next.

Of course, more time spent on the modeling part of this means less time spent on other things.  An important consideration for optimizing model performance, then, is: Which tasks deserve more time, and which less?

Experimenting with modeling algorithms at the end of a project will no doubt produce some improvements, and it is not argued here that such efforts be dropped.  However, work done earlier in the project establishes an upper limit on model performance.  I suggest emphasizing data clean-up (especially missing value imputation) and creative design of new features (ratios of raw features, etc.) as being much more likely to make the model’s job easier and produce better performance.

More Read

ai in business

Selecting the Right AI Business Model for Your Startup

Top 10 Powerful Data Modeling Tools For 2021
Learn Why Doctors Look To Data To Increase Patient Engagement
Spectral Clustering Can Be A Game Changer—Here’s How
How to Overcome Data Visualisation Problems

Consider how difficult it is for a simple 2-input model to discern “healthy” versus “unhealthy” when provided the input variables height and weight alone.  Such a model must establish a dividing line between healthy and unhealthy weights separately for each height.  When the analyst uses instead the ratio of weight to height, this becomes much simpler.  Note that the commonly used BMI (body mass index) is slightly more complicated than this, and would likely perform even better.  Cross categorical variables is another way to simplify the problem for the model.  Though we deal with a process we call “machine learning”, is is a pragmatic matter to make the job as easy as possible for the machine.

The same is true for handling missing values.  Simple global substitution using the non-missing mean or median is a start, but think about the spike that creates in the variable’s distribution.  Doing this over multiple variables creates a number of strange artifacts in the multivariate distribution.  Spending the time and energy to fill in those missing values in a smarter way (possibly by building a small model) cleans up the data dramatically for the downstream modeling process.

— Post by Will Dwinnell

DeanAbbott September 13, 2012
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

ai software development
Key Strategies to Develop AI Software Cost-Effectively
Artificial Intelligence
ai in omnichannel marketing
AI is Driving Huge Changes in Omnichannel Marketing
Artificial Intelligence
ai for small business tax planning
Maximize Tax Deductions as a Business Owner with AI
Artificial Intelligence
ai in marketing with 3D rendering
Marketers Use AI to Take Advantage of 3D Rendering
Artificial Intelligence

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

ai in business
Business Intelligence

Selecting the Right AI Business Model for Your Startup

11 Min Read
data modeling tools to analyze
Modeling

Top 10 Powerful Data Modeling Tools For 2021

8 Min Read
patient engagement
Big DataExclusiveModelingPredictive Analytics

Learn Why Doctors Look To Data To Increase Patient Engagement

9 Min Read
Big Data and the SME
AnalyticsModeling

Spectral Clustering Can Be A Game Changer—Here’s How

5 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

giveaway chatbots
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive
AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?