By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics in sports industry
    Here’s How Data Analytics In Sports Is Changing The Game
    6 Min Read
    data analytics on nursing career
    Advances in Data Analytics Are Rapidly Transforming Nursing
    8 Min Read
    data analytics reveals the benefits of MBA
    Data Analytics Technology Proves Benefits of an MBA
    9 Min Read
    data-driven image seo
    Data Analytics Helps Marketers Substantially Boost Image SEO
    8 Min Read
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Decision Tree Bagging
Share
Notification Show More
Latest News
big data mac performance
Data-Driven Tips to Optimize the Speed of Macs
News
3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
Artificial Intelligence
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics
data analytics reveals the benefits of MBA
Data Analytics Technology Proves Benefits of an MBA
Analytics
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Mining > Decision Tree Bagging
Business IntelligenceData MiningPredictive Analytics

Decision Tree Bagging

Editor SDC
Last updated: 2009/05/21 at 7:07 AM
Editor SDC
4 Min Read
SHARE
A few months ago I started looking for a new trading system idea following the same machine learning philosophy as the last one. 
The previous system was based on support vector regression, and used sliding window crossvalidation to set the kernel width and SVM cost parameters. It had a few problems:
1. MLE (maximum likelihood estimation) models are always uncertain because they may not be robust. Look up “model averaging” or “Bayesian methods”
2. Crossvalidation required retraining the SVM thousands of times which was extremely slow.
3. Support Vector Regression can only take numeric/ordered data as inputs, not categorical.
4. Regression estimates can be hard to interpret. e.g. if during training the system never saw a day where the price rose 20%, then it predicts 20% should you interpret it as a strong or a broken signal?
5. Crossvalidation over two parameters required….

A few months ago I started looking for a new trading system idea following the same machine learning philosophy as the last one. 
The previous system was based on support vector regression, and used sliding window crossvalidation to set the kernel width and SVM cost parameters. It had a few problems:
1. MLE (maximum likelihood estimation) models are always uncertain because they may not be robust. Look up “model averaging” or “Bayesian methods”
2. Crossvalidation required retraining the SVM thousands of times which was extremely slow.
3. Support Vector Regression can only take numeric/ordered data as inputs, not categorical.
4. Regression estimates can be hard to interpret. e.g. if during training the system never saw a day where the price rose 20%, then it predicts 20% should you interpret it as a strong or a broken signal?
5. Crossvalidation over two parameters required the creation of a 4 dimensional matrix to store performance. This was very hard to visualize, especially after not working on the system for a while. 
6. Coming up with a loss function for regression is hard for the application of trading. MSE is not perfect, nor is correlation. The loss function should penalize false positives because of transaction costs.
7. No natural confidence values.
8. Hard to interpret SVM. Infinite features w/ Gaussian RBF? The effect of changing C & kernel width are not easy to anticipate or interpret.
9. Non-linear but biased toward linearity (for ex., bad at learning XOR)
A few months ago I settled on a new learning algorithm to build a system on: the random forest. Random forest is a clever name for decision tree bagging (ensemble). And bagging is a clever conjunction of “bootstrap aggregating”. I especially liked that it could accept any data, numerical or categorical, gives confidence values, and is easier to interpret. Then I started printing and reading papers on decision trees, random forests, and the bootstrap. I read about four papers a week. The random forest also improves on the standard decision tree, which I wrote about previously, on problems 1 and 7 above. 
I’ll post the code in a few days once it has been tested.

Editor SDC May 21, 2009
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

big data mac performance
Data-Driven Tips to Optimize the Speed of Macs
News
3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
Artificial Intelligence
data analytics in sports industry
Here’s How Data Analytics In Sports Is Changing The Game
Big Data
data analytics on nursing career
Advances in Data Analytics Are Rapidly Transforming Nursing
Analytics

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows
Artificial Intelligence

3 Ways AI Has Helped Marketers and Creative Professionals Streamline Workflows

6 Min Read
ai in ppc advertising
Artificial Intelligence

5 Proven Tips for Utilizing AI with PPC Advertising in 2023

10 Min Read
ai in web design
Artificial Intelligence

5 Ways AI Technology Has Disrupted Website Development

7 Min Read
Digital Security From Weaponized AI
Security

Fortifying Enterprise Digital Security Against Hackers Weaponizing AI

11 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

data-driven web design
5 Great Tips for Using Data Analytics for Website UX
Big Data
AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?