By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data Analytics instagram stories
    Data Analytics Helps Marketers Make the Most of Instagram Stories
    15 Min Read
    analyst,women,looking,at,kpi,data,on,computer,screen
    What to Know Before Recruiting an Analyst to Handle Company Data
    6 Min Read
    AI analytics
    AI-Based Analytics Are Changing the Future of Credit Cards
    6 Min Read
    data overload showing data analytics
    How Does Next-Gen SIEM Prevent Data Overload For Security Analysts?
    8 Min Read
    hire a marketing agency with a background in data analytics
    5 Reasons to Hire a Marketing Agency that Knows Data Analytics
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Decision Tree Bagging
Share
Notification Show More
Aa
SmartData CollectiveSmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Mining > Decision Tree Bagging
Business IntelligenceData MiningPredictive Analytics

Decision Tree Bagging

Editor SDC
Last updated: 2009/05/21 at 7:07 AM
Editor SDC
4 Min Read
SHARE
A few months ago I started looking for a new trading system idea following the same machine learning philosophy as the last one. 
The previous system was based on support vector regression, and used sliding window crossvalidation to set the kernel width and SVM cost parameters. It had a few problems:
1. MLE (maximum likelihood estimation) models are always uncertain because they may not be robust. Look up “model averaging” or “Bayesian methods”
2. Crossvalidation required retraining the SVM thousands of times which was extremely slow.
3. Support Vector Regression can only take numeric/ordered data as inputs, not categorical.
4. Regression estimates can be hard to interpret. e.g. if during training the system never saw a day where the price rose 20%, then it predicts 20% should you interpret it as a strong or a broken signal?
5. Crossvalidation over two parameters required….

A few months ago I started looking for a new trading system idea following the same machine learning philosophy as the last one. 
The previous system was based on support vector regression, and used sliding window crossvalidation to set the kernel width and SVM cost parameters. It had a few problems:
1. MLE (maximum likelihood estimation) models are always uncertain because they may not be robust. Look up “model averaging” or “Bayesian methods”
2. Crossvalidation required retraining the SVM thousands of times which was extremely slow.
3. Support Vector Regression can only take numeric/ordered data as inputs, not categorical.
4. Regression estimates can be hard to interpret. e.g. if during training the system never saw a day where the price rose 20%, then it predicts 20% should you interpret it as a strong or a broken signal?
5. Crossvalidation over two parameters required the creation of a 4 dimensional matrix to store performance. This was very hard to visualize, especially after not working on the system for a while. 
6. Coming up with a loss function for regression is hard for the application of trading. MSE is not perfect, nor is correlation. The loss function should penalize false positives because of transaction costs.
7. No natural confidence values.
8. Hard to interpret SVM. Infinite features w/ Gaussian RBF? The effect of changing C & kernel width are not easy to anticipate or interpret.
9. Non-linear but biased toward linearity (for ex., bad at learning XOR)
A few months ago I settled on a new learning algorithm to build a system on: the random forest. Random forest is a clever name for decision tree bagging (ensemble). And bagging is a clever conjunction of “bootstrap aggregating”. I especially liked that it could accept any data, numerical or categorical, gives confidence values, and is easier to interpret. Then I started printing and reading papers on decision trees, random forests, and the bootstrap. I read about four papers a week. The random forest also improves on the standard decision tree, which I wrote about previously, on problems 1 and 7 above. 
I’ll post the code in a few days once it has been tested.

Editor SDC May 21, 2009
Share This Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

ai low code frameworks
AI Can Help Accelerate Development with Low-Code Frameworks
Artificial Intelligence
data Analytics instagram stories
Data Analytics Helps Marketers Make the Most of Instagram Stories
Analytics
data breaches
How Hospital Security Breaches Devastate Local Communities
Policy and Governance
analyst,women,looking,at,kpi,data,on,computer,screen
What to Know Before Recruiting an Analyst to Handle Company Data
Analytics

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

ai low code frameworks
Artificial Intelligence

AI Can Help Accelerate Development with Low-Code Frameworks

12 Min Read
data perspective
Big Data

Tackling Bias in AI Translation: A Data Perspective

9 Min Read
How AI is Boosting the Customer Support Game
Artificial Intelligence

How AI is Boosting the Customer Support Game

6 Min Read
AI analytics
AnalyticsArtificial IntelligenceExclusive

AI-Based Analytics Are Changing the Future of Credit Cards

6 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive
data-driven web design
5 Great Tips for Using Data Analytics for Website UX
Big Data

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?