By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
    benefits of data analytics for financial industry
    Fascinating Changes Data Analytics Brings to Finance
    7 Min Read
    analyzing big data for its quality and value
    Use this Strategic Approach to Maximize Your Data’s Value
    6 Min Read
    data-driven seo for product pages
    6 Tips for Using Data Analytics for Product Page SEO
    11 Min Read
    big data analytics in business
    5 Ways to Utilize Data Analytics to Grow Your Business
    6 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Voodoo Spectrum of Machine Learning and Data Sets
Share
Notification Show More
Latest News
cloud-centric companies using network relocation
Cloud-Centric Companies Discover Benefits & Pitfalls of Network Relocation
Cloud Computing
construction analytics
5 Benefits of Analytics to Manage Commercial Construction
Analytics
database compliance guide
Four Strategies For Effective Database Compliance
Data Management
Digital Security From Weaponized AI
Fortifying Enterprise Digital Security Against Hackers Weaponizing AI
Security
DevOps on cloud
Optimizing Cost with DevOps on the Cloud
Development
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Mining > Voodoo Spectrum of Machine Learning and Data Sets
Business IntelligenceData Mining

Voodoo Spectrum of Machine Learning and Data Sets

Editor SDC
Last updated: 2010/06/01 at 6:51 AM
Editor SDC
3 Min Read
SHARE
- Advertisement -

I used to be very gung-ho about machine learning approaches to trading but I’m less so now. You have to understand that that there is a spectrum of alpha sources, from very specific structured arbitrage opportunities -> to stat arb -> to just voodoo nonsense.

As history goes on, hedge funds and other large players are absorbing the alpha from left to right. Having squeezed the pure arbs (ADR vs underlying, ETF vs components, mergers, currency triangles, etc) they then became hungry again and moved to stat arb (momentum, correlated pairs, regression analysis, news sentiment, etc). But now even the big stat arb strategies are running dry so people go further, chasing mirages (nonlinear regression, causality inference in large data sets, etc).

- Advertisement -
In modeling the market, it’s best to start with as much structure as possible before moving on to more amorphous statistical strategies. If you have to use statistical machine learning, encode as much trading domain knowledge as possible with specific distance/neighborhood metrics, linearity, variable importance weightings, hierarchy, low-dimensional factors, etc.
It’s good to have a heuristic feel for the …

More Read

machine learning seo

7 Mistakes to Avoid When Using Machine Learning for SEO

Use this Strategic Approach to Maximize Your Data’s Value
Machine Learning is Invaluable for Mobile App Testing Automation
Top 8 Machine Learning Development Companies in 2022
Machine Learning Leads to Huge Breakthroughs in Trading


I used to be very gung-ho about machine learning approaches to trading but I’m less so now. You have to understand that that there is a spectrum of alpha sources, from very specific structured arbitrage opportunities -> to stat arb -> to just voodoo nonsense.

As history goes on, hedge funds and other large players are absorbing the alpha from left to right. Having squeezed the pure arbs (ADR vs underlying, ETF vs components, mergers, currency triangles, etc) they then became hungry again and moved to stat arb (momentum, correlated pairs, regression analysis, news sentiment, etc). But now even the big stat arb strategies are running dry so people go further, chasing mirages (nonlinear regression, causality inference in large data sets, etc).

In modeling the market, it’s best to start with as much structure as possible before moving on to more amorphous statistical strategies. If you have to use statistical machine learning, encode as much trading domain knowledge as possible with specific distance/neighborhood metrics, linearity, variable importance weightings, hierarchy, low-dimensional factors, etc.
It’s good to have a heuristic feel for the danger/flexibility/noise sensitivity (synonyms) of each statistical learning tool. I roughly have this spectrum in my head:
Very specific, structured, safe
Optimize 1 parameter, require crossvalidation
↓
Optimize 2 parameters, require crossvalidation
↓
Optimize parameters with too little data, require regularization
↓
Extrapolation
↓
Nonlinear (SVM, tree bagging, etc)
↓
Higher-order variable dependencies
↓
Variable selection
↓
Structure learning
Very general, dangerous in noise, voodoo
This diagram is worth expanding. If anyone has any suggestions, please leave them.

- Advertisement -
TAGGED: data sets, machine learning, modeling
Editor SDC June 1, 2010
Share this Article
Facebook Twitter Pinterest LinkedIn
Share
- Advertisement -

Follow us on Facebook

Latest News

cloud-centric companies using network relocation
Cloud-Centric Companies Discover Benefits & Pitfalls of Network Relocation
Cloud Computing
construction analytics
5 Benefits of Analytics to Manage Commercial Construction
Analytics
database compliance guide
Four Strategies For Effective Database Compliance
Data Management
Digital Security From Weaponized AI
Fortifying Enterprise Digital Security Against Hackers Weaponizing AI
Security

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

machine learning seo
Machine Learning

7 Mistakes to Avoid When Using Machine Learning for SEO

6 Min Read
analyzing big data for its quality and value
Big Data

Use this Strategic Approach to Maximize Your Data’s Value

6 Min Read
machine learning helps with the testing process for mobile app development
Machine Learning

Machine Learning is Invaluable for Mobile App Testing Automation

9 Min Read
companies use AI to improve their testing strategies
Machine Learning

Top 8 Machine Learning Development Companies in 2022

9 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

giveaway chatbots
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive
ai in ecommerce
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?