Voodoo Spectrum of Machine Learning and Data Sets
587 Views
I used to be very gung-ho about machine learning approaches to trading but I’m less so now. You have to understand that that there is a spectrum of alpha sources, from very specific structured arbitrage opportunities -> to stat arb -> to just voodoo nonsense.
As history goes on, hedge funds and other large players are absorbing the alpha from left to right. Having squeezed the pure arbs (ADR vs underlying, ETF vs components, mergers, currency triangles, etc) they then became hungry again and moved to stat arb (momentum, correlated pairs, regression analysis, news sentiment, etc). But now even the big stat arb strategies are running dry so people go further, chasing mirages (nonlinear regression, causality inference in large data sets, etc).
In modeling the market, it’s best to start with as much structure as possible before moving on to more amorphous statistical strategies. If you have to use statistical machine learning, encode as much trading domain knowledge as possible with specific distance/neighborhood metrics, linearity, variable importance weightings, hierarchy, low-dimensional factors, etc.
It’s good to have a heuristic feel for the …
I used to be very gung-ho about machine learning approaches to trading but I’m less so now. You have to understand that that there is a spectrum of alpha sources, from very specific structured arbitrage opportunities -> to stat arb -> to just voodoo nonsense.
As history goes on, hedge funds and other large players are absorbing the alpha from left to right. Having squeezed the pure arbs (ADR vs underlying, ETF vs components, mergers, currency triangles, etc) they then became hungry again and moved to stat arb (momentum, correlated pairs, regression analysis, news sentiment, etc). But now even the big stat arb strategies are running dry so people go further, chasing mirages (nonlinear regression, causality inference in large data sets, etc).
In modeling the market, it’s best to start with as much structure as possible before moving on to more amorphous statistical strategies. If you have to use statistical machine learning, encode as much trading domain knowledge as possible with specific distance/neighborhood metrics, linearity, variable importance weightings, hierarchy, low-dimensional factors, etc.
It’s good to have a heuristic feel for the danger/flexibility/noise sensitivity (synonyms) of each statistical learning tool. I roughly have this spectrum in my head:
Very specific, structured, safe
Optimize 1 parameter, require crossvalidation↓Optimize 2 parameters, require crossvalidation↓Optimize parameters with too little data, require regularization↓Extrapolation↓Nonlinear (SVM, tree bagging, etc)↓Higher-order variable dependencies↓Variable selection↓Structure learning
Very general, dangerous in noise, voodoo
This diagram is worth expanding. If anyone has any suggestions, please leave them.
Trending Now
1
Artificial Intelligence
93 shares3,371 views
93 shares3,371 views
Artificial Intelligence Chatbots Could Make Your Doctors Obsolete
Ryan Kade - January 12, 20192
Big Data
63 shares2,685 views
63 shares2,685 views
6 Ways Big Data Can Improve eCommerce For Your Business
Sean Mallon - January 11, 20193
Big Data
1,759 views
1,759 views
Is Your Data At Risk? Here’s How To Protect Your Information Online
Rehan Ijaz - January 11, 20194
Big Data
51 shares2,478 views
51 shares2,478 views
5 Genius App Data Monetization Strategies That Can Make You Bank
Annie Qureshi - January 10, 20195
Big Data
21 shares1,866 views
21 shares1,866 views
Fascinating Ways Big Data and DMPs Are Disrupting Online Publishing
Annie Qureshi - January 10, 20196
Analytics
37 shares3,313 views
37 shares3,313 views
How To Build A High-Performing Data Analytics Platform In The Cloud
Namita Awasthi - January 9, 20197
Big Data
66 shares2,465 views
66 shares2,465 views
You must log in to post a comment.