By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    construction analytics
    5 Benefits of Analytics to Manage Commercial Construction
    5 Min Read
    benefits of data analytics for financial industry
    Fascinating Changes Data Analytics Brings to Finance
    7 Min Read
    analyzing big data for its quality and value
    Use this Strategic Approach to Maximize Your Data’s Value
    6 Min Read
    data-driven seo for product pages
    6 Tips for Using Data Analytics for Product Page SEO
    11 Min Read
    big data analytics in business
    5 Ways to Utilize Data Analytics to Grow Your Business
    6 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Weirdness is the “Curse of Dimensionality”
Share
Notification Show More
Latest News
cloud-centric companies using network relocation
Cloud-Centric Companies Discover Benefits & Pitfalls of Network Relocation
Cloud Computing
construction analytics
5 Benefits of Analytics to Manage Commercial Construction
Analytics
database compliance guide
Four Strategies For Effective Database Compliance
Data Management
Digital Security From Weaponized AI
Fortifying Enterprise Digital Security Against Hackers Weaponizing AI
Security
DevOps on cloud
Optimizing Cost with DevOps on the Cloud
Development
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > Predictive Analytics > Weirdness is the “Curse of Dimensionality”
Predictive Analytics

Weirdness is the “Curse of Dimensionality”

Editor SDC
Last updated: 2009/03/01 at 11:03 PM
Editor SDC
3 Min Read
SHARE
- Advertisement -

I read the following well-written section in “The Elements of Statistical Learning” by Friedman, Hastie, & Tibshirani. This curse of dimensionality is profound. I am assuming you are familiar with the k-nearest neighbors classifier, which is used to introduce the idea.

This sparked ideas in two contexts: 1) human personalities and 2) trading.
1) If you think about human personalities being a combination of real-valued variables (ex. introversion-extroversion, affectionate-cold, optimistic-depressed, driven-apathetic, etc) then this basically says that everyone is weird. Let’s say there were only 10 personality traits, then (following the unit 10D-cube example) 90% of people are located over 80% away from the center toward the fringe.
One caveat- this assumes personality traits are uniformly distributed, but due to peer pressure this is probably not the case.
2) You can’t look into the past for a setup identical to what you are currently seeing. Also, the more data streams you feed into a system, and depending on the learner you are using (ex. k-NN), the more every time slice will look absolutely unique and the harder it will be to get a historical data set large enough to teach an…

- Advertisement -


I read the following well-written section in “The Elements of Statistical Learning” by Friedman, Hastie, & Tibshirani. This curse of dimensionality is profound. I am assuming you are familiar with the k-nearest neighbors classifier, which is used to introduce the idea.

This sparked ideas in two contexts: 1) human personalities and 2) trading.
1) If you think about human personalities being a combination of real-valued variables (ex. introversion-extroversion, affectionate-cold, optimistic-depressed, driven-apathetic, etc) then this basically says that everyone is weird. Let’s say there were only 10 personality traits, then (following the unit 10D-cube example) 90% of people are located over 80% away from the center toward the fringe.
One caveat- this assumes personality traits are uniformly distributed, but due to peer pressure this is probably not the case.
2) You can’t look into the past for a setup identical to what you are currently seeing. Also, the more data streams you feed into a system, and depending on the learner you are using (ex. k-NN), the more every time slice will look absolutely unique and the harder it will be to get a historical data set large enough to teach any trend.

More Read

predictive analytics helps Albanian bitcoin investors

Albanian Bitcoin Investors Tap the Power of Predictive Analytics

Predictive Analytics Improves Trading Decisions as Euro Rebounds
Can Predictive Analytics Help Traders Navigate Bitcoin’s Volatility?
Perks of Predictive Analytics for Businesses Big and Small
How can CIOs Build Business Value with Business Analytics?

Feel free to add your thoughts, this seems to be a very important result so I’m sure there are more conclusions that can be drawn.

Editor SDC March 1, 2009
Share this Article
Facebook Twitter Pinterest LinkedIn
Share
- Advertisement -

Follow us on Facebook

Latest News

cloud-centric companies using network relocation
Cloud-Centric Companies Discover Benefits & Pitfalls of Network Relocation
Cloud Computing
construction analytics
5 Benefits of Analytics to Manage Commercial Construction
Analytics
database compliance guide
Four Strategies For Effective Database Compliance
Data Management
Digital Security From Weaponized AI
Fortifying Enterprise Digital Security Against Hackers Weaponizing AI
Security

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

predictive analytics helps Albanian bitcoin investors
Blockchain

Albanian Bitcoin Investors Tap the Power of Predictive Analytics

9 Min Read
benefits of data analytics for financial management
Predictive Analytics

Predictive Analytics Improves Trading Decisions as Euro Rebounds

10 Min Read
predictive analytics can help bitcoin traders predict future price movements
Blockchain

Can Predictive Analytics Help Traders Navigate Bitcoin’s Volatility?

8 Min Read
predictive analytics
Predictive Analytics

Perks of Predictive Analytics for Businesses Big and Small

6 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots
ai in ecommerce
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?