We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
    From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
    12 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: Weirdness is the “Curse of Dimensionality”
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > Predictive Analytics > Weirdness is the “Curse of Dimensionality”
Predictive Analytics

Weirdness is the “Curse of Dimensionality”

Editor SDC
Editor SDC
3 Min Read
Weirdness is the "Curse of Dimensionality"
Illustration generated with FLUX.2 [klein 4B] via Cloudflare Workers AI.
SHARE

I read the following well-written section in “The Elements of Statistical Learning” by Friedman, Hastie, & Tibshirani. This curse of dimensionality is profound. I am assuming you are familiar with the k-nearest neighbors classifier, which is used to introduce the idea.

This sparked ideas in two contexts: 1) human personalities and 2) trading.
1) If you think about human personalities being a combination of real-valued variables (ex. introversion-extroversion, affectionate-cold, optimistic-depressed, driven-apathetic, etc) then this basically says that everyone is weird. Let’s say there were only 10 personality traits, then (following the unit 10D-cube example) 90% of people are located over 80% away from the center toward the fringe.
One caveat- this assumes personality traits are uniformly distributed, but due to peer pressure this is probably not the case.
2) You can’t look into the past for a setup identical to what you are currently seeing. Also, the more data streams you feed into a system, and depending on the learner you are using (ex. k-NN), the more every time slice will look absolutely unique and the harder it will be to get a historical data set large enough to teach any trend.

Feel free to add your thoughts, this seems to be a very important result so I’m sure there are more conclusions that can be drawn.

Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

Flat editorial illustration: The article connects downstream report failures, data-quality gaps, and conflicting metric definitio
Signs Your Data Architecture Needs an Outside Review
Big Data Exclusive
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Data-Driven Pultrusion: Best GFRP Rebar Machines & ROI
Big Data Exclusive
Flat editorial illustration: The article contrasts tools for gathering fresh interviews with tools for analyzing existing qualita
10 AI Market Research Tools That Turn Qualitative Data into Insights
Artificial Intelligence Big Data Exclusive
Flat editorial illustration: The article compares proxy MCP servers by what an AI agent can do through them: manage proxy account
Proxy MCP Servers: 7 Options and What Each Lets an AI Agent Do
Artificial Intelligence Big Data Data Collection

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

Great Analytics Vendors: 5 Must-Have Traits
AnalyticsBig DataBusiness RulesData MiningData VisualizationJobsKnowledge ManagementMarket ResearchModelingPolicy and GovernancePredictive AnalyticsSentiment AnalyticsSocial DataSocial Media AnalyticsText AnalyticsUnstructured DataWeb Analytics

Great Analytics Vendors: 5 Must-Have Traits

5 Min Read
chatgpt image jul 21, 2026, 04 34 30 pm
AnalyticsExclusivePredictive Analytics

4 Core Benefits of Predictive Maintenance after Vibration Analysis

10 Min Read
How is Performance Management like Multi-wavelength Astronomy?
Business IntelligenceData MiningPredictive Analytics

How is Performance Management like Multi-wavelength Astronomy?

4 Min Read
Spy Craft and Disease: Big Data's Impact on the Global Stage
Predictive Analytics

Spy Craft and Disease: Big Data’s Impact on the Global Stage

6 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai chatbot
How AI Website Chatbots Improve Customer Support and Lead Generation
Chatbots Exclusive
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
Chatbots

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?