Cookies help us display personalized product recommendations and ensure you have great shopping experience.

By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    data analytics and truck accident claims
    How Data Analytics Reduces Truck Accidents and Speeds Up Claims
    7 Min Read
    predictive analytics for interior designers
    Interior Designers Boost Profits with Predictive Analytics
    8 Min Read
    image fx (67)
    Improving LinkedIn Ad Strategies with Data Analytics
    9 Min Read
    big data and remote work
    Data Helps Speech-Language Pathologists Deliver Better Results
    6 Min Read
    data driven insights
    How Data-Driven Insights Are Addressing Gaps in Patient Communication and Equity
    8 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: Physicists, models, and the credit crisis, ctd.
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Mining > Physicists, models, and the credit crisis, ctd.
Data MiningPredictive Analytics

Physicists, models, and the credit crisis, ctd.

DavidMSmith
DavidMSmith
5 Min Read
SHARE

Allen Engelhardt, physicist and former quant, provides a thoughtful response to my post the other day, where I asked, “Do physicists and engineers get similar [statistical] training?”

“Of course we do”, says Allen. He makes the point, though, that statisticians and physicists have different world-views.  A physicist will “(first) try to understand the model … He will run thought experiments. Then he might run some statistical tests against the data.”  I agree that’s a different process than that for a statistician, where in practice the data often come first and then, after data cleaning and exploratory analysis, a model is refined. 

I’d go so far as to say that even the word “model” means different things in the two disciplines.  In physics a model is a “true” representation of a reality, invariant and concrete … at least until some errant data point is discovered and a new model can be proposed, tested, and accepted. I use “true” there in quotes to reflect that scientific process: Newton’s Three Laws were “true” reflections of the physical world until relativity called for a new “true” model.  For a statistician in most disciplines, though, a model is merely a useful summariza…

More Read

Mr Obama, smarten these systems!
Hardcoding + procedural code = bad news
Adventures in Data Profiling (Part 1)
Researchers mine millions of metaphors through computer-based techniques
Managing Details in Long-Range Planning

Allen Engelhardt, physicist and former quant, provides a thoughtful response to my post the other day, where I asked, “Do physicists and engineers get similar [statistical] training?”

“Of course we do”, says Allen. He makes the point, though, that statisticians and physicists have different world-views.  A physicist will “(first) try to understand the model … He will run thought experiments. Then he might run some statistical tests against the data.”  I agree that’s a different process than that for a statistician, where in practice the data often come first and then, after data cleaning and exploratory analysis, a model is refined. 

I’d go so far as to say that even the word “model” means different things in the two disciplines.  In physics a model is a “true” representation of a reality, invariant and concrete … at least until some errant data point is discovered and a new model can be proposed, tested, and accepted. I use “true” there in quotes to reflect that scientific process: Newton’s Three Laws were “true” reflections of the physical world until relativity called for a new “true” model.  For a statistician in most disciplines, though, a model is merely a useful summarization of noisy data. It reflects not an invariant truth about the underlying process that generated data, but a tool for identifying important effects and, sometimes, to make predictions.

I think we’re both in agreement that the problem isn’t the models themselves, or even the estimates from the models, but how those estimates were used and — crucially — by whom. Says Allen: “We knew and understood that the models were not valid on the tails, but there was no volume of trading on the tails, so it wasn’t very interesting.” That’s fine and dandy for the high-volatility trading group, where only consequences were the daily ups and downs within that one group. The danger came only when that model became a component of a VaR statistic reported to the upper management from the risk group, when the nuance of “not valid in the tails” was lost when it came to assigning capital allocation.

Allen goes on to say “Society is often expected to pick up the bill for tail effects… the cost of prevention may be bigger than the cost of fixing it.” Perhaps. But I certainly hope it remains in a financial institutions selfish self-interest (and I mean that only in a positive way) to avoid bankruptcy. This takes us into a longer discussion about the conflict between an individual banker’s short-term self-interest, and the long-term best interests of a bank. (I think Surowiecki had a column on this topic recently, but I can’t find it now.). But if there’s any lesson to be learned here, I hope it results in practices that lead these banks to understand the limitations of their modeling practices so that such failures can be avoided next time.

TAGGED:modeling
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

data analytics and truck accident claims
How Data Analytics Reduces Truck Accidents and Speeds Up Claims
Analytics Big Data Exclusive
predictive analytics for interior designers
Interior Designers Boost Profits with Predictive Analytics
Analytics Exclusive Predictive Analytics
big data and cybercrime
Stopping Lateral Movement in a Data-Heavy, Edge-First World
Big Data Exclusive
AI and data mining
What the Rise of AI Web Scrapers Means for Data Teams
Artificial Intelligence Big Data Exclusive

Stay Connected

1.2kFollowersLike
33.7kFollowersFollow
222FollowersPin

You Might also Like

using geographic data in analysis
Uncategorized

Using Geographic Data

8 Min Read

Why Learn R? It’s the language of Statistics

3 Min Read
Image
Business IntelligenceCRMMarket ResearchSocial DataSocial Media AnalyticsUnstructured Data

Big Data Social Intelligence: Five Reasons Corporations Need It

10 Min Read

Why is Modeling Foundational to Performance Management?

3 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI and chatbots
Chatbots and SEO: How Can Chatbots Improve Your SEO Ranking?
Artificial Intelligence Chatbots Exclusive
ai chatbot
The Art of Conversation: Enhancing Chatbots with Advanced AI Prompts
Chatbots

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-25 SmartData Collective. All Rights Reserved.
Go to mobile version
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?