We use cookies, including third-party cookies from Google to serve personalized ads through AdSense, to operate this site and understand how it is used. By continuing to browse, you accept this use. See our Privacy Policy and Terms of Use for details, including how to opt out of personalized advertising.
Accept
SmartData CollectiveSmartData Collective
  • Analytics
    AnalyticsShow More
    Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
    Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
    23 Min Read
    Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
    From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
    12 Min Read
    Flat editorial illustration: The article contrasts passive follower totals with a behavioral path from attention and trust to cli
    Creator Analytics Beyond Follower Count: Seven Metrics That Measure Audience Behavior
    11 Min Read
    Flat editorial illustration: The article connects AI and marketing analytics with managerial judgment grounded in consumer behavi
    How an Online MBA in Marketing Connects AI and Data Analytics
    11 Min Read
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers -- AI-generated illustration
    What Kind of Problem-Solving Distinguishes Data Analysts From Software Engineers
    7 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-25 SmartData Collective. All Rights Reserved.
Reading: How Do We Know Which Research to Trust? Rachel Kennedy
Share
Notification
Font ResizerAa
SmartData CollectiveSmartData Collective
Font ResizerAa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > How Do We Know Which Research to Trust? Rachel Kennedy
Analytics

How Do We Know Which Research to Trust? Rachel Kennedy

AnniePettit
AnniePettit
6 Min Read
How Do We Know Which Research to Trust? Rachel Kennedy
Illustration generated with FLUX.2 [klein 4B] via Cloudflare Workers AI.
SHARE

Welcome to this series of live blogs from the IJMR Research Methods Forum in London. Any errors, omissions, or silly side comments are my own.

09.25 Keynote address – How do we know which research to trust?
Rachel Kennedy, Associate Professor, Ehrenberg-Bass Institute, University of South Australia

[Great presentation!]

  • Would you trust EG or eyetracking? The neuroscience technologies look cool. It’s easy to be bamboozled. How do you know what’s right for your research?
  • Galvic skin measurements have been around since 1888 but we still don’t completely know what it means or what to do with it
  • “This is old therefore it’s good” “This is new, therefore it is better”  Willima Ralph Inge
  • #1 We need to get repeatable results among different researchers
  • #2 It must predict in market behaviours of interest, not interest, not liking, but sales
  • #3 Providers must be transparent – what we measure, how it is being analyzed, and what we expect to see in given conditions
  • #4 It should be actionable
  • We’d like to know that the research in journals is the trustworthy research, we prefer peer review journals. BUT we can’t always trust is.
  • 10% of psychologists report they have falsified data – selectively reporting only studies that worked, don’t report all independent measures, excluding post-hoc data
  • Most results from the TOP journals can’t be reproduced
  • Science has a lovely way of correcting itself over time. You can’t accept a single validation study. You need lots of validation.
  • Biometics are exciting because they can give you an objective measure of truth – that’s the gut feel. But really?
  • Case study 1: They put a salmon into an fMRI. It responded to the stimuli in the scanner. But the fish was dead. So what then when humans respond to ads? Are the researcher’s data being corrected for false positives?
  • Case study 2:  Study took the same data and put it through two different software systems. They got two different results. [Yikes, I hope SPSS and SAS don’t do this. :) ]
  • Case study 3: What test-retest reliability would you want to see ? [95% please] With fMRI, only one third to one half of the same neural activity occurs. [sooooo, chance]
  • Add one complex brain (experience, IQ) to one complex marketing stimuli (sound, colour, movement), how can you get any reliability?
  • Hemispheric asymmetry – some favour the left side, some favour the right. Do you balance your research responders on this? Should you? Does it matter? Asymmetry depends on time of day and time of year. How do you balance on that? We just don’t know which matter.
  • How do you analyze evoked potential? Patterns of response over the time of an event (an ad).
  • Skin conductance claims says 98% rank order correlations [as soon as you said rank ordered, I could tell the researcher was hiding something.]. This study can’t be found now, company doesn’t exist anymore.
  • In one study, at best, only biometrics OR traditional measure can be correct because they lead to different conclusions. They are measuring very different things.
  • Traditional measures such as pleasure, likeability don’t correlate highly with sales.
  • Don’t take the negativity as we should not do neuroscience, but as know what you’re getting into.
  • What about virtual shopping to understand real life? market share results are different.
  • People are less likely to buy store brand and more likely to buy premium products in virtual testing. The penalities in fake life aren’t the same. Can we calibrate these differences?  People know how to “play” shop.
  • Purchase rates and dollar estimates are inflated. Most (mode) people actually buy only one item. But in virtual environment, most people actually buy three to six items. Maybe they are showing you the average or maybe it’s just wrong. We are missing standard research control. [yeah baby!]
  • It’s really easy to get things completely wrong without rigorous controls.
  • What about a vending maching mini-shop study. How long at the machine, what they look at, how long they look at it. [never thought of that!]
  • Big data doesn’t mean necessarily NOT biased. e.g., facebook fan base can be huge, like Skittles. But that’s a biased base. You need to understand the bias. Facebook skews to heavy brand bias, the people who already like you and don’t need to be convinced. Most people are actually non-buyers, the completely opposite.
  • You must have good research design and the right researchers for the job.
  • Ask suppliers about their validation techniques, what they predict, their knowledge of marketing, their knowledge of market research (e.g., experimental control)
  • What is trustworthy? Well grounded measures, validated technologies, passive/unobtrusive measurement, natural environment [hmmm…. do I hear social media listening research…]
  • [Thank you Rachel for reminding people that we need GOOD research with experimental design not just cool data and neato conclusions]
 
TAGGED:#IJMR2012#MRXIJMRrachel kennedytrust
Share This Article
Facebook Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

Flat editorial illustration: The article is about joining public web signals with internal business metrics so BI teams can inter
Alternative Data in BI: Bringing Public Web Data Into Your Dashboards
Big Data Business Intelligence Exclusive Web Analytics
Flat editorial illustration: A SaaS-editorial scene showing product usage, support patterns, and renewal feedback informing a fle
Using Product Usage Analytics and Support Data to Build a B2B Product Roadmap
Business Intelligence Infographic
From Hiring To Payroll: Managing HR Data Across Business Systems -- AI-generated illustration
From Hiring To Payroll: Managing HR Data Across Business Systems
Big Data Business Intelligence Infographic
Flat editorial illustration: The article is about automating the collection, preparation, and delivery of marketing data across p
From Data to Dashboards: 5 Marketing Reporting Automation Software Picks
Analytics Business Intelligence Exclusive Marketing

Stay Connected

1.2KFollowersLike
33.7KFollowersFollow
222FollowersPin

You Might also Like

If You Want Trust in Washington, Get a Database?
Predictive Analytics

If You Want Trust in Washington, Get a Database?

6 Min Read
Trust is not a checklist
Data Quality

Trust is not a checklist

4 Min Read
Flat editorial illustration: The article’s core relationship is that restoring a refrigeration system does not establish trustwor
Artificial IntelligenceNews

After the Cyberattack, Can You Trust the Machine?

17 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots
How To Get An Award Winning Giveaway Bot
How To Get An Award Winning Giveaway Bot
Big Data Chatbots Exclusive

Quick Link

  • About
  • Contact
  • Privacy
Follow US
© 2008-26 SmartData Collective. All Rights Reserved.
Welcome Back!

Sign in to your account

Username or Email Address
Password

Lost your password?