By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    predictive analytics in dropshipping
    Predictive Analytics Helps New Dropshipping Businesses Thrive
    12 Min Read
    data-driven approach in healthcare
    The Importance of Data-Driven Approaches to Improving Healthcare in Rural Areas
    6 Min Read
    analytics for tax compliance
    Analytics Changes the Calculus of Business Tax Compliance
    8 Min Read
    big data analytics in gaming
    The Role of Big Data Analytics in Gaming
    10 Min Read
    analyst,women,looking,at,kpi,data,on,computer,screen
    Promising Benefits of Predictive Analytics in Asset Management
    11 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: The Dire Consequences of Analytics Gone Wrong: Ruining Kids’ Futures
Share
Notification Show More
Latest News
ai software development
Key Strategies to Develop AI Software Cost-Effectively
Artificial Intelligence
ai in omnichannel marketing
AI is Driving Huge Changes in Omnichannel Marketing
Artificial Intelligence
ai for small business tax planning
Maximize Tax Deductions as a Business Owner with AI
Artificial Intelligence
ai in marketing with 3D rendering
Marketers Use AI to Take Advantage of 3D Rendering
Artificial Intelligence
How Big Data Is Transforming the Maritime Industry
How Big Data Is Transforming the Maritime Industry
Big Data
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Analytics > Predictive Analytics > The Dire Consequences of Analytics Gone Wrong: Ruining Kids’ Futures
AnalyticsCommentaryPredictive Analytics

The Dire Consequences of Analytics Gone Wrong: Ruining Kids’ Futures

BillFranks
Last updated: 2011/11/12 at 12:28 AM
BillFranks
9 Min Read
SHARE

We always hear about the benefits of analytics done correctly and used well. What we don’t hear as often are the dire consequences of analytics done poorly and used inappropriately.

We always hear about the benefits of analytics done correctly and used well. What we don’t hear as often are the dire consequences of analytics done poorly and used inappropriately. There are occasions where people’s lives can literally be ruined because of an analysis that is poorly designed or incorrectly interpreted.

The story I will be telling here is a true story that happened in my neighborhood a few weeks ago. While I don’t have any personal stake in the school, teachers, or students in this case, the story has eaten at me ever since I heard about it. Anyone who understands analytics will likely be equally appalled.

The local schools in my area are the best in the state and there are a lot of very smart, motivated kids that attend the schools. One of our neighbors has a very good student who is a senior in high school. We’ll call her “Sue”. Sue has always had good grades, she has never gotten into trouble, and she is taking a variety of challenging classes so that she can get into a good college next year.

More Read

data analytics reveals the benefits of MBA

Data Analytics Technology Proves Benefits of an MBA

Top 8 Machine Learning Development Companies in 2022
3 Ways You Can Use Software to Improve Your Trading Decisions
4 Fascinating Big Data Educational Opportunities For Kids
Big Data Technology Is Crucial To Your Educational Pursuits

Sue is taking an Advanced Placement course through the English department. This summer, her class was asked to write a paper as an extra assignment before school started. So far, so good. A new teacher in the department brought up the point that the school wasn’t using any plagiarism checking software and that they really should start using it. Again, so far, so good. Then things went horribly wrong…

Editor’s note: Bill Franks is an employee of Teradata. Teradata is a sponsor of The Smart Data Collective.

The results of the plagiarism analysis showed that every single student in the class had cheated. All failed the assignment and initially the school planned to note the offense on each student’s official transcript. Goodbye, good schools! One might think that the school would question whether there were issues with either the software’s analysis or how the results were interpreted. After all, 100% of a group of “A” students with no history of trouble was flagged. The school did not question the results. The parents did, however. After some digging, the findings are quite troubling.

First, the software by default looks for any phrases of three words or more that match between two submitted papers. Each “offense” of a “copied” three-word phrase is tagged. Get tagged too many times, and you’re identified as a cheater. Let’s think about this criteria applied blindly without further thought. Assume students are writing about Tolstoy’s War and Peace. Two students start a sentence with “Tolstoy said that…” or “The meaning of…” or “The book refers to…”. They are now guilty of plagiarism. The software assumes nobody could have such phrases in common without copying from one another. Such tags are useful as a starting point. But, to be applied correctly, someone needs to review the papers and validate if any of the phrases really appear to be copied or just innocent matches like the examples. Nobody did that.

It gets worse. Part of the assignment was to provide definitions for a number of terms. Students were told that using dictionaries was allowed. Sue pasted definitions from a dictionary into her paper. It ends up that another student she barely knows had many of the same definitions. That, the parents were told, was the smoking gun. Clearly, the two girls had copied from each other. What wasn’t considered is that there are perhaps a handful of well-known dictionaries and dozens of kids in the class. Isn’t it reasonable that a couple of them may have used the same dictionary? Not according the English department. The only way the girls could both have a definition from Dictionary.com is if they copied from one another.

Sue’s parents managed to get the offense removed from her transcript, but not the 0% grade for the assignment. The teachers in the department are now unwilling to provide college recommendations for any of the students since they are all now viewed as cheaters. Some of these kids could quite literally have their futures stolen due to the misuse of analytics by a well-meaning school.

The point of this story is that analytics can’t be blindly followed. Results must be put in context. Unexpected results should be further examined. The English department’s personnel don’t understand what the plagiarism software is doing or why. They just assume that if the software says “potential cheater”, then it must be true. In this case, a horrible injustice has been done to some smart, honest kids.

As analytics become more pervasive and more and more tools make complex analytics “easy” and “push button”, we’ll continue to see examples like this. Those of us in the field have a responsibility to educate others of the limits and appropriate uses of analytics. In this case, the right course of action was to look at each student’s paper and interview them to get their explanation of the flagged passages. Also, the settings on the software should have been validated. To any fair-minded person willing to think logically, Sue did not cheat. To an English department using analytics software they don’t understand, she did. After all, the software found some red flags. What else do they need to know? The kids in the class may not get into a top college after all their years of work for they now have the brand of “Cheater” unfairly stamped on their chests.

Just like any other tool, analytics can be very powerful and helpful when used in the correct way. Analytics can also do major damage when used by those who don’t understand it and don’t use it correctly. In business, we’ve all seen cases where statistics or figures are shown to an executive without full context or needed caveats. After seeing the data, the executive takes some drastic actions that in some cases are not productive or required. It is imperative to ensure that analysis results are used correctly. Let’s review a few practices that should be standard when producing and using analytics. If these principles had been applied, this story would not have happened.

  • Make sure that someone in the process fully understands the analysis being done, its strengths, and its weaknesses. Not everyone has to understand the gory details, but someone must.
  • Make sure that the various settings and options utilized in a process have been chosen with good reason. Don’t just assume the default settings are best for your situation.
  • When unexpected results are found, investigate further and ask critical questions before jumping to conclusions. No algorithm or software package is omniscient. “Because the software said so” is not an acceptable answer.
  • When provided with additional facts or data that contradict an analysis, give them consideration. The goal should be the right answer, not your initial answer.
  • Be careful to enable people to execute only the types of analysis they are prepared to execute correctly. It is easy for people to be in over their head and not even realize it. The results, as we’ve seen, can be devastating.

TAGGED: education, software, testing
BillFranks November 12, 2011
Share this Article
Facebook Twitter Pinterest LinkedIn
Share
By BillFranks
Follow:
Bill Franks is Chief Analytics Officer for The International Institute For Analytics (IIA). Franks is also the author of Taming The Big Data Tidal Wave and The Analytics Revolution. His work has spanned clients in a variety of industries for companies ranging in size from Fortune 100 companies to small non-profit organizations. You can learn more at http://www.bill-franks.com.

Follow us on Facebook

Latest News

ai software development
Key Strategies to Develop AI Software Cost-Effectively
Artificial Intelligence
ai in omnichannel marketing
AI is Driving Huge Changes in Omnichannel Marketing
Artificial Intelligence
ai for small business tax planning
Maximize Tax Deductions as a Business Owner with AI
Artificial Intelligence
ai in marketing with 3D rendering
Marketers Use AI to Take Advantage of 3D Rendering
Artificial Intelligence

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

data analytics reveals the benefits of MBA
Analytics

Data Analytics Technology Proves Benefits of an MBA

9 Min Read
companies use AI to improve their testing strategies
Machine Learning

Top 8 Machine Learning Development Companies in 2022

9 Min Read
trading software helps
Software

3 Ways You Can Use Software to Improve Your Trading Decisions

7 Min Read
big data and high-tech education
Big DataExclusive

4 Fascinating Big Data Educational Opportunities For Kids

9 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai in ecommerce
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence
ai is improving the safety of cars
From Bolts to Bots: How AI Is Fortifying the Automotive Industry
Artificial Intelligence

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?