By using this site, you agree to the Privacy Policy and Terms of Use.
Accept
SmartData Collective
  • Analytics
    AnalyticsShow More
    data science anayst
    Growing Demand for Data Science & Data Analyst Roles
    6 Min Read
    predictive analytics in dropshipping
    Predictive Analytics Helps New Dropshipping Businesses Thrive
    12 Min Read
    data-driven approach in healthcare
    The Importance of Data-Driven Approaches to Improving Healthcare in Rural Areas
    6 Min Read
    analytics for tax compliance
    Analytics Changes the Calculus of Business Tax Compliance
    8 Min Read
    big data analytics in gaming
    The Role of Big Data Analytics in Gaming
    10 Min Read
  • Big Data
  • BI
  • Exclusive
  • IT
  • Marketing
  • Software
Search
© 2008-23 SmartData Collective. All Rights Reserved.
Reading: Powering Personalized Medicine with Hadoop and Apache Spark
Share
Notification Show More
Latest News
SMEs Use AI-Driven Financial Software for Greater Efficiency
Artificial Intelligence
data security in big data age
6 Reasons to Boost Data Security Plan in the Age of Big Data
Big Data
data science anayst
Growing Demand for Data Science & Data Analyst Roles
Data Science
ai software development
Key Strategies to Develop AI Software Cost-Effectively
Artificial Intelligence
ai in omnichannel marketing
AI is Driving Huge Changes in Omnichannel Marketing
Artificial Intelligence
Aa
SmartData Collective
Aa
Search
  • About
  • Help
  • Privacy
Follow US
© 2008-23 SmartData Collective. All Rights Reserved.
SmartData Collective > Big Data > Data Mining > Powering Personalized Medicine with Hadoop and Apache Spark
Data Mining

Powering Personalized Medicine with Hadoop and Apache Spark

kingmesal
Last updated: 2015/11/23 at 7:13 AM
kingmesal
6 Min Read
Image
SHARE

Image

Image

The Obama administration plans to dedicate $215 million in the 2016 budget to the “Precision Medicine Initiative.” This project centers on the collection and analysis of data from a projected million volunteers. Funding includes $130 million for the National Institutes of Health (NIH) to collect the data for analysis; $10 million to the U.S. Food and Drug Administration (FDA) to architect databases; and $5 million to protect the privacy of the collected data.

Personalized Medicine Data Analytics

More Read

data mining

Data Mining Technology Helps Online Brands Optimize Their Branding

Can Data Mining Aid with Off-Page SEO Strategies?
3 Data Mining Tips for Companies Trying to Understand their Customers
5 Data Mining Tips to Leverage the Benefits of Surveys
Perform Data Mining With Web Scrapers to Track Prices

Under the 2016 budget, the National Cancer Institute (NCI) will receive $70 million for a pilot project that will investigate precision treatments for cancer. In this clinical trial, patients will receive targeted treatments based on the genetic abnormalities in their tumors, regardless of the type of cancer they have.

Personalized medicine is well underway in cancer treatment programs. Genetic profiling of tumors is being used to develop treatment plans that can best attack the cancer, while also reducing the severity of treatment side effects. The National Cancer Institute, noting that “cancer is a disease of the genome,” states that Oncology is already well along the path to precision medicine with “many genetically targeted therapies currently available to cancer patients, and many more are expected to become available in the near future.”

Personalized Medicine is also showing promise in cardiovascular and infectious disease treatment and diagnostics.

Personalized Medicine in the Marketplace

A new research report by Kelly Scientific Publications puts the Personalized Medicine marketplace at $60 billion by 2019, up from its current worth of $42 billion. The market includes companion diagnostics and targeted therapeutics.

The research shows that personalized therapeutics “will be more specific and effective thereby giving pharma/biotech companies a significant advantage to recuperate R&D costs. Personalized medicine will reduce the frequency of adverse drug reactions and therefore have a dramatic impact on health economics. Developmental and diagnostic companies will benefit from lower discovery and commercialization costs and more specific market subtypes.”

Hadoop’s Role in Personalized Medicine

The amount of data generated by diagnostic testing is enormous. In 2011 the U.S. healthcare system alone reached 150 exabytes (150 billion gigabytes), and IDC predicts that U.S. healthcare data will grow to 2,314 exabytes by 2020.

The cost of storing, securing, and making this data accessible to analysts is a central issue in the development of Personalized Medicine. Much of the data is semi-structured or unstructured. A traditional data warehouse and relational database architecture is obviously not the best choice for working with huge, mixed format data sets. Apache Hadoop’s HDFS storage and MapReduce computation system enables complex analytics across large sets of structured, semi-structured, and unstructured data. Apache Spark, a data processing platform for Hadoop, brings improved performance to data analysis and increased abilities to quickly extract intelligence from huge, dissimilar data sets.

Supporting projects developed by Hadoop vendors and community members greatly enhance Hadoop’s Big Data analytics capabilities, adding increased abilities to pull actionable insights from clinical, non-clinical, and genomic information. Hadoop already delivers high performance on commodity servers, providing cost-effective storage of vast data volumes. But teamed with Apache Spark, which supports rapid, large-scale data processing, Hadoop is the best choice to power Personalized Medicine.

Sparking Fresh Potential for Hadoop

Apache Spark’s speed and simplicity are precisely the attributes that are needed in medical diagnostics. Spark is a general-purpose engine for large-scale data processing that offers distributed, in-memory data processing speed at scale.

Apache Spark is written in Scala, a functional programming language. Rapid application development is enabled with easily understandable programming APIs for Scala, Java, and Python. Code can be reused across batch, interactive, and streaming applications.

The Spark stack includes:

  • Spark SQL, allowing relational queries expressed in SQL, HiveQL, or Scala to be executed using Spark. Supported data sources include Parquet files, JSON datasets, or Apache Hive data stores.
  • Spark Streaming, which provides fault-tolerant stream processing of live data stream from many sources.
  • MLlib (Machine Learning Library), a library of Machine Learning algorithms.
  • GraphX for graph-analytics of data sets from sources such as social networks.

But it is Spark’s base platform abilities to deliver real-time analytics across diverse datasets that will most directly speed the development of Personalized Medicine. Clinicians agree that a critical aspect of targeting therapies to patients will involve the combination of data collected from the patient via biosensors—“smart” patches, mobile technology, or fixed devices—which would then often be combined with diagnostic data sets (to determine the potential outcome of patient data), and analyzed in real time.

Returned data about the patient’s condition can then also be monitored in real time by caregivers or healthcare providers, and fast decisions can be made to address any developing problems. The hope is that rapid response will improve patient outcomes while also reducing medical care expenditures that grow as disease progresses.

kingmesal November 23, 2015
Share this Article
Facebook Twitter Pinterest LinkedIn
Share

Follow us on Facebook

Latest News

SMEs Use AI-Driven Financial Software for Greater Efficiency
Artificial Intelligence
data security in big data age
6 Reasons to Boost Data Security Plan in the Age of Big Data
Big Data
data science anayst
Growing Demand for Data Science & Data Analyst Roles
Data Science
ai software development
Key Strategies to Develop AI Software Cost-Effectively
Artificial Intelligence

Stay Connected

1.2k Followers Like
33.7k Followers Follow
222 Followers Pin

You Might also Like

data mining
Data Mining

Data Mining Technology Helps Online Brands Optimize Their Branding

7 Min Read
data mining helps with offsite SEO
Data Mining

Can Data Mining Aid with Off-Page SEO Strategies?

10 Min Read
using data mining to learn more about customers
Big Data

3 Data Mining Tips for Companies Trying to Understand their Customers

6 Min Read
surveys data
Data Mining

5 Data Mining Tips to Leverage the Benefits of Surveys

11 Min Read

SmartData Collective is one of the largest & trusted community covering technical content about Big Data, BI, Cloud, Analytics, Artificial Intelligence, IoT & more.

ai in ecommerce
Artificial Intelligence for eCommerce: A Closer Look
Artificial Intelligence
AI chatbots
AI Chatbots Can Help Retailers Convert Live Broadcast Viewers into Sales!
Chatbots

Quick Link

  • About
  • Contact
  • Privacy
Follow US

© 2008-23 SmartData Collective. All Rights Reserved.

Removed from reading list

Undo
Go to mobile version
Welcome Back!

Sign in to your account

Lost your password?