# An Extensive Glossary of Big Data Terminology

> Big data comes with a lot of new terminology that is sometimes hard to understand. Therefore we have created an ABC of big data that should give some insights. Some of the definitions refer to a corresponding blog post. Big…
> URL: https://www.smartdatacollective.com/extensive-glossary-big-data-terminology/
> Last modified: 2013-07-31T08:00:00+00:00

Big data comes with a lot of new terminology that is sometimes hard to understand. Therefore we have created an ABC of big data that should give some insights. Some of the definitions refer to a corresponding blog post. 
Big data comes with a lot of new terminology that is sometimes hard to understand. Therefore we have created an ABC of big data that should give some insights. Some of the definitions refer to a corresponding blog post. Of course this big data glossary is not 100% complete, so please let us know if there are missing terminology that you would like to see included.

**A**

**Aggregation** – a process of searching, gathering and presenting data
 **Algorithms** – a mathematical formula that can perform certain analyses on data
 **Analytics** – the discovery of insights in data
 **Anomaly detection** – the search for data items in a dataset that do not match a projected pattern or expected behaviour. Anomalies are also called outliers, exceptions, surprises or contaminants and they often provide critical and actionable information.
 **Anonymization** – making data anonymous; removing all data points that could lead to identify a person
 **Application** – computer software that enables a computer to perform a certain task
 **Artificial Intelligence** – developing intelligence machines and software that are capable of perceiving the environment and take corresponding action when required and even learn from those actions.
 

### **B**

**Behavioural Analytics** – analytics that informs about the how, why and what instead of just the who and when. It looks at humanized patterns in the data
 **Big Data Scientist** – someone who is able to develop the algorithms to make sense out of big data
 **Big data startup** – a young company that has developed new big data technology
 **Biometrics** – the identification of humans by their characteristics
 **Brontobytes** – approximately 1000 Yottabytes and the size of the digital universe tomorrow. A Brontobyte contains 27 zeros
 **Business Intelligence** – the theories, methodologies and processes to make data understandable
 

### **C**

**Classification analysis** –  a systematic process for obtaining important and relevant information about data, also meta data called; data about data.
 **Cloud computing** – a distributed computing system over a network used for storing data off-premises
 **Clustering analysis** – the process of identifying objects that are similar to each other and cluster them in order to understand the differences as well as the similarities within the data.
 **Cold data storage** – storing old data that is hardly used on low-power servers. Retrieving the data will take longer
 **Comparative analysis** – it ensures a step-by-step procedure of comparisons and calculations to detect patterns within very large data sets.
 **Complex structured data** – data that are composed of two or more complex, complicated, and interrelated parts that cannot be easily interpreted by structured query languages and tools.
 **Computer generated data** – data generated by computers such as log files
 **Concurrency **– performing and executing multiple tasks and processes at the same time
 **Correlation analysis** – the analysis of data to determine a relationship between variables and whether that relationship is negative (- 1.00) or positive (+1.00).
 **Customer Relationship Management **– managing the sales and business processes, big data will affect CRM strategies
 

### **D**

**Dashboard **– a graphical representation of the analyses performed by the algorithms
 **Data aggregation tools **– the process of transforming scattered data from numerous sources into a single new one.
 **Data analyst **– someone analysing, modelling, cleaning or processing data
 **Database** – a digital collection of data stored via a certain technique
 **Database-as-a-Service** – a database hosted in the cloud on a pay per use basis, for example Amazon Web Services or
 **Database Management System** – collecting, storing and providing access of data
 **Data centre** – a physical location that houses the servers for storing data
 **Data cleansing** – the process of reviewing and revising data in order to delete duplicates, correct errors and provide consistency
 **Data custodian** – someone who is responsible for the technical environment necessary for data storage
 **Data ethical guidelines** – guidelines that help organizations being transparent with their data, ensuring simplicity, security and privacy
 **Data feed** – a stream of data such as a Twitter feed or RSS
 **Data marketplace **– an online environment to buy and sell data sets
 **Data mining** – the process of finding certain patterns or information from data sets
 **Data modelling** – the analysis of data objects using data modelling techniques to create insights from the data
 **Data set **– a collection of data
 **Data virtualization** – a data integration process in order to gain more insights. Usually it involves databases, applications, file systems, websites, big data techniques, etc.)
 **De-identification** – same as anonymization; ensuring a person cannot be identified through the data
 **Discriminant analysis **– cataloguing of the data; distributing data into groups, classes or categories. A statistical analysis used where certain groups or clusters in data are known upfront and that uses that information to derive the classification rule.
 **Distributed File System **– systems that offer simplified, highly available access to storing, analysing and processing data
 **Document Store Databases** – a document-oriented database that is especially designed to store, manage and retrieve documents, also known as semi structured data.
 

### **E**

**Exploratory analysis** – finding patterns within data without standard procedures or methods. It is a means of discovering the data and to find the data sets main characteristics.
 **Exabytes** – approximately 1000 petabytes or 1 billion gigabytes. Today we create one Exabyte of new information globally on a daily basis.
 **Extract, Transform and Load (ETL) **– a process in a database and data warehousing meaning extracting the data from various sources, transforming it to fit operational needs and loading it into the database
 

### **F**

**Failover **– switching automatically to a different server or node should one fail
 **Fault-tolerant design** – a system designed to continue working even if certain parts fail
 

### **G**

**Gamification** – using game elements in a non game context; very useful to create data therefore coined as the friendly scout of big data
 **Graph Databases** – they use graph structures (a finite set of ordered pairs or certain entities), with edges, properties and nodes for data storage. It provides index-free adjacency, meaning that every element is directly linked to its neighbour element.
 **Grid computing** – connecting different computer systems from various location, often via the cloud, to reach a common goal
 

### **H**

**Hadoop** – an open-source framework that is built to enable the process and storage of big data across a distributed file system
 **HBase** –an open source, non-relational, distributed database running in conjunction with Hadoop
 **HDFS** – Hadoop Distributed File System; a distributed file system designed to run on commodity hardware
 **High-Performance-Computing (HPC) **– using supercomputers to solve highly complex and advanced computing problems
 

### **I**

**In-memory **– a database management system stores data on the main memory instead of the disk, resulting is very fast processing, storing and loading of the data
 **Internet of Things **– ordinary devices that are connected to the internet at any time any where via sensors
 

### **J**

**Juridical data compliance**** – **relevant when you use cloud solutions and where the data is stored in a different country or continent. Be aware that data stored in a different country has to oblige to the law in that country.
 

### **K**

**KeyValue Databases** – they store data with a primary key, a uniquely identifiable record, which makes easy and fast to look up. The data stored in a KeyValue is normally some kind of primitive of the programming language.
 

### **L**

**Latency **– a measure of time delayed in a system
 **Legacy system** – an old system, technology or computer system that is not supported any more
 **Load balancing **– distributing workload across multiple computers or servers in order to achieve optimal results and utilization of the system
 **Location data **– GPS data describing a geographical location
 **Log file **– a file automatically created by a computer to record events that occur while operational
 

### **M**

**Machine2Machine data **– two or more machines that are communicating with each other
 **Machine data** – data created by machines via sensors or algorithms
 **Machine learning **– part of artificial intelligence where machines learn from what they are doing and become better over time
 **MapReduce** – a software framework for processing vast amounts of data
 **Massively Parallel Processing (MPP) **– using many different processors (or computers) to perform certain computational tasks at the same time
 **Metadata** – data about data; gives information about what the data is about.
 **MongoDB** – an open-source NoSQL database
 **Multi-Dimensional Databases **– a database optimized for data online analytical processing (OLAP) applications and for data warehousing.
 **MultiValue Databases** – they are a type of NoSQL and multidimensional databases that understand 3 dimensional data directly. They are primarily giant strings that are perfect for manipulating HTML and XML strings directly
 

### **N**

**Natural Language Processing** – a field of computer science involved with interactions between computers and human languages
 **Network analysis** – viewing relationships among the nodes in terms of the network or graph theory, meaning analysing connections between nodes in a network and the strength of the ties.
 **NewSQL** – an elegant, well-defined database system that is easier to learn and better than **SQL**. It is even newer than NoSQL
 **NoSQL** – sometimes referred to as ‘Not only SQL’ as it is a database that doesn’t adhere to traditional relational database structures. It is more consistent and can achieve higher availability and horizontal scaling.
 

### **O**

**Object Databases** – they store data in the form of objects, as used by object-oriented programming. They are different from relational or graph databases and most of them offer a query language that allows object to be found with a declarative programming approach.
 **Object-based Image Analysis** – analysing digital images can be performed with data from individual pixels, whereas object-based image analysis uses data from a selection of related pixels, called objects or image objects.
 **Operational Databases** – they carry out regular operations of an organisation and are generally very important to a business. They generally use online transaction processing that allows them to enter, collect and retrieve specific information about the company.
 **Optimization analysis **– the process of optimization during the design cycle of products done by algorithms. It allows companies to virtually design many different variations of a product and to test that product against pre-set variables.
 **Ontology** – ontology represents knowledge as a set of concepts within a domain and the relationships between those concepts
 **Outlier detection** – an outlier is an object that deviates significantly from the general average within a dataset or a combination of data. It is numerically distant from the rest of the data and therefore, the outlier indicates that something is going on and generally therefore requires additional analysis.
 

### **P**

**Pattern Recognition **– identifying patterns in data via algorithms to make predictions of new data coming from the same source.
 **Petabytes **– approximately 1000 terabytes or 1 million gigabytes. The CERN Large Hydron Collider generates approximately 1 petabyte per second
 **Platform-as-a-Service** – a services providing all the necessary infrastructure for cloud computing solutions
 **Predictive analysis** – the most valuable analysis within big data as they help predict what someone is likely to buy, visit, do or how someone will behave in the (near) future. It uses a variety of different data sets such as historical, transactional, social or customer profile data to identify risks and opportunities.
 **Privacy **– to seclude certain data / information about oneself that is deemed personal
 **Public data** – public information or data sets that were created with public funding
 

### **Q**

**Quantified Self** – a movement to use application to track ones every move during the day in order to gain a better understanding about ones behaviour
 **Query **– asking for information to answer a certain question
 

### **R**

**Re-identification **– combining several data sets to find a certain person within anonymized data
 **Regression analysis** – to define the dependency between variables. It assumes a one-way causal effect from one variable to the response of another variable.
 **RFID** – Radio Frequency Identification; a type of sensor using wireless non-contact radio-frequency electromagnetic fields to transfer data
 **Real-time data** – data that is created, processed, stored, analysed and visualized within milliseconds
 **Recommendation engine** – an algorithm that suggests certain products based on previous buying behaviour or buying behaviour of others
 **Routing analysis** – finding the optimized routing using many different variables for a certain means of transport in order to decrease fuel costs and increase efficiency.
 

### **S**

**Semi-structured data **– a form a structured data that does not have a formal structure like structured data. It does however have tags or other markers to enforce hierarchy of records.
 **Sentiment Analysis** – using algorithms to find out how people feel about certain topics
 **Signal analysis** – it refers to the analysis of measurement of time varying or spatially varying physical quantities to analyse the performance of a product. Especially used with sensor data.
 **Similarity searches** – finding the closest object to a query in a database, where the data object can be of any type of data.
 **Simulation analysis** – a simulation is the imitation of the operation of a real-world process or system. A simulation analysis helps to ensure optimal product performance taking into account many different variables.
 **Smart grid** – refers to using sensors within an energy grid to monitor what is going on in real-time helping to increase efficiency
 **Software-as-a-Service** – a software tool that is used of the web via a browser
 **Spatial analysis** – refers to analysing spatial data such geographic data or topological data to identify and understand patterns and regularities within data distributed in geographic space.
 **SQL** – a programming language for retrieving data from a relational database
 **Structured data** – data that is identifiable as it is organized in structure like rows and columns. The data resides in fixed fields within a record or file or the data is tagged correctly and can be accurately identified.
 

### **T**

**Terabytes** – approximately 1000 gigabytes. A terabyte can store up to 300 hours of high-definition video
 **Time series analysis **– analysing well-defined data obtained through repeated measurements of time. The data has to be well defined and measured at successive points in time spaced at identical time intervals.
 **Topological Data Analysis** – focusing on the shape of complex data and identifying clusters and any statistical significance that is present within that data.
 **Transactional data** – dynamic data that changes over time
 **Transparency **– consumers want to know what happens with their data and organizations have to be transparent about that
 

### **U**

**Un-structured data **– unstructured data is regarded as data that is in general text heavy, but may also contain dates, numbers and facts.
 

### **V**

**Value** – all that available data will create a lot of value for organizations, societies and consumers. Big data means big business and every industry will reap the benefits from big data.
 **Variability** – it means that the meaning of the data can change (rapidly). In (almost) the same tweets for example a word can have a totally different meaning
 **Variety** – data today comes in many different formats: structured data, semi-structured data, unstructured data and even complex structured data
 **Velocity** – the speed at which the data is created, stored, analysed and visualized
 **Veracity** – organizations need to ensure that the data is correct as well as the analyses performed on the data are correct. Veracity refers to the correctness of the data
 **Visualization** – with the right visualizations, raw data can be put to use. Visualizations of course do not mean ordinary graphs or pie-charts. They mean complex graphs that can include many variables of data while still remaining understandable and readable
 **Volume** – the amount of data, ranging from megabytes to brontobytes
 

### **W**

**Weather data **– an important open public data source that can provide organisations with a lot of insights if combined with other sources
 

### **X**

**XML Databases** – XML Databases allow data to be stored in XML format. XML databases are often linked to document-oriented databases. The data stored in an XML database can be queried, exported and serialized into any format needed.
 

### **Y**

**Yottabytes** – approximately 1000 Zettabytes, or 250 trillion DVD’s. The entire digital universe today is 1 Yottabyte and this will double every 18 months.
 

### **Z**

**Zettabytes** – approximately 1000 Exabytes or 1 billion terabytes. Expected is that in 2016 over 1 zettabyte will cross our networks globally on a daily basis.

image: [*glossary/shutterstock*](http://image.shutterstock.com/display_pic_with_logo/923519/99692534/stock-photo-on-line-reading-99692534.jpg)