Mashing up multiple data sources to generate a single source of truth is an integral part of data analysis. It allows you to compare and cross-reference records stored in different formats and locations, and to perform queries and calculations. This article will run you through some basic concepts in data analysis that you should become familiar with when joining data stored in multiple tables.
Mashing up multiple data sources to generate a single source of truth is an integral part of data analysis. It allows you to compare and cross-reference records stored in different formats and locations, and to perform queries and calculations. This article will run you through some basic concepts in data analysis that you should become familiar with when joining data stored in multiple tables.
Why do I need to know this?
Short answer: In most cases, you don’t. If you’re using a strong business intelligence tool and working with simple datasets, the bulk of the data preparation process should be handled by the software itself. However, it’s useful to understand how your data is transformed in the process and how the new, combined data will look in tabular form. Additionally, you might need this info when working with more complex data.
So let’s get started!
What’s a Join? And isn’t that a verb?
In the context of SQL and database management, a join is a way of combining records from multiple tables. A join requires common fields between the two tables in order to form a logical connection, and is the basis for combining different data sources and an integral part of data analysis.
Inner Joins
Used for connecting identical fields
An inner join is used to connect two or more tables that contain fields with identical records. For example, if we look at an example of two tables:
These two tables would be combined via an inner join based on the common field – Product. The result would be one table which contains data from the previous two:
If one of the tables contained a record that does not appear in the other (e.g. if the Product-Stock table would contain a fourth row with details of banana stock), that data would be disregarded.
Outer Joins
Used for connecting common, but not identical fields
Outer joins are divided into left, right and full joins. To understand the difference between the two, let’s once again look at two tables:
As you can see, there is no column with completely matching records between these two tables. There are three main ways we can go about combining this data:
- Left join: Generates a table that contains all the records from the lefthand table, along with any matching records found on the righthand one. In our example:
- Right join: Same as left join, but will contain all records from the righthand table.
- Full outer join: Will essentially perform both a left and right join, combining the two tables despite any records not matching, e.g.:
These are the main types of joins you’re likely to encounter. Each might be used in different scenarios, depending on the analyses you will want to perform on the combined data.
Row Level Joins vs. Summarized Views
Business Intelligence software tools have two different ways of joining multiple tables:
A row level join means that after two data sets are combined, the original records are still kept within the data model and accessible. This, once again, is a question of the strength and sophistication of the engine that powers your analytics tool: A robust one should be “smart” enough to understand the data and model it in such a way that will allow you to drill down into the full granularity of your data, as it was before the tables were joined.
In contrast, BI software that relies on a less powerful back-end will instead create a summarized view of the data by aggregating it, thus compromising its granularity. The guiding line should be this: can you easily access the original, row-level details of your data after the join is performed? If the answer is no, your BI software might be lacking in terms of its mash-up capabilities.
Illustration:
Learn more about mashing up data sources
Read a quick guide covering:
- Joins vs. aggregations
- Avoiding many to many relationships
- Joining data sources with BI software
