A recent blog “Why use an Industry Data Model?” discussed how industry data models bring exceptional business value to customers by providing a logical “blueprint” for data integration and enterprise analytics within well-defined business spaces.
A recent blog “Why use an Industry Data Model?” discussed how industry data models bring exceptional business value to customers by providing a logical “blueprint” for data integration and enterprise analytics within well-defined business spaces. If this data model is expanded to include a template industry physical data model (iPDM), can the model realistically be expected to support a database physical implementation?
If we follow the premise that these added PDM components can provide a starting point for building an integrated solution using field-proven physical design patterns and industry best practices, then we see that this “template iPDM” can greatly improve physical implementation productivity by leveraging from field-proven designs and practices.
In terms of a solution framework as it applies to an integrated data warehouse environment, both Integrated Data and Semantic Layers are essential to a well-architected data environment. The Integrated Data Layer contains data and metadata that is “neutral” from the data source perspective, and from the perspective of data and metadata usage. It is in the Semantic Layer that structures should be implemented for specific business uses.
The most foundational aspect of the integrated data warehouse design is the availability of a well-architected data model. As has long been the case, a logical data model (LDM) contains data elements organized to support a specific business or industry. The physical data model (PDM) components are the framework for the implementation of these structures, providing the details necessary to generate the DDL for the warehouse. The physical model resides alongside the logical model, expanded to include the components necessary to generate physical database structures like tables, views and indexes, designed to ensure optimum performance. . Simplicity as well as efficiency is realized with a combined logical/physical model, so whenever possible, PDM elements should retain the dynamic nature of their LDM counterparts to ensure ongoing flexibility.
Industry data models should be considered as templates that require further refinement as part of a customer implementation. This includes the validation of modeling requirements with a customer by mapping to business scenarios and data sources, identifying any gaps in functionality, followed by extending and trimming the model to fill the gaps. For all logical modeling changes there will be parallel physical modeling changes, and vice versa, so a flexible modeling approach affords the best solution, with a keen eye kept to insure that changes to each side maintain support for all business requirements.
Properly customized, the physical model can be quickly and correctly instantiated using the tools provided within the data modeling tool. Too much physical modeling time has been spent in the past on tasks such as manual naming of indexes and tables. Templates for generating DDL for both tables and views are provided, utilizing standard abbreviation files to ensure that logical names are consistently abbreviated within table, column and index names. The ability to derive these structures directly from the model file preserves data model relevancy and integrity throughout the implementation, and allows developers to devote time to things like evaluating source data and usage analytics, allowing them to more effectively implement a complete solution that includes value compression and appropriate partitioned primary index choices.
It is important to produce data model structures that allow you to implement the physical solution expressed in your logical model without the need to de-normalize. If you are building an integrated data warehouse, you need to use a database platform that supports high performance and a data modeling framework that facilitates the building of normalized databases for analytics right from the start. You need to start with an architecture that integrates seamlessly with “big data”, BI/OLAP relational analytics and advanced analytics engines. The Teradata database engine in concert with their Industry Data Models provides you with the perfect place to start.