Metadata versus Taxonomy

I’ve advocated for many years that Information Management should be a superset of related disciplines including data warehousing, document management, library science, enterprise search et cetera. While this is an easy statement to make, it is really hard to execute.

The problem is that practitioners from the different technical backgrounds have radically different approaches to handling information in all of its forms. While the technologies are different (using solutions as diverse as relational databases, file systems and even physical shelving) this is not the real reason why the disciplines are so hard to bring together.

Practitioners coming from unstructured and structured data backgrounds use subtly different definitions of metadata and I argue that it is these differences that cause most of the angst that comes through in disparate repositories, governance and a lack of integrated business solutions.

Unstructured data came first, and its filing is primarily treated as a problem of taxonomy. The most famous approach is, of course, the Dewey Decimal System. When unstructured data practitioners talk of metadata they include the taxonomy and attributes of the data itself such as the author, publication date, copyright and other core attributes (best defined by Dublin Core).

Structured data practitioners have, for the past forty years, relied on relational database theory as the foundation of their information management practices. Relational data generally includes as data, rather than definition, the key elements of people, place and time. Such an approach is very neat, with metadata being literally data about data and being restricted to data structures and the definition of the data elements themselves. As a result, the metadata for structured data is much more succinct.

While succinct is a good thing for computer programmers, it seldom translates well for the rest of society. As a result, structured database metadata has seldom found its way out of technical departments within large organisations. At the same time, the need to understand who authored a record, who it was about and how it relates to other events in a timeline remain as important as ever. As a result, we now have “master data”.

Perhaps the solution is for all Information Management practitioners to concede that Metadata should encompass both the metadata that structured data practitioners advocate and the master data that the unstructured data practitioners have long advocated as being essential. We just have to get over our fixation on the titles. I’ve tried to define an approach that does this in my new book, Information-Driven Business.