The Evolution of Metadata Management Solutions
It’s important to know where are you coming from to know where are you heading.
That’s why I asked Petr Stipek, our VP of Business Development, to sum up the evolution of metadata management solutions in order to shed some light on the messy, overbuzzed area.
In the past, metadata was a 2nd-class citizen, the “not-really data”. Nevertheless, it was needed for the system to operate. Metadata management was not really recognized as a discipline.
Example solutions: Most relational database management tools.
And then, luckily, metadata started to be handled as a first class entity for the very first time. Minimal integration and automation was deployed, though. It was stored in separate silos for different kinds of metadata. Only a relatively small part was handled meaningfully: structured metadata. The rest was still basically just comments and many various metadata sources were still ignored.
Example solutions: The first modeling tools and metadata management software.
Metadata of various provenances (models, ETL definitions, SQL code, business definitions) was finally integrated to some extent, and the metadata life cycle was at least partially addressed. For the first time, custom code was analyzed, like, big time. Some clever data scientists started using Big Data technologies to mine, or at least preserve, additional metadata. Business definitions are, sadly, still just comments; some metadata sources are, even more so, still ignored.
Example solutions: Current leading metadata management platforms integrated with a code analysis solution (like, well, Manta Flow).
Imagine a world where all available metadata is managed in an integrated environment. Most significant metadata sources are mined, everything is automated, and the metadata life cycle is properly managed. Business definitions are integrated on the semantic level (which would require human-level artificial intelligence).
Example: None. This is the future, folks.