Have you heard about the API economy? I’ll bet your answer is yes. It is an even buzzier buzzword than #BigData. (OK, not really, but it’s almost there!)

Almost every company is now building its public API, trying to open up a little bit to allow its users to access existing services in a programmatic way, not just by working through GUI. Hundreds of mashups (combinations of services) pop up every day. Thousands of new applications (mainly mobile) exist only due to this phenomenon. But is this really something so new?

Not at all. We’ve had all these things for so many years. B2B has been about programmatic integration from the very beginning. The core principles (technology, algorithms) are still the same, but the market size/opportunity is what differs now. Just think about mobile phones – this great and simultaneously sometimes kinda lame (for all those unhappy users of non-Apple phones) piece of equipment is so widespread that, together with widely accessible Internet, it defines a completely new market.

Now it finally makes sense to create public application services because there is such a big chance someone will use it (and maybe even pay for it – that’s why we call it “an economy”). And with a growing number of consumers, the group of producers is also growing. In the end, it really leads to better user experience and satisfaction in many cases. As with all other buzzwords, there is something to it.

What’s up with the API Economy?

I (and everybody else) like the idea of openness and smooth integration. And with a growing number of application services published by highly regulated and conservative companies like banks, I am becoming more and more disappointed with the world of traditional data management tools. As you know, our product Manta Flow is a great solution when you need to understand what is inside your SQL-like code. We are highly specialized in custom code in BI environments, and as such we mostly find ourselves in situations where our customer is interested in extracting valuable metadata from their BI programs and applications and loading it into their existing metadata manager. Why?

Simply because most tools ignore those complex parts of the BI environment like stored procedures, scripts and other kinds of code. This means that the data lineage is not complete, it’s not easy to do impact analyses, and you waste a lot of money on a tool which gives you so little. But to be able to integrate, we need to find a way to load metadata extracted by Manta Flow into an existing tool and combine it with the metadata already stored inside. And now, ladies and gentlemen, the real fun begins.

It’s Not Easy to Be API-Ready

Usually every vendor has some kind of API to support integration. A nice group of well designed services exists with amazing documentation. So you try it. And you fail. So you try it a different way. And you break something. And because you have to come up with a solution to satisfy a customer, you continue – and it hurts. Believe me, we go through this painful process every damn day. How is it possible? Simply because those big monolithic tools were not designed to support integration with other software applications. I understand! To design and develop a big and really pluggable software system is not an easy task, even for experienced engineers. But so many things are broken! Some tools try to analyze SQL code but with bad results. Surprisingly, there is no easy, standardized way to replace wrong metadata without breaking links to other parts of the repository.

Some tools support internally many different kinds of metadata structures but offer only a very limited part of their capabilities in the form of API. That means you are forced to work directly with the existing structures in the repository! Some tools have serious performance issues when loading external metadata through their API. There is also one data management product infamously known for its strict legal protection against any integration. Trying to integrate with that tool is like trying to open the doors of a dark dungeon. Such a project can be so painful that the customer decides to stop integration entirely. Does that sound like the 21st century to you? Not to me.

New Hope

But this cloud has a silver lining as several traditional vendors have announced massive redesigns. Also, young and more focused players, like business glossaries, are better designed for integration from the very beginning. (Some great examples are Collibra and Diaku Axon, but there are many more of them.) We implemented several integration scenarios requested by our clients with some of these young stars, and you know what – everything went surprisingly well. It seems that finally, after so many years, the API approach is also entering into our beloved data management world.

And that is great not just for other-than-huge vendors like us, but especially for customers. Metadata is a very complex beast to tame and no product can become the ultimate solution for it. Great API support allows you to combine several great tools to get a more complete solution. And that’s what matters the most.

Long live the API economy!

Any questions or comments? Just let Tomas know at manta@mantatools.com or via the contact form on the right. 

Related Resources