Let a Thousand Data Silos Bloom

It’s a story as old as time. It’s a story of evolution, a story of freedom. And it’s a story of stuffing a genie back into its bottle.

Yes, I speak of the ancient enterprise battle royale: Productivity vs. Control.

These battles are constantly happening throughout the enterprise, usually with very good intentions. But, like so many good intentions, they often pave the road to, um… employee circumvention. Hey, gotta get stuff done, right?!?

Winning Hearts, Minds and Data

With data-related issues, the classic battle was fought between, on one hand, centralized data warehousing systems and, on the other, user-driven tools such as Microsoft Excel and Access. One lumbering version of the truth vs. many agile versions of truthiness.

According to an oft-quoted study from Gartner, 70 to 80 percent of business intelligence projects fail. In the meantime, Excel enabled (and enables) people to get stuff done.

The next battle is being fought against the Web itself.

Do companies embrace quick, innovative SaaS solutions along with their data silos – or do they need to combine their line of business requirements into a monolithic, controlled, integrated package?

Forbes recently posted an article that advocated “Tearing Down The Silos That Cloud Apps Create”. It begins by praising the great value that SaaS systems can offer:

The Cloud is a wonderful thing. For companies, departments, teams and employees it means freedom and, to a degree, self-determination. They can quickly find and start using applications that empower them to be more creative, efficient, collaborative, and competitive. Free to tap into virtually any SaaS application without IT involvement, the era of the Cloud has unleashed waves of productivity and innovation.

But, after discussing the virtues of an all-in-one, integrated platform, it concludes:

…frankly, cloud silos need to be torn down for companies to scale efficiently.

And there’s the dichotomy. As much value and productivity as cloud applications may deliver, they stray from centralized control and providing “a single source of truth”. So we have no choice but to tear them down, right?

Do you Build, Tear or Connect?

Certainly, information silos have long been an issue within the enterprise – and they get exponentially worse as you add more divisions, departments and competing priorities. And, certainly, to their credit, the enterprise has gotten better at providing centralized data to their employees, from enabling extracts from ERP systems to providing real-time logistics to building out data warehouses for common needs.

But. But! There’s a reason why employees continue using Excel. And it’s the same reason they’re drawn to all these SaaS applications and their meddling data silos.

It meets an immediate need. It provides self-service unlike anything else at their disposal. It gets the job done. Today.

So, the question is what to do…

Do you embrace the silo or do you tear it down? Do you give your employees a boost in productivity now or do you make them wait a year while The. Integrated. Solution. gets vetted by 10-20 people across various disciplines and then implemented at a hefty cost?

Here, I believe, it is most apropos to recall the sage words of Princess Leia:

The more you tighten your grip, Tarkin, the more star systems will slip through your fingers.

Centralized control can only go so far. To me, the clear answer is to embrace the silo for the sake of productivity and anticipate that the market will find ways to connect, rather than tear down, these silos. This new breed of cloud system surely isn’t going away – the value is far too great. If you can’t beat ‘em, join ‘em.

The Cavalry is Coming

The very good news is that the market is recognizing the need to tie together our disparate data ecosystem in an agile way. A recent article entitled “How the cloud will change in 2015“ notes:

With businesses relying on an increasing number of disparate cloud services, there’s an emerging need for value-added services that enable them to leverage all of that data.

And another recent article touting disruptive trends for 2015 says:

We believe that self-service data preparation and harmonization will complement and coexist with IT’s traditional data management tools, which will continue to address critical issues around data security, compliance and governance.

Every SaaS service seems to have its own API these days, which is an acknowledgement that “no app is an island”. And within this data ecosystem, you have the content creators (e.g., Salesforce, Mailchimp, Google Analytics, etc.) and you have the folks that connect and add value to these content silos. Some quick examples:

• Segment – Push your web data to these guys and they’ll map it to any number of SaaS services for you.
• DataHero – Pull your data from lots of different SaaS services for data visualization.
• Revert – Connect to any of your SaaS services and make a backup of your data.

The more cloud apps generate value, the more silos will be created, the more important it becomes to embrace them and tie them together.

In our neck of the woods, this is one of the things we’re trying to accomplish. It’s fine to have a tool that makes it easy to prepare and transform data for analysis, but the true magic happens when it becomes trivial to connect to these disparate data silos, do your data prep and spit it out… and then rinse, repeat tomorrow (btw, shameless plug – sign up for our Flex.io beta if you haven’t already!).

So in this new battle with the Web, I say let a thousand data silos bloom! More productivity will make for better companies. And more SaaS silos will make for new tools that tie together this wonderful data ecosystem.

Images by: Anthony Arrigo and alessandro canella