The proliferation of specialized databases is a double-edged sword. On one hand, it’s great to have the flexibility of new database types to better solve specific business problems. On the other hand, introducing a new database is only a good idea if you have resources to manage it.
Recently, I sat down with a group of IT stakeholders at a leading global insurance provider to talk about optimizing their database environment. The company supports more than 10 different databases, including traditional ones like Oracle, SQL Server and DB2, in addition to new types, such as MongoDB, Hadoop and Cassandra. Interestingly enough, many of the people in the room didn’t know what their colleagues were doing, so it took a meeting to gain the much-needed visibility into their environment.
This scenario is increasingly common yet underscores a growing dilemma: how do you balance the idea of being flexible and open to new platforms to solve problems without overburdening the database management team?
Today, there are nearly as many database types as there are use cases. This means you can get very prescriptive about which type best fits your business need: Will a database built to serve as a document repository help you better tame big data? Or is a transactional system that scales rapidly more suited for your high-volume data requirements? It’s also important to note that some databases are better designed to handle more sophisticated analytics, which is a big trend as companies continue to move away from centralized data warehouse architectures. Most of these new types represent non-relational, graphic, document or columnar databases, each requiring a different and non-transferrable skillset.
So, what happens when you end up with too many databases and too little time to manage them? This question is often overlooked as companies are more focused on finding new and better ways to integrate and analyze data. But, orchestration and harmonization of the overall environment is a critical factor. If you don’t have the skills or resources to perform those functions as well as manage the system that houses all that information, all the money spent on analytics will break the upstream process.
For that reason, I always caution customers to step back and first line up their business problems to the platform they think will best serve them. Then I ask about the impact on existing resources and plans to secure, develop, manage and optimize the new platform on an ongoing basis. After all, it’s all about putting the right data on the right platform for the right workload. Undeniably, figuring all that out is the hard part.
That’s why I always probe to find out more. Is this a single project or one with legs that will impact other parts of the company? Could the problem be adequately solved with an existing platform? Does the business case warrant the 10X improvement promised by a new system? Or can you get by with an existing platform that won’t overburden your database administrators? Not everyone needs a Ferrari, especially if it ends up in the shop more than it’s on the road.
To get the most out of your analytics and big data projects, make database management a primary concern. Remember, there’s a skill shortage on this side of the house too, so piling on the databases will backfire unless you invest in ways to help the database team work smarter and become more efficient. One of my colleagues, Shawn Rogers, refers to this as the dilemma posed by emerging requirements of hybrid data ecosystems. In determining if a new solution is warranted, you first need to look at data load, data structure, response time, complexity of workload and the need for economic advantage.
At Dell, we’ve spent years looking to automate database development and management. Our database-agnostic Toad solutions enable customers to apply consistent processes and universal tools across all their database environments. This helps them transition from one platform to another more easily because we look at the problem of managing and integrating databases through a data lens—not through the lens of a particular database company.
This holistic approach also can lower cross-platform management challenges by accelerating onboarding of resources as you can apply a similar set of tools and standardized processes to each new database type. So, while there’s no need to reinvent the wheel each time you bring on a new platform, you still need to be sensitive to the overall management impact and need to orchestrate the landscape.
The bottom line: Don’t forget your DBAs as the number of databases climbs. If you can use one development or administrative platform across five or six different database environments, it will ease the management burden when there’s too many databases and too little time.
What are you doing to make life easier for DBAs? Connect with me on Twitter at @simpleisbetter to share your ideas.