Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Articles Your Database: The Threat That Lies Within

Your Database: The Threat That Lies Within

Much attention is given to databases, so they will not become THE risk factor of IT. Budgeting DRPs, backups, top of the line DBAs - and yet, they still pose a major threat. Why?

Being Agile

In a fast-moving competitive market, if your competitor delivers relevant products, faster and with better quality, you’re eventually going to lose market share. This is exactly why companies need to be agile.  They need to have better control over information, make quicker decisions, accelerate software delivery, and optimize quality control. Agile development was born to deal with these challenges, empowering organizations to move faster and deal with ever-changing requirements, while ensuring optimum quality. The need to take Agile concepts to production (production environment, customer sites, etc.) and to link development with operations, has given rise to DevOps.

Software Development Life Cycle (SDLC) best practices did not drastically change when Agile and DevOps emerged, as these are steps of a software development evolution, rather than a revolution. However, these best practices are now much more important to implement and follow if you are traveling in the fast lanes of the agile world. Furthermore, automation is crucial to increasing overall efficiency and reducing the risks inherent in frequent changes and releases.

Automating processes and implementing continuous integration, continuous delivery, and continuous deployment have become proven practices to ensure the overall process is efficient and reliable.

The Database Requires Special Attention

Automation-based repeatability and continuous processes will drive safer and less error-prone deployments. The success of frequent changes cannot rely on a person's ability to remember everything that's already been done, or their ability to identify the full scope of what the current change might influence. The key is eliminating as much manual work as possible.

But, unlike other software components and code or compiled code, a database is not a collection of files. It cannot just be copied and pasted from development to testing and to production, because it is a container of your most valued asset – your business data, which must be preserved. In most cases, database development is also performed in a very different way than application code (.Net or Java development), as developers and DBAs are accessing and changing a shared resource, a central database, rather than a local copy on their workstation.

The difference between database development and application code development is so great, that in time, silos were created.

While application developers are moving forward, adopting great new tools and practicing Agile, automation, and continuous delivery, database development is done using a less controlled process. More art than engineering, database development is often disconnected from the entire SDLC process. Tools, processes, and best practices are not shared.

The database, being such a valued asset, can very easily become that wobbly wheel that destabilizes the whole car.

How Should Database Change Be Addressed?

We need to stop looking at the database as something different, just because it is constructed differently. Make sure all teams, code developers, DBAs, and database developers are following the same processes. Proven software best practices should be followed and applied to the database, as they are for code development.

In order to minimize the risks databases pose during the development cycle, the following should be addressed:

  • Agile development - Run short development cycles, and implement database changes as part of task-based development. This will ensure that code changes and database changes are done hand-in-hand, and can later be deployed together based on change requests.
    - Task management solutions such as Jira can help correlate tasks, issues, change requirements, or trouble tickets to actual work being done. It can help you tune your efforts towards well-defined deliverable outcomes, and later deploy these changes based on task-centric views.
  • The provisioning of development environments has been made more efficient by Chef, Puppet, and Jenkins. But what about the database? How can you provision environments with large databases? Or even small ones? The database can still stand as a barrier to agility.
    - Using Delphix to easily create virtual database branches can help spawn development environments with ease - and make parallel development more accessible. A virtualized copy of a production environment will allow you to test production deployments, true to production's latest data, without having to deal with storage penalties, or actual production risks.
  • Content masking can become a real challenge, as you try to build a good development, testing, UAT, or pre-production environment. The more true-to-life your test data is, the more confident you'll be with quality assurance. But what happens if you give your developers unfettered access to a copy of production? Are your CC numbers sitting on some random laptop? Will this leak?  On the flip side, having made up data or no data at all will fail to represent a good testing environment, and will be a poor measurement for performance.
    - A solution such as DMsuite can help replace confidential data with realistic but fictional data, without any coding. If you need to do so for compliance purposes (healthcare, or finance content), or if some of the information is just sensitive, this will reduce your "hackable footprint by replacing the confidential data in your non-production environments.
  • Database change management – Implement database tools to make sure database changes are properly managed as part of SDLC, and are not working outside of main development processes. Also, make sure each change is correlated to a change request, which will help manage the database as part of general development or change efforts.
    - By using DBmaestro to enforce version control and track all changes, you'll always have full control over who is doing what, where, and why. This will eliminate out-of-process changes. Once a development cycle has concluded, relevant changes can be automatically deployed into integration environments, identifying change conflicts and merging them, or pushing them to production.
  • Testing automation - Building proper automated tests requires big investments. Consider this a long term goal, not a one-time project, and start accumulating automation tests for each new development or major change.
    Unit testing: Following change-focused testing and making sure nothing breaks as a result of changes will provide a solid safety net
    - Running unit tests with Oracle SQL Developer or Microsoft-focused tSQLt is a cost-free way to introduce unit testing for the database.
    Regression testing: Yet another way to test your database along with the application code, as one unit.
    - Using TestComplete to validate your common -  and later less-common - scenarios will provide great confidence in your ability to deliver releases .
  • Release Automation - A major key for risk mitigation. The more you automate, the more accurately you can reproduce the changes introduced in development, test in testing environments, and repeat safely when going to production. A healthy SDLC with minimum risk and maximum efficiency can be achieved once you automate its processes and address its challenges on daily basis.
    - Using IBM Urbancode Deploy can help manage your release activities andyour configurations, and orchestrate the overall process.

Best of all,  these types of solutions play very nicely with each other. So creating a virtual development branch with Delphix, masking its confidential content with DMsuite, managing development tasks with Jira, version controlling changes and packaging a release with DBmaestro, automatically testing this release with tSQLt or SQL Developer, and orchestrating the release process with uDeploy can all be combined into one slick, automated, and safe process.


The database presents a real challenge when it comes to following best practices. This is exactly how silos are created. A development department might follow different processes when dealing with code development vs. database development, or follow best practices for only some of its change efforts.

The best way to eliminate major risks is to stop looking at these differences from a technology point of view, and find the right way – be it a process or a tool - to follow and enforce best practices and automation.

The database, being such a valued asset, must not become that wobbly wheel that will destabilize the car.

About the Author

Yaniv Yehuda is the Co-Founder and CTO of DBmaestro, an Enterprise Software Development Company focusing on database development and deployment technologies. Yaniv is also the Co-Founder and the head of development for Extreme Technology, an IT service provider for the Israeli market. Yaniv was a captain in Mamram, the Israel Defense Forces computer centers where he served as a software engineering manager.

Rate this Article