Evolutionary database design
Evolutionary Database Design
Evolutionary Database Design involves evolutionary improvements of your database schema so that it can be updated with respect to changes in the customer requirements. Today as people across the globe work on the same piece of software at the same time, there is a need for techniques that allow evolution of database as the design develops. Such techniques utilize automated refactoring and continuous integration in order to support agile methodologies for software development.
Over the last few years, the software industry has witnessed a rise in adoption of agile methodologies for software development. Agile methodologies don’t assume requirements to be permanent at any stage of the software life cycle. These methodologies are designed to support sporadic changes in requirements and dynamic business environments unlike planned and evolutionary design techniques. Before the adoption of these methodologies project design was a stage that used to be completed before the code actually went into construction. The same principle applied to the database schema as well and it was considered to be a reflection of the software requirements which were developed out of collaboration between the customer, end-users, business analysts etc and these requirements were not expected to change as we progressed in the software life cycle. But this approach proved to be cumbersome because as time progressed the redundancies in the existing database schema in the form of unused rows or columns was evident. This redundancy along with data quality problems went on to become a costly affair. These defects concluded that it is a bad practice to have not have design interleaved with construction and testing. This is true for database schemas as well and such evolutionary development is only possible with collaboration between data professionals in the team and application developers. This development techniques are applied on systems that are in pre-production stage as well on systems that already been released.
Evolutionary Database Design Techniques
As mentioned in the previous section evolutionary methods are iterative in nature and these methods have become immensely popular over last 2 decades. Evolutionary database design aims to construct the database schema over the course of the project instead of building the entire database schema at the beginning of the project. This method of database design can capture and deal effectively with the changing requirements of projects.
There 5 evolutionary database design techniques that can aid developers to build their database in an iterative fashion. A brief overview about the five techniques are provided below,
Database Refactoring
Refactoring is the process of making changes to the program without affecting the functionality of the program. Database refactoring is the technique of implementing small changes to the database schema without affecting the functionality and information stored in the database. The main purpose of database refactoring is to improve the database design so that the database is more in sync with the changing requirements. The user can modify tables, views, stored procedures and triggers. Dependency between the database and external applications make database refactoring a challenge.
Evolutionary Data Modeling
Data modeling is the technique of identifying entities, associating attributes to the entities and deciding the data structure to represent the attributes. In evolutionary data modeling the technique of data modeling is performed in an iterative manner. This kind of data modeling is practiced in an agile environment and it is performed in a collaborative manner.
Database Regression Testing
Whenever a new functionality is added to a system it is essential to verify that the update does not corrupt or render the system unusable. In a database the business logic is implemented in stored procedures, data validation rules and referential integrity and they have to be tested thoroughly when any change is implemented in the system. Regression testing is the process of executing all the test cases whenever a new feature is added to the system. Test First Development (TFD) is a form of regression testing followed in evolutionary database design. The steps involved in TFD approach are,
- Before adding a new function to the system, add a test to the test case suite such that the system fails the test
- Run the tests, either the entire set of test cases or just a subset and ensure that the newly added test does indeed fail
- Update the function such that the test passes
- Run the tests again to ensure that all the tests pass that is system is not broken
Configuration Management of Database Artifacts
Configuration management is a detailed recording of versions and updates that have been applied to any system. Configuration management is useful in rolling back updates and changes which have impacted the system in a negative manner. To ensure that any updates made in database refactoring can be rolled back it is important to maintain database artifacts like Data Definition Language scripts (DDL scripts), Data Model files, Reference Data, Stored Procedures, etc. in a configuration management system.
Developer Sandboxes
Sandbox is a fully functional environment in which the system can be built, tested and executed. In order to make changes to the database schema in an evolutionary manner it is ideal for every developer to have his/her own physical sandbox, copy of source code and a copy of database. In a sandbox environment the developer can make changes to the database schema and run tests without affecting the work of other developers and other environments. Once the change has been implemented successfully it is promoted to pre-production environment where in acceptance testing
Best practices in Evolutionary Database Design
Stuff By Sneha
Advantages and Disadvantages
Advantages
- High quality of database design: In evolutionary database design the developer makes small changes to the database schema in an incremental manner and this achieves a highly optimized database schema.
- Handling Change: In a traditional database approach a lot of time is spent in remodeling and restructuring the database when the requirements change. In evolutionary database technique the schema of the database is adjusted periodically to keep up with the changing requirements. Hence Evolutionary Database Design technique is better suited in handling the changing requirements.
- Guaranteed working of system at all times: The Evolutionary Database Design approach follows Test First Development model, in which the complete working of a system is tested before and after implementing an update. Hence it is guaranteed that the system always works.
- Compatible with software development: The IT industry is progressing towards agile method of software development and Evolutionary Database Design ensures that data development is in sync with software development.
- Reduce Overall Effort: In an evolutionary environment only the functionality that is required at that moment is implemented and no more.
Disadvantages
- Cultural Impediments: Evolutionary Database Design approach is relatively a newer concept and many well qualified data professionals still advocate the traditional approach. Therefore, most of the databases are still being designed in a serial fashion and Evolutionary Database Design is yet to gain support and traction among experienced data professionals.
- Requires a learning curve: Most of the developers are more familiar with the traditional approach and it takes time to learn evolutionary design as it is not intuitive.
- Complex: When the database has a lot of external dependencies, making changes to the schema becomes all the more complicated as the external dependencies should be updated to cope up with the changes made in the database schema. With the increase in number of dependencies Evolutionary Database Design approach becomes extremely complex.
Comparison between Traditional and Evolutionary Database Design Techniques
Sneha