I am currently reading the book
Site Reliability Engineer that is
written by a group of Google Engineers. The book is a great resource
for knowledge on reliability engineer's work and how it differentiates
from a the job of a common system administrator or software
engineer. It describes how google is designing and integreating
automation or monitoring systems to deal with the large scale projects
that are Google's everyday business. While not first an foremost
beeing a book about software design one comment got my attention
particularly (page 271):
Don’t wait for the perfect design; rather, keep the overall vision in mind while moving ahead with design and development. When youn encounter areas of uncertainty, design the software to be flexible enough so that if process or strategy changes at a higher level, you don’t incur a huge rework cost. But at the same time, stay grounded by making sure that general solutions have a real-world–specific implementation that demonstrates the utility of the design.
The quote reminded me a lot of my own work I am currently doing at the university. We have to design a lot of modules that need to work together as performant and robust as possible. As the requirements in this particular constellation haven't been very broadly implemented or designed we had to do a some assumptions in the beginning of the design process. For instance one if the requirments was to iterate over relational data that could be best described as a graph. Therefore (as we are implementing this project in Go) we thought we are best advised to index the data using Cayley, which is a Graph Database written in pure Go with multiple backend implementations. However later on we discovered that we wouldn't use most of the features Cayley gave us. Hence the overhead caused by Cayley only lowered performance of the software. One of our best decisions was to hide such implementations behind interface definitions such that we would be able to switch out implementations easily in case we would find a better solution. Therefore we decided to implement the interface using BoltDB only which gave us a huge performance improvement. As the interface was well designed the rewrite of the module costed one engineer only half a day. As all unit tests for this module have been written to use the interface only we could be assured that the rewrite is stable enough to be merged into upstream.