Hadoop

Technical debt : Understand and manage it…

As we go on writing a piece of code or building a solution we always create some sort of technical debt. It is not always a bad thing but it is something we should learn over time how to manage, control and track. Initially in my career, I always thought and looked for creating a perfect solution and sometimes that mode of thinking put me into a situation where my manager thinks that I am putting a lot of effort into the analysis of the solution rather than delivering them fast. You would not believe that since the last six years after entering into Big Data, everyone was impressed by me not because I have learned how to create a perfect solution but because I have learned how to deliver fast with some sort of technical debt into my (deliverable to pay later). I agree that I’ve been accumulating a lot of technical debt during those years but with that debt some companies(product and manufacturing) have got investors on project contacts after POC’s, they have made a working product after applying experts on the solutions. And obviously someone out there paying for code in complexity after I left.

When working on building applications, developers sometimes implement a tactical solution to make a functionality work. Most of the time the developers wish to replace the tactical code with a more well-written code. These solutions are not intended to make any changes to the delivered product but somehow remain as a  part of the codebase for a long time. As the application evolves and teams get changed, more and more of this tech debt keeps on piling up. This tech debt makes the applications difficult to understand and maintain. The business also does not want to prioritize removing this as they do not see how this adds to the final value. Tech debt is also not a rare thing. It is difficult to find a developer who has spent a significant amount of time coding and has not encountered Tech debt. 

In the basic scenarios, when doing a POC, we may tolerate a certain amount of tech debt as the focus is more on delivering a working solution fast. It is to demonstrate to the client that we have the ability to understand their problem and offer a viable solution. Same thing as we do with the prototype in the real world.

But once we are asked to build a full-fledged product, the focus is more on the quality of the delivered products. We need to focus on quality and a well-engineered product.

Both scenarios have different intentions and many times we forget to educate the teams and do not take these points while doing project planning.

So here are two scenarios, 

One, no technical debt after I have created a perfect solution by looking into all future scenarios and that solution/code, not getting touched again after delivery. 

Or, on the other hand, technical debt consists in performing a development or fix in a quick and dirty way because it takes less time, even though it will make the code harder to work with, in the long term.

To quote from a personal simple example, once I was assigned to work on a windows application with a SQL back-end. The application was making use of dates that were being stored as a string. Not only that there was a lot of logic split across various files that were being used to parse the dates to convert it to a valid date. Many times I came across differences in the logic of various classes. After a discussion with my team, we decided to update the database to have date columns instead of string columns and did one-time parsing of the older data. Once that was done, we compared the output data and some of the reports that the application generated. Finally, we were able to get the correct data being displayed and overall performance and maintainability of the system also improved.

Strategies to handle technical debt:

  • Leave everything that you touch better than you found it.
  • When working on implementing a new functionality that is a portion of an old code base, try to do incremental refactoring of the code that you are touching.
  • You can also write unit tests and automation tests. In most cases, when working with old codebase, it is not very likely for the code to be written with test-driven development in mind. To solve the problem developers can write integration tests from the beginning and later move on towards writing unit tests after refactoring.

Unlike financial debt, which should be controlled by various organization-internal mechanisms, tech debt is often not something management is conscious about (at least when it is “created”). I’ve seen far too many examples of horrible code that can simply be explained with laziness or incompetence, but I’m pretty sure management never called for a hard to use, unnecessarily complicated API or DB procedure. One of the example codes I am referring to is a migration code where a few processes are migrated by some developers, as-it-is. There was a huge scope of improvement in the code and therefore I’ve taken a step forward by adding comments of the source system, purpose of code and refactoring the code by removing unnecessary joins. I wrote, “Man you are paid by the client to write a good code not just migrate the process as it is otherwise that was better in legacy system :)”.

I am an Architect and I am proud to build things for most beautiful consciousness that evolved, would say technology built by the man.

Leave a Reply

Your email address will not be published. Required fields are marked *