Thoughts on foundational framework development

Foundational framework development is very important in any technology companies. It solves common problems shared across departments, teams or projects. It generally is lauded by management. Many great open-source frameworks, e.g. AngularJS, React, I believe, are stemmed from in-house framework development then taken from companies like Google and Facebook to public.

There are two schools of thoughts to develop foundational frameworks: bottom-up or top-down. Bottom-up approach is to build it prior to any applications by foreseeing and analyzing various potential needs of applications, finding the common area then developing it. Sometimes the analysis phase becomes a bit of guessing work and shot in the dark. And at organizational level if there is division between foundational team and application team, it could makes this approach even harder. However this approach fits well to develop a well-defined public API or standard like JDBC driver for an in-house database.

Top-down approach is more practical, teams, without the division of foundational team and application team, start to build the applications to meet business needs and both with well-defined architectures. During the course of development teams start to discover the relationship between difference layers and modules, continue to refine the architecture and code-base. At the end of project or even after project delivery teams set out to refactor and harvest a framework from the applications. This approach usually has grounded success since it is based on real-life applications to solve real problems. Also this approach is well aligned with refactoring and iterative agile development.

The following diagrams further illustrate my thoughts above:

The bottom-up approach tends to assume the interfaces between foundation layer and applications are well-defined or the boundaries can be easily discovered.

 

However in reality, the picture looks more like this:

 

 So bottom-up approach ends up like this: 

 

What happens above is that application logics are everywhere in the foundation layer. In the code we see lots of if/else, case/switch statements, and any changes in foundation layer to accommodate one application will impact other applications.  That foundational layer eventually becomes a monolithic application.

So what is best way to develop foundation layer? Based on my experiences, there are few OO design principles and patterns can greatly help us.

1. Foundational interfaces should be minimum. Try not solve everything, leave as much as you can to application unless you are sure the functionality is needed. Develop a toolkit not a specific problem solver.Using java's java.util.List as example, it provides a method called get(index) and doesn't provide method getFirst() or getLast() since it leaves the clients to do that. In that way you leave the client assembles the jdk methods to any requirement it may need to accomplish.

1. Close to change, open to extension.

2. Dependency injection (IoC) or Hollywood principle  - Don't call me, I call you.

3. Develop pluggable interfaces.