The situation is usually as follows: An idea for a software system arises in a company, usually a web application or an app. At some point, the decision is made to move ahead with the implementation. Here, the first questions arise:

  • Should we do internal or external development?
  • Which technologies should we use?
  • How do we identify (very) good developers?
  • Do we move to the cloud or do we host ourselves?
  • …

With an initial team, you look for answers to these questions and reach the necessary decisions. Then the implementation starts with great enthusiasm. After a while, the typical problems of software development occur:

  • Everything takes longer than expected.
  • There are many small problems that require the attention of the business owner and management.
  • A complex technology update is suddenly required that sets development back by several weeks or months.
  • Project communication and planning are more time-consuming than anticipated.
  • Team members change, new views on the technology and approach create additional effort.
  • After three years of development and two attempts to build the system, the application is still not live.

Although it may sound unrealistic or exaggerated at first glance, this is actually the story we hear - in one form or another - on a regular basis.

So what is the problem? Is it the technology? Is it the development process? Or is it even the team?

In our experience, it’s none of these. There exists the same intrinsic conflict of objectives in every software project that causes and reinforces most of the problems mentioned above.

What are the conflicting goals?

To be successful with their software project, the team must meet two conflicting requirements:

  1. Doing the right thing. In other words, develop something that provides great value to the user or client and is thus successful in the market. In software development, we are operating in an area of high uncertainty. In order to do the right thing, we therefore have to learn fast and with minimal effort, which means:

    • building prototypes to answer the most important open questions quickly,
    • responding flexibly to new requirements,
    • releasing new versions quickly and always incorporating new feedback.

  2. Doing things right. In other words, develop the system with a long-term view. Software development is mostly about developing and operating systems over a long period of time, so:

    • selecting technologies that are future-oriented,
    • writing good and maintainable code,
    • ensuring the scalability of the system
    • choosing an architecture that enables the implementation of future requirements.

While on the one hand things have to be kept as simple as possible in order to be able to act quickly, on the other hand one wants to act with as much foresight as possible.

This conflict is not easy for a company to solve, as there are strong forces pulling in both directions.

Especially here in the German-speaking countries - from our experience - the urge ’to do things right’ is very strong and often dominant. For example, a German car manufacturer spent seven(!) years developing a new system - in the process with over 100 people - before a first version of the application went into test mode.

The other extreme also exists: Something that was only meant as a prototype goes into production and suddenly has to be continuously updated. This causes a lot of problems. Development takes longer and longer, bugs keep popping up, the code becomes less and less maintainable.

Of course, these are only examples, but the alternation between the extremes is something that can be observed again and again in software development in one form or another.

How to escape from this dilemma?

The key to the solution is to recognize that there are several ways to move quickly and respond flexibly. On the one hand, one can simply do all things ‘faster’. This usually comes at the expense of quality and ends up in the problems mentioned above.

The far better way is to simply not do unnecessary things and instead focus on what offers the greatest benefit - both for the user and for the further development. You then have the necessary time to do these things properly.

The tenth principle of the Agile Manifesto gets straight to the point: “Simplicity–the art of maximizing the amount of work not done–is essential.”

So the question a team must ask is: What can we eliminate and still make our software development sustainable in the long term?

When it comes to technology, this is usually much.

However, the team also has to think about the technical requirements. What is the core problem of users? Where is the greatest benefit of the application? Is a 2-factor authentication with its own mobile app really necessary from day one? A design sprint can help to get a clear perspective here. It is also a question of the right mindset, which should be present especially in senior/lead developers and architects.

In this post, we’ll stick with technology and look at a simple example to illustrate the point.

Example: Ordering system for gastronomy

In this example, an ordering system for restaurants shall be developed, where customers can directly order food online for pickup or delivery. A pilot phase of 3 months with 10-20 restaurants is planned for the launch of the system. Each restaurant expects to receive approximately 100 orders per week. Shorter system outages of less than 15 minutes are acceptable during this phase.

Now the task is to select the technologies and roughly plan the architecture before development starts. The software team consists of four developers, a designer and a product owner. The team’s first idea is to start with a modern microservice architecture. The system should be deployed in the cloud to take advantage of its high availability and scaling capabilities. However, is this the right solution at this stage?

A microservice architecture is particularly well suited to enable several teams to work independently of each other and thus scale software development. The problem does not exist in this example and will only become relevant when the team grows by a factor of 5 to 10. Other advantages, such as the use of different technologies, do not offer any significant benefits from the team’s perspective at this stage. So a microservice architecture will only increase complexity at this point, but will not provide any benefit.

Of course, this may change in the future. However, that is still a long way off. Until then, there will be many changes in the requirements and also in the implementation. In order to keep the development simple for now, the decision is made in favor of a monolithic architecture. However, the internal structure of the system is already being tailored so that a change to a microservice architecture can be implemented later if needed.

The expected load of the system is relatively low. Since short failures are not a problem at the beginning, mechanisms for scaling and high availability are not necessary for the time being. So this complexity can also be removed for now. In order to be able to quickly add capacity and availability later as needed, the team opts for an EC2 (Elastic Cloud Computing) instance at AWS (Amazon Web Services). In this way, the team has the option to quickly add mechanisms for scaling and high availability at a later point in time.

As you can see from this simple example, the team has already taken a lot of complexity out of development by making these two choices. Furthermore, it keeps the door open for further adjustments by these choices. If the team’s other decisions follow the same principle, they should be able to enter the pilot phase with an initial version of the ordering system after no more than three months of development and move forward quickly from there.

In order for this to be feasible, it is certainly not enough to apply the principle of simplicity - in the sense of maximizing the work that is not done - only to technology. At least as important is the focus on the truly relevant requirements. If both are given, the team has the chance to move forward and learn quickly, while at the same time developing a system that is sustainable in the long term.

Conclusion

Any software development struggles with the intrinsic conflict of on the one hand trying to ‘do things right’ and on the other hand ‘doing the right things’.

To resolve this conflict, the tenth principle of the agile manifesto is key: “Simplicity – the art of maximizing the amount of work not done – is essential.”

Basically, it’s about eliminating as much complexity as possible from the start. This applies equally to technology, architecture, implementation, requirements, processes and communications.

Following this principle leaves enough time to do the remaining ’things right’ and still learn quickly whether one is ‘doing the right things’.

Webinar: Scaling software development as a SaaS company without becoming consumed in the growth process.

The webinar addressed exactly the pain points that are occurring as we grow. With an ever-increasing number of employees on the Dev & Product team, we've had to find out for ourselves how difficult it is to set up a sensible structure that brings out the maximum efficiency and performance in everyone, while also leading to a healthy working atmosphere.
Pascal von Briel, Co-Founder & CPO @ exporto GmbH