To facilitate development between projects, the team that develops Infomaniak’s webmail has grouped the code for the apps into a single repository (monorepo). This approach is designed to improve maintenance processes, facilitate collaboration and accelerate ecosystem evolution. Webmail is a large-scale pilot project with several interconnected apps (kDrive, Email, Calendar, Contacts) and more than 1 million daily users. Our developers talk about this transition, its benefits, its challenges and also its negative impacts.

We will also look to enhance the performance of our solutions through our own organisation and processes. Julien Arnoux, VP of Engineering at Infomaniak

Why switch to a monorepo architecture?

The Webmail life cycle requires us to adapt our code repository strategy. Until now, the different parts (projects, libraries) of Webmail were developed in separate code repositories. One team developed the code of a library for its project independently without passing on the version changes in real time to other projects in progress. Monitoring was becoming complex:

“Developers working on transverse changes systematically had to locate and align the versions of each library and project that had been manually modified. This could result in a lot of extra manipulations. The idea was to abandon all these intermediate steps and focus on development itself.” Zachary Volpi, Webmail Developer at Infomaniak

Projects could also be out of syncif a team had a faster release cycle than other dependent libraries. The cumbersome manual multirepo publishing and updating also increased tedious manipulations and the risk of errors to the detriment of value creation. 

The benefits of monorepo architecture

With this new platform, our DevOps approach becomes more efficient. Automated cycles make it possible to compile, test and deploy code error-free (in particular the threat of human error). These advances make it possible to streamline all the application delivery steps.

1. Simplified code pooling and validation 

With a monorepo architecture, source code versions are shared between all parts of the project. A single change takes effect everywhere without the risk of breaking an adjacent feature. No need to target the right version for each dependency. Our scripts detect projects linked by an update and initiate validation processes associated with the modified areas.

2. Easier management of DevOps processes

Centralising the code allows pipelines to be created for the entire project with huge time-savings. Our continuous integration (CI) and continuous delivery (CD) pipelines are becoming more thorough with proven, predictive and repeatable processes. It makes it easier to add new portions of code.

We can now couple pipelines depending on the stage (build, test, deployment). Jobs are started in parallel, which was impossible or severely restricted in multirepo. As each test set is common, the development of features that affect multiple apps is much faster.

3. Easier collaboration and better overview

Developers can retrieve and integrate an entire modification in progress, thereby enhancing overall understanding. No need to go through multiple projects to detect hidden issues; everything is at your fingertips. We modify and test immediately, shortening the validation circuit. Changing context is also easier: a developer working on several projects or having to incorporate the work of a colleague can compare differences much faster.

4. Improving the quality of the code

We no longer have to duplicate, embed a library into a parent project or constantly go back and forth. Having the entire code to hand makes it easier to identify problems more quickly. We can also lighten the apps by removing dead code without fear of it being used elsewhere. Sharing the same environment reduces discrepancies between teams.

When we modify a library, these verifications are done first: does the code meet current standards? Does it work as expected? We then run tests of the apps that use it to confirm its successful integration. It is an onion (layered) approach, moving from the simplest (unitary) tests to the most complex (end-to-end).

5. Accelerating development

The monorepo facilitates the regular release of new features. We receive modifications from other developers much more quickly. Together with process automation, this makes production set-ups more harmonious. With a faster feedback loop, we can iterate more quickly.

Switching to monorepo: what does it change?

There is nothing better than a diagram to illustrate the impact of the monorepo on our organisation:

Improvement that requires several modifications to the different libraries used in the project:

With the monorepo:

Implementing monorepo in a production environment

We wanted to make the transition completely transparent for our customers and almost entirely transparent for the employees working on the project. 

“Unlike a fledgling project, modifying the organisation of the Webmail code involved huge consequences, as the app is in production. It’s like opening a construction site on a main road.” Zachary Volpi, Webmail Developer at Infomaniak 

Thetransition had to be divided into several steps to avoid too big a change that could have triggered service unavailability, with no possibility of going back. Therefore, a lengthy process of preparation and validation took place:

  • Splitting the migration into several phases by incorporating the potential impact for colleagues.
  • Testing each step beforehand where possible.
  • Detailing each step with a “ready-to-use” task list for the go-live day.
  • Planning migration milestones during periods of low traffic.
  • Simulating crisis scenarios to anticipate problems and plan actions to be implemented.

Our team has also developed incident documentation procedures to allow for a certain degree of back-pedalling.

The choice of PHP

DevOps is often written in bash or python, rarely in PHP. The DevOps pipelines we intended to use had to be understandable for all our colleagues. Our apps are also written in PHP and it is entirely possible to use it for scripting. We also already had runtime environments configured and available for further steps. It was therefore easier to reuse them directly instead of using blank sheets of paper.

Finding the right changeover pace

Prolonged periods of transition were to be avoided. Each task therefore had to be carried out over a period long enough to ensure security and stability, but relatively short in order to avoid creating fragility in the system.

The main challenges posed by migration

Migration without the possibility of going back

This had an impact on the entire workflow (structure of the steps, extraction of the common portions, file locations, modification history) and all the work environments of the Webmail team (production, development, testing, package release, notifications, etc.). It was therefore important that the transition phase should not last too long.

In multirepo, each modification to a component requires manual adaptation and verification of each of its dependencies. In monorepo, almost everything is automated: as soon as a component is modified, the entire system is automatically updated. It’s a bit like switching from a manual gearbox to an automatic gearbox on a car. Everything is so much simpler! Julien Arnoux, VP of Engineering at Infomaniak

Consider and anticipate all scenarios

It’s paradoxical, but it took a lot of complexity to simplify everything. Centralising the code means rethinking everything. To define the pipeline triggering logics, we had to understand their impact on each project and develop the understanding of the context that goes with it for:

  • the stages (continuous integration + continuous development)
  • the jobs common to the apps and libraries
  • the different build stages
  • the execution of the tests and validations by the parts concerned

The main challenge was that we couldn’t test all the steps under real conditions, especially those in the production environment. Thanks to the ability to concentrate gitlab-ci definitions, we were able to separate our different scripts by theme and impact area. This has drastically reduced file duplication and other copy-paste problems.

Balance sheet: monorepo or multirepo? 

Was it a good idea to switch to a monorepo? The answer is yes.

The new organisation offers major benefits:

  • Simplified retrieval of modifications.
  • Pooling of the configuration of some tools.
  • Reduced handling during modifications on multiple applications.
  • Simplified development: everything is in one place and it is much easier to navigate between areas.

This development also has negative impacts:

  • It is less easy to find your way around than in a conventional repository. The learning curve of a newcomer is a little longer.
  • Knowledge of CI/CD tools, in particular gitlab-ci, is required to make adjustments to pipelines.

The developers who handled the transition to monorepo

Julien Arnoux, VP of Engineering at Infomaniak (but never afraid to get his hands dirty):

I have been a developer since I was 15 years old and I love everything related to the automation of our workflows. When I develop, and depending on my mood, I can listen to hard-rock/metal as well as drum’n’bass. Actually, I listen to a little bit of everything, but these two styles really give me a boost 😅

Zachary Volpi, Full Stack Developer in the Webmail team:

I started development in August 2014. I like to see (and understand) all aspects of application development. I’m always looking for ways to simplify a process or achieve better performance. When I’m working, I adapt the music according to what I want to do: it goes from chill to dub or metal. As for references, I find logrocket’s blog very relevant. They have very well-designed articles on very broad as well as cutting-edge subjects. Otherwise, I follow the channels of the tools I use directly (on X & LinkedIn mainly).

Should you also use a monorepo architecture?

Come talk to our developers about X.