1
 
 
Profil
In deinem persönlichen Profilbereich kannst du den Status deiner Bewerbung einsehen, unvollständige Bewerbungen zwischenspeichern und aktuelle News und Events einsehen
01. Dezember 2022

Scaling Agile @ OTTO

What is the article about?

Agile Development is definitely nothing new at OTTO – the term has long since reached Peak Hype. The first generation of Developers for whom 'Agile' is ancient history (or simply their regular working routine) are now almost Senior Devs. In brief, the term has lost its gloss and what we might now call 'Agile' is often a mix of Agile, Lean, XP and other working principles.

Quite a bit has already been written about our working approaches at http://otto.tech – but comparatively little about product development in cross-functional teams, and absolutely nada about 'triads'. So it's high time we changed that!

Agile teams

At the beginning of OTTO's Agile journey from 2011 onwards, all the 'Feature teams' (FTs) involved in the development of the Webshop were agile in the sense that they worked based on Scrum. Maybe not exactly according to textbook Scrum, but quite close. For a company that had previously driven traditional project management as defined by ITIL, with a pool of developers from which the project teams were recruited and with project durations of 18 months, this was already major progress.

Agile team (the Kanban board is hidden just around the corner).
Agile team (the Kanban board is hidden just around the corner).

To accompany the process we hired a consultancy that specialized in the introduction of Scrum. All teams were required to stick to THE PROCESS – in fact, they all had the exact same meeting sequence and schedule, from sprint planning right through to the retrospective.

This changed rapidly, as the teams were granted very far-reaching competencies at an early stage. In the target 'Lhotse' architecture, from the word go we distinguished between microarchitecture (for which the teams had full responsibility) and macroarchitecture (for which we established few fundamental rules).

After some discussion, the teams had adapted the processes to their own requirements latest by 2012.

These days we've long taken it for granted that methods like Scrum are tools, and that teams should be allowed to select the tools for their daily work themselves – and adapt these if required. Nevertheless, looking back I think it was a smart move that all of us, coming from a classic project organisation, shared a uniform working approach to begin with and only made adjustments when we'd understood well enough what Agile Development is all about.

Arbeitssituation
Arbeitssituation

“We are now in the “post-Agile” age. ‘Post’-something means that we are in an in-between period, where one thing has been incorporated into our culture and the next thing has not yet formed to the point of naming.“
-- Alistair Cockburn

Most teams today tend to work more in a Kanban system with the best of Scrum, i.e. daily standups and retrospectives.

What remain are the core values from the Agile world, enriched by certain elements from Lean, DevOps and XP:

  • Work at least in pairs
  • Focus on the customer
  • Work iteratively in small increments
  • Go into production continuously
  • Always benchmark your current state of progress
  • Test on the customer whether you are achieving your goals
  • Limit work-in-progress
  • Automate everything
  • Act responsibly
  • etc.

Some teams naturally are further down the line in their journey than others and in some places you can certainly still see some cargo-cult Agile.

Cross-functional teams

In order to develop high-quality software and keep an overview of the customer's needs, we need to get ourselves into a rapid feedback cycle. We develop the upcoming feature in small increments and make it explicit how we intend to measure success. We continuously measure whether we are achieving success as planned, and learn from this what our next development steps are.

One of many cross-functional teams
One of many cross-functional teams

In a complex system, nobody can predict how a specific change will be received by the customer, so only constant testing on the customer will help. Of course, this entails that deviations from whatever the plan is become normal – and that far-reaching solution-level plans which stretch too far into the future make very little sense. Staying aware of the objectives is all the more important in preserving clarity on which parameters should be measured.

“Without data you are just another person with an opinion.“ -- W. E. Deming

In order to develop software further in a working approach like this, you need a diverse range of know-how. To be able to work at high speed in a perpetual cycle, you definitely need these skills in your team – otherwise you'll constantly be chewing your nails during the frequent waiting times in your feedback cycle.

Incidentally, the teams must also be permanently able to go into production independently of other teams in a largely automated manner, which is why many teams at OTTO now practice Continuous Deployment: practically every single 'git push' triggers a deployment pipeline and therefore leads to a go-live. Thousands of deployments per day are now complete normality.

In our company, this means that all Feature teams are cross-functional. Whether it's Frontend and Backend Development, Operations, Quality Assurance, Web Analysts, or User Experience (UX), all the expertise needed to develop and operate software is available within the team itself as often as possible.

Some roles are also shared. For example, Quality Assurance could be handled by 'the Developers' instead of by a single, dedicated Tester. In other cases (e.g. Analysts, UX), one expert is assigned to several teams. However, cross-team roles like this are often an unfortunate compromise: either they only participate in every second routine meeting, or they have two at the same time – and somehow they only partly 'belong' to the team. A clear 1:1 assignment would be better but sometimes it's impossible to implement this sensibly and we have to live with a compromise.

As a general rule, some team-members have several roles. Although there are also explicit 'Frontend Developers' and 'Backend Developers', most teams prefer Developers who are at least willing to slip into a different role – for instance to test and explore the new software's limits for a day, or to plunge into the abyss of JavaScript and different browser versions. I should tell you at this point that we work with microfrontends and don't have a monolithic frontend. This offers many advantages and is worth a blogpost in its own right! One of the advantages, however, is that it often makes continuous customer-effective Feature Development at OTTO possible without interacting with other teams.

Work in a Feature team is therefore very diverse – which implies that every experienced person will come across things they still need to learn. Paradise for curious spirits!

You build it, you run it

Long gone are the 'bad old days' when a pure Development team threw a new chunk of software developed under time-pressure over the fence to an understandably risk-averse Operations team... or even argued with them about whether it's actually necessary to migrate to a newer version of the JDK again.

Instead, today all Feature teams also hold responsibility for operating 'their' software. Obviously, this expands the number of roles that a cross-functional team has to take on. Some teams have one or more 'Ops Specialists', whose main task is the operational topics, while in other teams the Developers share these tasks.

Since our entire infrastructure has now moved to the cloud, the boundaries between Dev and Ops are becoming increasingly blurred anyway. Infrastructure as Code sounds like a good thing (it is), but it still means that a team has to master an additional tools 'zoo'.

Not a stock photo – just two of my colleagues in day-to-day business
Not a stock photo – just two of my colleagues in day-to-day business

A common pattern is the daily assignment of operational topics to a 'pair' – pair programming is very common with us and pair operations, pair testing or simply 'pairing' is therefore normal in most teams. The so-called "Daily Action Pair" is responsible for everything that's difficult to plan, including critical bugs, analysing log files, looking at the metrics, improving the deployment pipelines, updating service dependencies, bringing the next Java version into production, improving the automation of standard tasks, being the person in charge for other teams' ad hoc requests – whatever's needed!

Nobody expects everyone in the team to be able to handle everything. It's enough if the overall team can complete all tasks, which means that the individual (even if this hurts) somehow becomes dispensable. DevOps is not a role, DevOps is a mindset.

Product Development teams

The Feature teams at OTTO are therefore Agile, cross-functional teams that are end-to-end responsible to a very large degree for their product (see also Products over Projects by Siriam Narajan). Unlike in a project organisation, at OTTO we develop software intended to be used by and for our business for several years. Software like this is never 'finished' and the initial development is not sufficient. Someone has to take responsibility for operation, bug-fixing, monitoring and of course further development.

We try to define and delineate the responsibility of the teams by business criteria as far as possible. For instance, Team X is responsible for the shopping cart, Team Y for the product recommendations. The goal is to define the teams so that they can deliver changes to the customer in most situations without needing to involve another team. With more than 40 teams currently working in parallel, this is our biggest challenge right now.

Feature teams are responsible for their product
Feature teams are responsible for their product

Wherever possible we try to avoid technical delineations (e.g. Team X does the frontend, Team Y the backend) or delineations by roles (e.g. Team X develops, Team Y runs the software). This is because delineations like this almost always lead in practice to tightly coupled teams which are hardly able to take action without involving other teams.

A team's product responsibility needs to be extensive because the team will gather both technical and business expert knowledge that can't be gathered to the same extent outside the team.

  • The team should be able to further develop the product functions 'close to the customer' and continuously carry out A/B testing.
  • Depending on the test results, further development should be adaptable. The fewer process hurdles there are, the better and more efficiently development will progress.
  • Since the team is also responsible for Operations, no one knows better whether and when measures are required to restore or maintain quality. The question of whether one 'technical' measure is more important than the next technical topic can be taken quickly and responsibly in a team that holds product responsibility.

At OTTO, among other things 'product responsibility in the team' means that product management – and therefore technical responsibility – is part of the team's mandate. For us, Feature teams are not pure Development teams made up of a bunch of techies. In each team, at least one person is responsible for the business roadmap and, as we will see in a moment, not in a secondary role, but in a decisive one.

Triads

In Scrum there's the outstandingly important role of the Product Owner. This role is generally placed outside the Development teams – so to speak as a proxy of the departments interfacing with the Development team – and it is in the hands of one person.

We found this very impractical.

If you're dealing with numerous teams and have to minimize their external dependencies (to avoid waiting times in Development – e.g. Team A can only continue with project X when Team B has completed something), the Product Owner role belongs in the teams. Only then can we avoid:

  • every tiny detail having to be worked out in advance, including all eventualities, and
  • waiting times occurring for every change or query, because you have to schedule a meeting with the Product Owner first.

A further problem is that there are always different drivers for the required development efforts, at least business and technical factors. Differing competencies are also called for to assess the urgency of the specific development case: is it more important to build the next feature, or do we need to improve monitoring first? Both are important – so we must not only make the decision from a business perspective, and neither purely from a technical one. 

Almost nobody is able to assess a fairly complex product from both a technical and a business perspective equally competently. It's also tremendously difficult to assign this role competently – so we decided not to attempt it! We split Product Ownership in each team into a business and a technical facet, with a Business Designer responsible for the business side and a Technical Designer for the technical side (cool labels are unfortunately not our strong point). Both have to agree on what the team will tackle next and neither can simply outvote the other – at least theoretically.

Triads
Triads

When we then add the Project Lead, Business Designers and Technical Designers form a 'triad'. This term is a little bizarre, as the Project Lead is neither responsible for a project (rather for a Product team or Feature team), nor are they the 'Lead'. They are simply responsible for processes and methods in the team – the Agile Coach where necessary, the team assistant, the communicator, the good soul of the team. We could also say that this role is the 'centre of lateral leadership responsibility'.

We would be smart to rename these roles, perhaps like this:

  • BD -> Business Lead
  • TD -> Technical Lead
  • PL -> People Lead

For us in the teams this no longer makes a difference, as every company has its own internal nomenclature. After a few years it's tough to get roles like this renamed anyhow – we might succeed in external communication at most.

Two-pizza teams

No, this is not the path we're pursuing – some things you just have to leave behind! Cross-functional teams with product responsibility are large, and to match our teams' appetite we'd need XXL American pizzas and a team coming straight from the 'Elbe' (not the river but the OTTO 'Elbe' canteen).

A delicious pizza in the OTTO canteen 'Elbe' – which feeds only one lone, hungry developer
A delicious pizza in the OTTO canteen 'Elbe' – which feeds only one lone, hungry developer

However, our admittedly large teams do work in two groups: one group in the team fills the backlog pipeline, so to say, while the other group processes it. Smooth coordination is only possible because it is one team – but that still makes a team pretty big.

Scaling Agile

We've grown from 4 to well over 40 teams in around 10 years and yet we still work in the same team structure. You could call that a success story – and it is!

Nevertheless, we are slowly reaching the limits of growth. It's quite easy to imagine that overarching topics (business topics as well as technical ones) can be discussed and decided on quickly and unbureaucratically in a small group, while with 40 autonomous teams involved, discussing some questions takes a little longer. The transformation of otto.de into a marketplace also makes the situation trickier: there are now over 100 teams to coordinate, ranging from the Webshop to Logistics. Balancing alignment and autonomy is no small feat.

Autonomy & Alignment
Autonomy & Alignment

So it's quite possible that we will soon combine several teams from a single business domain into an overarching structure. This will let us work (both professionally and technically) to achieve greater decoupling between the business domains so that more items can be decided in a small group. 

But it's also fully conceivable that we'll find a completely different solution. This journey never gets boring!


*Source: https://www.defensie.nl/binaries/content/gallery/defensie/content-afbeeldingen/onderwerpen/militaire-muziek/fanfare-korps-nationale-reserve/kenmerken.jpg?download

Read more about our working methods.

11 Personen gefällt das

0Noch keine Kommentare

Dein Kommentar
Antwort auf:  Direkt auf das Thema antworten
1661 + 8

Written by

Guido Steinacker
Guido Steinacker
Executive Software Architect

Similar Articles

We want to improve out content with your feedback.

How interesting is this blogpost?

We have received your feedback.

Cookies erlauben?

OTTO und drei Partner brauchen deine Einwilligung (Klick auf "OK") bei einzelnen Datennutzungen, um Informationen auf einem Gerät zu speichern und/oder abzurufen (IP-Adresse, Nutzer-ID, Browser-Informationen).
Die Datennutzung erfolgt für personalisierte Anzeigen und Inhalte, Anzeigen- und Inhaltsmessungen sowie um Erkenntnisse über Zielgruppen und Produktentwicklungen zu gewinnen. Mehr Infos zur Einwilligung gibt’s jederzeit hier. Mit Klick auf den Link "Cookies ablehnen" kannst du deine Einwilligung jederzeit ablehnen.

Datennutzungen

OTTO arbeitet mit Partnern zusammen, die von deinem Endgerät abgerufene Daten (Trackingdaten) auch zu eigenen Zwecken (z.B. Profilbildungen) / zu Zwecken Dritter verarbeiten. Vor diesem Hintergrund erfordert nicht nur die Erhebung der Trackingdaten, sondern auch deren Weiterverarbeitung durch diese Anbieter einer Einwilligung. Die Trackingdaten werden erst dann erhoben, wenn du auf den in dem Banner auf otto.de wiedergebenden Button „OK” klickst. Bei den Partnern handelt es sich um die folgenden Unternehmen:
Google Inc., Meta Platforms Ireland Limited, elbwalker GmbH
Weitere Informationen zu den Datenverarbeitungen durch diese Partner findest du in der Datenschutzerklärung auf otto.de/jobs. Die Informationen sind außerdem über einen Link in dem Banner abrufbar.