OTTO Display Network & COMPASS – #1 Market Modelling

Orchestrating and optimising display advertising campaign management – i.e. to display the optimal advertising medium to every user at the right moment and at the best-possible price – is the daily challenge of online marketing at OTTO. Besides our own campaign portfolio, it’s also important to take into account the campaigns of those platform partners for whom OTTO acts as an agent.
For some time now, OTTO has relied on an in-house developed programmatic stack, the OTTO Display Network (ODiN). Through the correct application of data science, the implementation of campaign selection and the bidding service can be automated to a high degree. The following blog post is intended to provide a small insight into this implementation – the product we call COMPASS.

Before we get to grips more intensively with the COMPASS management component, let’s take a quick look at the general requirements of display advertising and the use of the ODiN set-up.

Programmatic advertising concerns the purchase and sale of advertising space. Ad spaces are offered on the web by marketers and can be purchased via auction. For the vendors who want to place their products, the main goal is to tune the advertising content optimally to its recipients. This needs to be fully automated and carried out in real time. An infrastructure which consists of the SSP (Supply Side Platform) and the DSP (Demand Side Platform), as well as the use of data management platforms, has established itself on the market. These platforms are operated by intermediaries acting between the marketers and the providers, and who charge for their services. In order to save these costs, OTTO decided to build its own in-house solution ODiN, which enables a direct connection to the marketers and therefore avoids extra costs paid to middlemen. This not only makes savings possible when purchasing advertising banners, but at the same time also provides transparency and access to a large amount of data generated during auctions. By connecting to a large number of marketers, OTTO reaches about 90% of users in the German-speaking region and participates in about 1.5 billion auctions every day.

The ODiN ecosystem comprises several components, which interact with one other:

In the following we will take a closer look at the COMPASS management component. COMPASS processes gigantic amounts of data and applies AI to determine the appropriate advertising content and the optimal bid level for every single user: this improves visibility, raises inventory quality and enables a deeper understanding of the market than was previously possible.

To select appropriate advertising content, OTTO can not only leverage its own brand, assortment and promotion campaigns, but also the marketing campaigns of OTTO platform partners. Campaign management needs to take both user-relevance and the target KPIs of the individual advertising campaigns into account. Thus, a volatile set of campaigns exists with different content, running times, daily budgets, possibly different optimisation requests, and a cookie pool of known and unknown users. Banner advertising, as described above, is auctioned in real time in digital auctions. Defining the optimal bid in order to maximize campaign performance is therefore the overriding objective.
The following sections describe how we use our knowledge of users to assign campaigns to the optimal recipient groups, and how we calculate perfect bids to optimise each campaign.

COMPASS can be broken down into three main components. Environment analysis is carried out within market modelling, assigning users to suitable campaigns takes place on the basis of KPI forecasts and assigning and determining bids are defined via bid optimization. These three areas will be explained in greater detail below.

Just like many other optimisation problems, Compass also initially concerns gathering information and that way getting to know and understand the advertising market.
Communication between the websites and the Orbidder creates enormous amounts of data in the form of auction logs. With 1.5 billion auctions and 1.5 billion bids, as well as tens of millions of winning notifications , several terabytes of data accumulate every month. The Orbidder's infrastructure is so powerful that it can process traffic peaks of 50,000 requests per second – and even higher. Here’s an extract of a dataset:

In raw form this data contains everything we need to know about the market, and as a result can be processed according to different market characteristics. We focus on the following three aspects:

1. Auction Density: distribution of auctions throughout the day
2. Auction Timeseries: itemised forecast of the number of auctions per day
3. Bid Landscapes & Hard Floor Price: winning probability forecast, depending on the bid

Auctiondensity delivers an empirical description of auctions over the course of the day. It is therefore a measure of German-language website users’ surfing intensity .

In the evaluation, one of several interesting insights is that traffic on Saturday and Sunday increases on average one hour later than on the other days of the week, due to the ‘late-riser’ effect.
The lunch-break peaks from 12:00 onwards are only evident on weekdays, and apparently many users’ weekends already start on Friday at noon. Furthermore, there is generally less web surfing on Saturdays before peaking for the week on Sunday afternoon. This helps us to estimate at any time how many auctions will still be available for the rest of the day.

Auction timeseries
In order to gain detailed knowledge of which advertising spaces will come up for auction, we model thousands of time series for auction volume to a very fine level via fbprophet, determine short and long-term trends, and log the forecasting error. Our modelling levels are:
1. URLs (netloc: e.g. m.spiegel.de),
2. PlacementIDs (clusters of AdSlots similar in format and website placement)
3. Knowledge of the auction user.

The model and forecasting are recalculated daily.

Bidlandscapes & HFP
Bidlandscapes allows us to visualise the relationship between the auction bid and the probability of winning. As in the Auction Timeseries, here we also look at each URL, placement and user status separately. In addition, the time and day of the week also play a role in deriving the probability of winning a bid. To learn these connections we use a Bayesian model and insights from auction theory. Here, we not only look at correlations at the current moment, but also predict the change in competition over the course of the day and week. We also visualise what the bidlandscapes would look like if there were 50% more or 50% fewer competitors in the market. The following examples show that it makes sense to model the relationships individually:

The plots on the left show the bidlandscape in blue and the winning probability in green/red with less/more competition. Plot A and B are based on exactly the same placements (size and position of advertising media) of the same websites. The only difference is that the user is unknown in A but known in B. A 1 € bid with unknown users already wins 100% of the auctions (users unknown to us are usually unknown everywhere on the web and therefore uninteresting for almost all competitors). On the other hand, a 1 € in B wins about 25% of the auctions (because these users are also known by other companies and interesting for them).
Hardfloor prices (HFP) are not shown in the plot but are also tracked; an HFP represents a hard lower limit for permitted bids per PlacementID – bids below this limit are automatically rejected.

This level of market analysis is extremely exciting, as it helps us understand how competitors act and why campaign performance changes retrospectively; it also opens up a new way of working with publishers.
Most importantly, though, it forms the basis of our bidding strategies – which we’ll explain in more detail in an upcoming blog post!