The Future of Programmatic Real Time Bidding

While real time bidding has been widely adopted for over a decade, the concept of real time optimization has been slower to develop. But, there has been some recent and exciting progress. While the auctioning process is programmatic, setting bids and campaign management are still done by a human. In the near future, all of display advertising could be automated. While not yet implemented in the wild, academics are currently researching the first steps to full automating RTB.
Grasping the process of real time bidding auctions is key to understanding how the industry will change. These auctions are currently managed by hand, but due to the sheer volume of data and the potential performance gains through its analysis, machine learning algorithms will eventually take over this role. The example below shows how the auction works, the problems managing it by hand present, and how a machine would solve those using real time optimizations.

The table above shows 4 bids by the same advertiser for four separate impressions on the same user. The default bid the advertiser has set is $8. During each impression auction, the base bid is modified in real-time by hundreds of factors chosen by the advertiser. Each auction has thousands of different advertisers competing to win the impression. The bid is actually for 1,000 impressions, so whoever wins the auction will have their bid divided by 1000 to determine what they pay. But why do these bids vary by almost $20 despite occurring just seconds apart?

Programmatic Auctions

The first bid starts at $8, is modified 2.5x because the user has yet to see an ad from this advertiser. ( 8 * 2.5 = $20). The website the user is on has historically had bad performance for this advertiser, so the bid is modified by 0.8x ($20 * 0.8 = $16). The time of day is 4:23PM, so the bid is modified by 1.05x. This process is repeated hundreds of times until the final bid is calculated at $23.50. The advertiser loses the auction though because another advertiser has outbid them. The following impressions and their auctions occur, each with the same process. The difference between bids is because each impression is different – the number of times that user has seen an ad, the time of day, the website the user is on, etc. These auctions last only 200 microseconds and occur billions of times every second.

A media buyer manages this process by creating the rules which modify each bid. The majority of advertisers analyze historical performance and make decisions based off of that data. A common critique of this system is the difficulty in managing all of the modifiers. Real time bidding algorithms are almost purely a subject of academic research and not used in the wild but they solve almost all of the difficulties in this space.

After a few days of performance, a media buyer can analyze the data and change a bid modifier. For example, halfway through the week they might see that Sundays perform poorly, and set a bid modifier for that day to 0.5. While decently effective, machine learning is incredibly more successful at this process.

Data Overload

During each auction hundreds of bid modifiers are applied. It is simply impossible for a single person to analyze the performance of each of these factors and create a bid modifier on a per impression basis. As discussed previously, most of this work can be done in advance by utilizing CTR and conversion rate prediction. This allows an advertiser to know, before the campaign even starts, what factors are meaningful and how to adjust them.

Delayed Optimizations

The second problem with optimizing bids by hand is the time delay. If an advertiser were to optimize their bid based on the day of the week (which they should), they would either have to do an analysis every single day, or wait a week and do one large analysis. A machine learning algorithm can instead analyze performance in real time. It will watch performance on Sunday morning, recognize the lower performance is caused by the day of the week, and lower bids for the rest of the day. This optimizations saves an entire day’s worth of spend. The algorithm does not stop there – it makes continuous real-time optimizations on every single bid factor.

Despite being in its infancy, real time optimization algorithms have shown promise. One study’s algorithm, in a test environment, outperformed other strategies 76% of the time. Not all studies have seen such success though. In the coming decade advertisers may transition into real time bidding optimizers, a first step towards a fully automated media landscape. Advertisers can prepare for this eventuality by working on small automated campaign optimizations.