Surviving deadline day – how we built a photo competition system to scale

Recent Work

In 2014 we were asked to build an online system, from the ground up, to sell and manage entries for the BJP’s International Photography Awards – a prestigious competition where photographers around the world enter series of photographs in the hope of winning a gallery show in London and a place in the British Journal of Photography. By the time we closed it down, around 5,000 images had been received, and our server was feeling the strain.

Since then we’ve used iterations of this system for three annual IPA awards, two seasons of the Breakthrough award for new photographic talent, and two annual Portrait of Britain competitions, the latest of which has just closed. The most popular awards now attract several thousand entries, with the count of individual images running in five figures. Each time we close down an iteration of the system we’ve analysed the results to see how it could be improved, and what we can learn from the entry pattern.

The biggest lesson we’ve learned? There’s nothing quite like a deadline to give your infrastructure a workout.

Even in an open call that runs for months, a very large proportion of all entries will arrive in the few hours just before the deadline. A recent example: over 2,000 photographers entered this year’s Portrait of Britain, submitting around 8,000 images in total, and of those over 25% entered so in the 24 hours before the deadline. Requests to our servers climbed throughout deadline day and peaked around half an hour before the midnight cutoff time:

We’ve used this pattern to inform not only how our competition infrastructure is managed during an open call period, but also how it’s designed. Rather than managing server hardware, much of which would sit idle during less busy periods, for the last two years our application has been deployed to Heroku dynos – logical cloud-based containers – allowing us to immediately scale it by adding additional dynos in parallel at any point – even automatically, based on the load.

All image data is stored in Amazon S3 – providing an effectively infinite amount of storage space, and allowing us to perform uploads directly into that infrastructure – avoiding a key bottleneck that held back our first system. For additional security, a separate server provides a 24/7 ongoing backup of all images to another location. Processing thousands of entries requires many thousands of emails, so we send those via Mandrill (owned by podcast-favourites MailChimp). Payments are handled via Stripe‘s rock-solid credit card infrastructure, while a promotional code system built into our front-end allows for discount campaigns or free entries.

Our multi-dyno formation, Stripe, S3 and Mandrill services give the platform the ability to stay up and keep processing both new entries and image uploads even through the busiest final few hours of a competition. With the deadline passed, we can scale the formation back down – but the process of running a competition doesn’t end there.

Having thousands of images, and tens of thousands of items of metadata, is useless without being able to sort and view them. Our platform handles judging online, allowing multiple judges – whether they’re around the world, or together in one room in London, to filter and browse the entries, working towards a shortlist, and then a winner. Our system manages this process to the very end – where the winning images can be downloaded, and the winners contacted. In some cases, an additional component is used to request even higher-quality images from shortlisted photographers.

The judging team for Portrait of Britain has just reached the end of the process, and we expect the winning images to be announced shortly – with the results on show on JCDecaux digital billboards all around the UK later this year.

Meanwhile, we’re already gearing up the next iteration of the system for another competition due to open shortly. Interested in working with us on a competition? Drop us a line – hi@mndigital.co.