Gallery

Technology you can bet on

Posted on 14 Oct 2008 at 12:37

"The infrastructure behind the software is load-balanced across several servers," he adds. "The back-end we run on hardware that would support Linux and Unix software. On application servers, we'd probably be using Intel or AMD multicore processors."

Paddy Power's datacenter is based in the Isle of Man, which O'Donnell admits is for tax reasons rather than because he "thought we'd find world-class hosting facilities there". But the company certainly isn't skimping on hardware. "We use IBM Informix [data servers] for the back-end stuff. We have Sun or Fujitsu application servers running Solaris. We've got an Apache front-end running on Solaris on Sun boxes, with Cisco load balancers. It's all state of the art."

And keeping the single database that handles all of Paddy Power's betting transactions running smoothly is O'Donnell's prime concern. "We're certainly handling thousands of transactions a minute," he said. "We're handling betting transactions, deposits, withdrawals, registrations, account look-ups - in terms of the database terminology, it's tens of thousands of individual look-ups a minute.

"We have four copies of our database - a primary, a backup, a disaster recovery and a reporting. They're pretty much identical copies. On busy days, we'll actually split transactional loads across the primary and the backup."

As well as spare copies of the database, the bookmaker also has spare processing cores on hand for busy periods. "In recent years, we've got capacity-on-demand deals, which basically means we can run a dozen processors [on the database] most of the year, but when we need it we can put on an additional six or eight - depending on what the requirement is - for a set number of days," O'Donnell added.

Peaks and troughs

Yet, even with spare processing cores or servers at hand, it isn't only the horses and jockeys who face steep hurdles on Grand National day: the bookmakers are praying their sites don't fall at the first. "Probably the biggest one, if you're choosing one event, is the Grand National, because it's the one day a year people who don't normally place a bet come out and want to place a bet," said Orbis' Boyle.

Paddy Power's John O'Donnell candidly admits his site, like other bookmakers', has suffered performance problems on National day. His company uses a modified version of an open-source load-testing package called The Grinder (http://grinder.sourceforge.net), as well as a number of self-developed apps, to test the performance of the site and predict server behaviour patterns on the big day.

"One of the biggest challenges from a technology perspective is the amount of change that happens in the environment on an annual basis," O'Donnell said. "It's not like you've got the same site you had last year and you anticipate that you're going to be 10% busier this year. Typically, our site gets heavier every year: there's more product on it, there's richer product, there's more graphics. The whole nature of the beast changes. You can't take anything from the previous years - you've got to run through your tests and take a blank worksheet. Every year you're starting from scratch."

Yet, even with extensive testing, O'Donnell admits it isn't a precise science. "You can predict what your volumes are going to be, but you can't predict how your site's going to behave. Sometimes, we simply get our modelling wrong. You're trying to anticipate user behaviour. You can try to steer them down various paths on the site, but sometimes you just don't get your engineering right and you can't service the capacity."