:: By Mike Azevedo, Clustrix ::
In an ideal world, the end users of click tracking and affiliate marketing services would be blissfully unaware of what's going on under the hood of the platforms they use.
If you are leveraging these services, however, it’s important to understand that just as not every online advertising approach is created equal, not every online marketing services platform has an equal technical backbone.
Affiliate marketing is transaction intensive and demands fast response times - both of which put serious strains on databases. A little basic knowledge about how a service provider has built their “technical backbone” (specifically how they’re meeting the technical challenges of scaling their database) will go a long way toward picking a partner that will not saddle you with unsightly outages and other problems you may otherwise face.
One affiliate marketing company--Afftrack--which offers a SaaS platform from which to build, deploy, manage and optimize affiliate marketing campaigns--faced some of the most intense technical challenges that the industry presents. Their story shows how the company accounted for a variety of key factors and built a technical foundation that not only enables it to handle heavy demands, but also to sustain an innovative new pricing model.
Affiliate marketing can be brutal on a database
Affiliate marketing leverages varied advertising methods such as organic SEO, PPC, email marketing, content marketing and display advertising. Afftrack offers a platform that enables affiliate marketers to accurately track clicks in email, banner ads and on mobile devices, and gives advertisers and agencies full statistical analysis for the affiliate and email campaigns. It’s software includes fraud prevention, campaign planning, deployment, targeting, monitoring and optimization.
All of these features placed heavy demands on the database engine that powers Afftrack’s platform. The company required an engine that could handle not only their application platform, but also billing, account management, and support infrastructure on top of the already complex application platform. Their workload included three data types: transactional data that is stored forever; summary tables for fast queries; and static data that doesn’t change very frequently. The variety of database transactions involved--which range from PPC to fraud prevention--required a database engine that can process massive concurrent, mixed workloads without slowing down.
Supporting high growth, and a new pricing model
Affiliate marketing success is measured in terms of metrics like site visitors or customer conversions. In contrast to standard CPM-based affiliate marketing tracking platform prices, Afftrack’s flat-fee model allows advertisers to run a virtually unlimited number of affiliate marketing campaigns for one low, predictable price. Therefore, it may be no surprise that its flat-fee pricing and all-in-one platform that includes backend accounting, have been wildly popular--to the point of disrupting the industry, and driving tremendous company growth. However, all this success, combined with a service model that inherently involves more data throughput, placed tremendous strains on the database engine that powered Afftrack’s services.
Afftrack, like many other ad tech companies, started out leveraging MySQL--a fairly common option for many applications that require what is referred to as ACID (Atomicity, Consistency, Isolation, Durability) compliance--standards that ensure data integrity in applications where error is intolerable. They discovered that when their services were required to scale in the face of high growth, MySQL posed insurmountable limitations, and had to look for another solution.
There’s scaling, and there’s scaling: understanding horizontal versus vertical scaling
Afftrack found itself at a place in which making the wrong choice could seriously affect services. In order to avoid problems such as outages and delays, it needed to be able to scale massively. Yet “scalability” in the database world is a loaded term - methods for scaling differ dramatically in ways that have serious implications for a company’s ability to achieve sustained growth.
Vertical scaling, which is inherent in MySQL and most MySQL-derived solutions, can be boiled down to a process of expanding capacity/performance by running the database on a bigger, more powerful computer. This causes several problems when companies are trying to keep up with increased demands. Using just one machine means that downtime is unavoidable, because once you’ve maxed out the capacity on your existing server, migrating to a larger one requires taking your application offline for a period of time. Companies may try to mitigate this by scheduling system upgrades at off-peak times, but this becomes increasingly problematic as the number of users and diverse demands increase. In other words, the more successful you are, the more you’ve got to lose with downtime.
Additionally, in affiliate marketing - which relies on accurate, up to date data in order to create a real-time picture of what is happening - holes in the available data caused by downtime could obscure the very insights that the campaign is trying to identify.
Scaling “out” by adding nodes offers a better approach
Horizontal scaling, on the other hand, involves increasing database capacity by linking multiple servers (referred to as “nodes”) together so that they function as a logical whole--though the database resides on multiple servers, the application views it as one big database. The difference between horizontal and vertical scaling is like the difference between a train and a bus. If you’re relying on a bus - which is akin to vertical scaling - once you run out of room you’ve got to get a bigger one. This means you’ve got to unload your cargo and reload it onto the new bus. And, just as there are limits to the physical size of a bus, there are also limits to the size of the available servers, and this ultimately caps your scalability.
When you’ve reached maximum capacity on the biggest available server, there are methods that can be used to ratchet up database performance, but they come with a price. Afftrack, for example, considered read slaves - one common method for scaling MySQL - but this would have made the task of managing the database on an ongoing basis very complex, and it also would have required making some undesirable changes to their platform application.
If you need to haul more on a train on the other hand, you can add cars, which is akin to scaling out by adding server nodes, or “horizontal” scaling. Horizontal scaling allows for adding more machines with no theoretical upper limit, and practical upper limits far in excess of those found with vertical scaling. The horizontal scaling approach also offers the advantage of distributing the downtime: if one of the servers goes down, the operation can continue running. This gives the affiliate marketing platform provider the ability to deliver continual service without interruptions that might be caused up system upgrades, server outages or other events.
Building the foundation for a bright future in affiliate marketing
Afftrack ultimately decided to swap out MySQL for a MySQL replacement of the “NewSQL” variety, which allowed them to scale horizontally while maintaining ACID compliance. They are currently expanding their footprint to 12 data centers regionally distributed across the U.S. and Canada to offer affiliate marketers 99.999 percent uptime, lightning fast service and immunity to virtually any network failure. By choosing a horizontally scaling option, they have also been able to sustain its flat-fee pricing model.
Choosing a service partner for affiliate marketing services can be complex, but knowing about the “under the hood” challenges companies like this are facing and how they're addressing them will help you find partners that are making strategic choices that support long-term objectives. If the provider is leveraging a vertically scaling database, they may very well be “kicking the can down the road,” as scalability problems might not be felt for a little while, but will inevitably surface and have to be dealt with. And when considering a platform for affiliate marketing services, it's wise to ask whether the solution is ready for the long haul. By making the right choice from the outset, you’ll never have to think again about what's under the hood.
About the Author
Mike Azevedo is CEO at Clustrix, provider of the first scale-out relational database designed to meet the needs of large, fast-growing e-commerce sites and other big Web- and cloud-based applications.