By Tom Verbiscer
As Director of Software Architecture, Dev Ops, and Attribution at DialogTech, I can tell you that we have several “good problems to have.” One is the success of SourceTrak, our call tracking service, and, more specifically, keeping up with customer demand and scalability.
With SourceTrak you can track phone calls from your website back to any marketing interaction, including digital ads, keyword searches, emails, and web pages. This allows marketers to connect the dots between the online and offline journeys as leads transition from a website activity to a telephone call to visiting a business location.
While DialogTech has historically handled millions of telephone calls per month, handling the web session traffic of all of our SourceTrak customers has risen dramatically over the last few years. We now handle hundreds of millions of web session requests per month. So how exactly does DialogTech ensure we provide the most scalable and reliable platform to our customers?
As our customer base grows, and as our customers’ businesses grow, our volume of tracked web sessions continues to rise. We routinely examine our infrastructure and growth trends in order to assure that we will be prepared for month-over-month and year-over-year growth.
Because of this ongoing diligence, we are able to identify potential bottlenecks in our current infrastructure. By addressing these roadblocks aggressively, we continue to provide room for future growth.
Our call tracking solution runs primarily on Amazon Web Services (AWS). AWS is a comprehensive, evolving cloud computing platform used by some of the largest enterprise brands and public sector organizations, including Kellogg’s, Dow Jones, Airbnb, the FDA, and hundreds more. Many of AWS components are horizontally scalable, meaning that we can quickly and easily add more servers, and there is a direct relationship to the amount of additional capacity we gain.
We previously used a relational database to store configuration data, application runtime data, and data used for historical reporting. Unfortunately, relational databases are not as flexible when it comes to adding capacity.
Therefore, we have replaced our relational database with a non-relational database, which is known for its ability to scale in a horizontal manner. While it can place restrictions on the ways that data is aggregated and explored, in our case of retrofitting an existing application, the data use cases are fairly well known. We can structure the data to allow for quick and easy access to what is required, while gaining the ability to incrementally scale up our reporting database.
Much like the adage of “half of my marketing dollars are wasted, but I don’t know which half,” when collecting metrics on web session traffic, the first and last touches are often the most interesting. The problem is, one doesn’t know which touch is the last one until some point down the road. Consequently, every touch must be stored and considered the valuable “last touch” until moments later when another touch occurs. At this moment, that prior touch is suddenly far less interesting. This process significantly adds to the load on the relational database.
To combat this, we’ve implemented an in-memory buffer system to initially hold every touch. This is a very fast (and redundant) data store used to temporarily keep each touch without the need for the slower write to the durable data store. This offloads a significant portion of the database load and allows us to do fewer writes to long-term storage.
With these practices and solutions in place, we continue to reduce the overall load on our database systems and gain horizontal scalability. Overall, we have created a software architecture that can scale in small increments. We retain the same application feature set, while continuing to serve our existing customers, large and small.
By investing in a flexible and scalable approach, we can accommodate large new customers in a short timeframe without risk to our existing user base. We also create a strong base on which we will continue to add new functionality to keep SourceTrak the market leader in the call tracking space.
If you’d like to learn more about call tracking and our SourceTrak technology, let’s chat! Request a demo today.
Get the latest straight to your inbox.