Why You Need to Understand Caching for Voice-Powered Apps

Amazon CloudFront is a powerful ingredient for any voice first application, including Alexa Skills.

The ability to build and deploy an entire web application, with a back-end API, without provisioning a single server, is amazing. The ability to connect a VUI (Voice User Interface) to the infrastructure is equally amazing. With this increased convenience comes a steep learning curve, and many new concepts to master. Caching is only one mechanism essential to serverless voice and web applications.

CloudFront provides a powerful caching solution for any voice-powered serverless application.

How CloudFront Helps our Apps

Let’s start by understanding the benefits of using a web cache in general. Web caches increase the overall performance of our applications by 1) reducing redundant data transfers; 2) reducing network bottlenecks; 3) reducing demand on origin servers; and 4) reducing distance delays (Gourley and Totty 161).

Voice-powered applications such as Alexa Skills work with existing back-end infrastructure, so the same caching benefits apply.

Amazon CloudFront provides all the benefits listed above, with a special emphasis on reducing network bottlenecks and reducing distance delays. The Amazon CloudFront Developer Guide’s section “What is Amazon CloudFront” emphasizes CloudFront’s network enhancements:

CloudFront speeds up the distribution of your content by routing each user request through the AWS backbone network to the edge location that can best serve your content.

In addition to the networking benefits, CloudFront takes care of reducing delays caused by the distance of the client requesting a resource and the location of the resource on the origin server. CloudFront does this by utilizing over 200 Points of Presence (POPs) in 77 cities across 37 countries. Think about that level of scale. We can now ensure that users all over the world will be able to quickly access our content. We can do this with a couple of button clicks.

The Cache Life-cycle

Next, we will break down the typical cache life-cycle and see how much of it is implemented behind-the-scenes when we use CloudFront. A typical cache goes through 7 processing steps:

1. Receiving the request from the network;
2. Parsing the message, extracting the URL and headers;
3. Checking for a local copy of the requested object, or fetching a new copy;
4. Checking if the local copy of the object is fresh, and requesting a new object if it is not fresh;
5. Sending the response to the requesting client, including new headers and cached body;
6. Optionally logging the transaction

(Gourley and Totty 172)

CloudFront handles receiving, parsing, and checking for local copies automatically. The Amazon CloudFront Developer Guide’s section “How CloudFront Delivers Content to Your Users” describes how CloudFront uses a DNS algorithm to route the incoming request to the CloudFront POP closest to the requesting client’s location. When the request hits the POP, CloudFront checks the POP’s cache for the requested file. If the file does not exist, it grabs it from the origin server.

This is very powerful stuff, and you get it automatically.

The fourth point above deals with “freshness checks”. The modern way for a cache server to determine whether a cached object is fresh, is to check the Cache-Control:max-age header. The Mozilla Developer Network’s “Cache-Control” section provides the following definition of the Cache-Control:max-age header. Specifically, the cache-control:max-age header …

Specifies the maximum amount of time a resource will be considered fresh. Contrary to Expires, this directive is relative to the time of the request.

Again, you get this automatically when using S3 as the origin server, and may never need to fiddle with any cache-control policies.

Let’s look at a potential scenario with a voice-powered application (Alexa Skill). Imagine a clothing store that provides a daily deal to users. The daily deal varies each day and applies to a specific clothing product. Having a daily deal encourages users to invoke the skill daily to see whether they can get a deal on something they like.

From the store’s perspective, they could provide the daily deal as a text-to-speech file, or they could provide a recording (preferred method). Considering a day is equal to 24 hours, we can implement this functionality automatically, while maintaining all the benefits of using a cache.

Automatically Refreshing Content in a Voice App (Alexa Skill)

The voice workflow above takes advantage of the 24-hour CloudFront default expiration for files in a CloudFront cache. There are 2 traditional cache mechanisms at work here: 1) the expiration check; and 2) object revalidation. CloudFront checks the cache-control: max-age header and determines whether the object is expired. If the object is not expired, it is immediately returned to the requesting client. If the object is expired, a 2nd check is conducted whereby CloudFront checks either the If-Modified-Since header, or If-None-Match header while analyzing the ETag on the document. This is how CloudFront determines whether the object in the cache should be updated with the new object from the S3 bucket.

So, if we look back at the clothing store scenario. The users of the Alexa Skill will hear the new daily deal every 24 hours. The skill administrators would simply need to upload a new file to the S3 bucket, overwriting the existing audio file.

For the sixth process in the cache life-cycle, you could use one of the many monitoring options for CloudFront distributions.

Conclusion

In this work we touched on the life-cycle of a typical cache server, and aligned the process with a modern CloudFront cache. We also examined how we can use default settings to quickly and easily add dynamic functionality to a voice-powered application. Performance is important in serverless web applications, and equally as important in voice-powered applications. Going forward, CloudFront will be an indispensable part of our toolkits as serverless, voice application developers.

References

Gourley, David, and Brian Totty. HTTP The Definitive Guide. O’Reilly, 2002.

Amazon CloudFront Developer Guide. Amazon Web Services, docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide, Accessed 7 November 2019.

“Cache-Control” Mozilla Developer Network, developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Cache-Control, Accessed 7 November 2019.