Buffering, burstables, and better websites

Most people know the frustration of a lagging website. You want to watch another show on Netflix, but it won’t stop buffering. You want to open another tab, but the browser crashes. You want to click a link, but your cursor turns into the spinning beach ball of death. Luckily, people like Carlee Joe-Wong are working to make website performances—and the revenues of the people who run them—the best they can be.

Joe-Wong is an assistant professor in Electrical and Computing Engineering at Carnegie Mellon University Silicon Valley. She researches computer and information networks, with a particular focus on economics and how to apply theory to practice. For the last year and a half, she has been investigating how cloud providers can optimize their use of burstable instances.

A cloud provider is a company that delivers computing-based services and infrastructure. Users usually pay cloud providers for a fixed amount of resources so they can host websites and other information online. However, sometimes a website can get a sudden influx of visitors, and the fixed amount isn’t enough anymore. The users then have to choose: pay for higher services they normally don’t need, or let their website buckle under the weight of the traffic.

Burstable instances attempt to circumvent this problem. “With burstable instances, you get a baseline amount of resources most of the time,” Joe-Wong says, “and then you can request bursts of resources, which is a temporary increase in the amount of resources available to you.” This means that users can customize their resources to fit their needs.

Although burstable instances don’t cost much, cloud providers still need to regulate them to prevent people from bursting all the time and using up all available resources. Most providers handle this with something called a token bucket. “A token is essentially a virtual piece of money,” Joe-Wong says. “You can accumulate tokens at a predetermined rate over time, and then when you want to request a burst, you have to spend some of your tokens.”

Burstable instances are relatively new, and the research community has done little to understand them from a theoretical perspective. Joe-Wong hopes to fill that gap. She worked with a visiting Ph.D. student, Yuxuan Jiang, and collaborators at Princeton on a framework that can be used to model, analyze, and optimize burstable instances.

“We were looking at a couple different things,” Joe-Wong says. “How fast can you accumulate tokens? How often can you burst? And how does this affect the cloud provider’s ability to manage everybody’s burst requests?”

They referenced various empirical papers, adjusting and adapting techniques along the way. Joe-Wong and Jiang mostly worked on the theoretical model, while the Princeton collaborators worked to validate it. The end result is a model that will be able to help both cloud providers and users alike.

“We used our model to figure out how cloud vendors should price these different classes of burstable instances,” Joe-Wong says. “And that allows them to actually get more revenue.” If application developers use burstable instances better, then that could translate to cheaper services for users.

The model could also help users pay for resources more efficiently. “Any given user has a profile of needs,” Joe-Wong says. “So we look at that and ask, which class of burstables is better for you? Is the baseline service class enough, or is it actually worth it for you to pay for a higher service class?”

Their paper on the topic, Burstable Instances for Clouds: Performance Modeling, Equilibrium Analysis, and Revenue Maximization, was presented at the IEEE International Conference on Computer Communications. The team has hopes it will inspire more conversation in the area. “This is really the first model about burstable instances, and there’s lot of ways it can be refined,” Joe-Wong says. “So I definitely hope other people will follow on to this.”

The team is also going to continue investigating this topic. “We’re continuing to work on this project and refine the model,” Joe-Wong says. “We’re still trying to make it more realistic and investigating better methods for optimizing user experience.” With any luck, the model will only get better. Goodbye, spinning beach ball of death; hello, revenue and high performance.