09 Oct 24

Processing flow in VLC

The Varnish Configuration Language (VCL) is at the heart of Varnish, one of the most powerful and flexible HTTP caching servers in the world. This domain-specific language allows you to customize how you handle requests, employ caching, and optimize the interaction between the origin server and the client.

If you want to acquire skills with this language, the first thing is to understand its logic and the way it is executed.

Language with a granular and dynamic approach

Understanding the processing cycle in VCL is crucial to taking full advantage of Varnish’s capabilities. It is certainly the first step to designing advanced rules that improve the efficiency of your web infrastructure.

Thanks to the granular approach of this language, you can intervene at key moments and thus adjust the caching logic to ensure that your web application or API works optimally.

VCL allows you to write a series of subroutines that are invoked at different stages of the request life cycle.

One of the most powerful and versatile features is its ability to dynamically rewrite requests and responses, allowing subroutines to be activated only when relevant to the current request.

Understanding VCL Processing Logic

Instead of executing your code sequentially, as in other programming languages, VCL code is triggered at different stages of the request, providing fine-grained control over how requests and responses are handled within the server.

And unlike other languages, VCL doesn’t run independently, but rather its instructions are processed at specific times in the lifecycle of an HTTP request. This way, you can control at a fine-grained level what goes in and out of your cache. 

esquema flujo VCL

These phases mainly include:

vcl_recv: This is the first stage that is triggered when Varnish receives a request from a client. Here you can decide whether the request should be served from the cache or passed to the origin server. It is also the ideal place to perform initial validations and rewrite URLs, apply authentication policies, or redirect requests to other servers.

For example to determine which backend server will handle the request based on the domain of the original request:

sub vcl_recv {
    if (req.http.Host == "mi-sitio.com") {
        set req.backend_hint = backend1;
    } else {
        set req.backend_hint = backend2;
    }
}

vcl_backend_response: When a request cannot be served from the cache and is sent to the origin server, this subroutine handles the server’s response before it is stored in the cache. Here you can decide whether you want to store the response or just deliver it to the client without storing it.

This snippet checks if the response from the backend has the Cache-Control header set to private. If so, it is delivered directly to the client without being cached:

sub vcl_backend_response {
    if (beresp.http.Cache-Control ~ "private") {
        return (deliver);
    } # Caching by default.
}

vcl_deliver: the final step where the response is sent to the client. Here you can modify the response headers or log information about the request. For example, you can add custom headers to track response time or cache status.

This would be an example of adding an X-Cache header to the response that indicates whether the request was served from the cache (HIT) or had to be fetched from the backend (MISS).:

sub vcl_deliver {
  set resp.http.X-Cache = "MISS";
  if( obj.hits >0 ) {
    set resp.http.X-Cache = "HIT";
}

Basic Tip: It’s important to remember that while VCL has similarities to other programming languages ​​(such as conditional constructs and subroutines), you’re ultimately configuring the behavior of a cache server, so every decision you make will directly impact the performance and latency of your application.

Optimize log management and monitoring in Varnish

Varnish does not write log files by default due to performance reasons, but instead uses an in-memory circular buffer. To improve your diagnostic and monitoring capabilities, you should familiarize yourself with tools such as varnishlog, varnishncsa and varnishstat, which will allow you to analyze real-time information about requests and responses. This will help you identify bottlenecks and optimize your caching policies more accurately.

How to advance your knowledge of VCL

Varnish is a widely used technology and there are many resources to improve your VCL skills. Some of the ones we suggest are:

  • Webinars and Tutorials on Varnish Website: The official Varnish website offers a wide variety of webinars and tutorials that can help you dive deeper into VCL best practices and advanced features.
  • Official documentation: The Varnish manual is an excellent reference source. Take the time to explore the sections dedicated to cache optimization to understand how to apply VCL to complex projects.
  • You can also check out The Varnish Book, which provides a detailed guide for users of all levels, helping you master the basics without a steep learning curve.
  • Forums and communities: Joining forums like Varnish Cache or participating in communities like Stack Overflow will allow you to resolve specific questions and learn from the experience of other developers.

Developing and improving your VCL skills will not only allow you to take full advantage of the power of Varnish, but will also give you a competitive advantage by managing your edge applications accurately and efficiently.