Previously: Tomcat Clustering, Varnish and Blue/Green deployment - part 1

In part 1, we set up a Tomcat cluster with parallel deployement and cluster-wide deployment. Go back to the first part if you have not read it, this article assumes that you have read it.

Installing Varnish in front of everything

Now, let’s install Varnish. For this to work, you need the curl vmod. Installing this vmod is outside the scope of this post, so it is assumed that it is already installed.

As a reminder, the problem we have is that the cache of the 2 versions of the app will be mixed up in Varnish since they have exactly the same URLs.

The solution

The solution for this is to modify the vcl_hash function in Varnish and to introduce the app’s version in the objects identifiers in the cache. 

Here is how the function looks like:

sub vcl_hash {
        hash_data(req.url);
        hash_data(req.http.host);
        if(req.http.X-Version) {
                hash_data(req.http.X-Version);
        }
        return(hash);
}

Of course, now we have to fill this “X-Version” header with some data. Here is the code you need:

import std;
import curl;
 
acl trusted {
    "10.0.0.0"/24;
}
 
sub vcl_recv {
[ ... some stuff ... ]
if ((req.url ~ "^/version"  && client.ip !~ trusted) {
error 403 "Forbidden";
} elsif (req.url ~ "^/version")  {
return (pass);
}
[ ... some stuff ... ]
call app_version;
[ ... some stuff ... ]
return(lookup);
}

sub app_version {
        if (req.http.Cookie ~ "version=") {
                # If there is a version cookie, then we use it
                set req.http.X-Version = regsub(req.http.Cookie, ".*version=([^;]+).*", "\1");
        } else {
                # If there is no such cookie we call the app to determine the last version
                curl.set_proxy("");
                curl.header_add("Host: "+req.http.host);
                curl.get("http://localhost/version");
                if (curl.status() == 200) {
                        set req.http.X-Version = curl.body();
                        curl.free();
                } else {
                        curl.free();
                        std.log("Got an error "+curl.error()+" from version API: generating a 500 error.");
                        error 500 "Internal Server Error";
                }
        }
}

Let’s walk through this:

  • The app_version function does the following:

    • If the request comes with a « version » cookie, then we use the value of this cookie to fill the « X-Version » header that will be used in vcl_hash.
    • Else, we make a call to an API in the Tomcat app that returns the version of the application:
      • This calls has to go through Varnish again (that’s why we call « http://localhost/version ») to use the load-balancing provided by Varnish. If we do not request through Varnish again, we have no garanty that the chosen backend is healthy.
      • As this call has no Cookie (thus, no session), it will be answered by the newest version of the app available in Tomcat.
  • The vcl_recv function must include:
    • A restriction on the API. It’s an internal API and must not be accessible from outside. The answer to this call must also not be cached.
    • Of course, a call to app_version.

App requirements

For this to work, a few things must be implemented in the app:

  1. It must be able to answer the API call.
  2. The version number must be different for each WAR file deployed in Tomcat.
  3. The app must place a Cookie called « version » which has the same value as the API answer on users that do not alread have it. This way, Varnish will call the API only for new users that have no cookies.

+ Astuce : the caching policy is here implemented in the app: it sends the responses with the right Cache-Control headers to tell Varnish what to cache and for how long.

! Avertissement : a lot of the VCL code has been stripped from this example. You still have to implement the rest of the configuration around this. This depends on your app.

tomcat-1.png

Results

Here is a graph of the trafic on one of our Varnish servers during a deployment.

tomcat-2.png

The deployment was made just after 9a.m. You can see a (very) little drop in the hit rate on Varnish. This is because new users arrive on the new version of the app and Varnish has no cache for this new version. But since at this moment most of the users still are on the old version, Varnish still serves most of the requests from its cache. As time passes, the cache for the new app warms up and more and more users are on the new app. The two processes (cache warmup and trafic transition to the new app) naturaly occurs at the same time, keeping the hit rate in Varnish high. On this infrastructure, the old app is usually undeployed 30 minutes after the new one was deployed.