Introduction¶
Table of contents:
- What is Varnish?
- Benefits of Varnish
- Open source / Free software
- Varnish Software: The company
- What is Varnish Plus?
- Varnish: more than a cache server
- History of Varnish
- Varnish Governance Board (VGB)
What is Varnish?¶
Varnish is a reverse HTTP proxy, sometimes referred to as an HTTP accelerator or a web accelerator. A reverse proxy is a proxy server that appears to clients as an ordinary server. Varnish stores (caches) files or fragments of files in memory that are used to reduce the response time and network bandwidth consumption on future, equivalent requests. Varnish is designed for modern hardware, modern operating systems and modern work loads.
Varnish is more than a reverse HTTP proxy that caches content to speed up your server. Depending on the installation, Varnish can also be used as:
- web application firewall,
- DDoS attack defender,
- hotlinking protector,
- load balancer,
- integration point,
- single sign-on gateway,
- authentication and authorization policy mechanism,
- quick fix for unstable backends, and
- HTTP router.
Varnish is Flexible¶
Example of Varnish Configuration Language (VCL):
vcl 4.0;
backend default {
.host = "127.0.0.1";
.port = "8080";
}
sub vcl_recv {
# Do request header transformations here.
if (req.url ~ "^/admin") {
return(pass);
}
}
Varnish Cache and Varnish Plus¶
Topic | Varnish Cache | Varnish Plus |
---|---|---|
VCL | Yes | Yes |
varnishlog | Yes | Yes |
varnishadm | Yes | Yes |
varnishncsa | Yes | Yes |
varnishstat | Yes | Yes |
varnishhist | Yes | Yes |
varnishtest | Yes | Yes |
varnishtop | Yes | Yes |
directors | Yes | Yes |
purge | Yes | Yes |
ban | Yes | Yes |
force cache misses | Yes | Yes |
Hashtwo/xkeys (Varnish Software Implementation of Surrogate Keys) | Yes | Yes |
vagent2 | Yes | Yes |
Massive Storage Engine (MSE) | No | Yes |
Varnish Administration Console (VAC) | No | Yes |
Varnish Custom Statistics (VCS) | No | Yes |
Varnish High Availability (VHA) | No | Yes |
SSL/TLS frontend support with hitch | Yes | Yes |
SSL/TLS backend support | No | Yes |
Varnish Cache is an open source project, and free software. The development process is public and everyone can submit patches, or just take a peek at the code if there is some uncertainty on how does Varnish Cache work. There is a community of volunteers who help each other and newcomers. The BSD-like license used by Varnish Cache does not place significant restriction on re-use of the code, which makes it possible to integrate Varnish Cache in virtually any solution.
Varnish Cache is developed and tested on GNU/Linux and FreeBSD. The code-base is kept as self-contained as possible to avoid introducing out-side bugs and unneeded complexity. Therefore, Varnish uses very few external libraries.
Varnish Software is the company behind Varnish Cache. Varnish Software and the Varnish community maintain a package repository of Varnish Cache for several common GNU/Linux distributions.
Varnish Software also provides a commercial suite called Varnish Plus with software products for scalability, customization, monitoring and expert support services. The engine of the Varnish Plus commercial suite is the enhanced commercial edition of Varnish Cache. This edition is proprietary and it is called Varnish Cache Plus.
Table 1 shows the components covered in this book and their availability for Varnish Cache users and Varnish Plus customers. The covered components of Varnish Plus are described in the Varnish Plus Software Components chapter. For more information about the complete Varnish Plus offer, please visit https://www.varnish-software.com/what-is-varnish-plus. A list of supported platforms can be found in https://www.varnish-software.com/customers/#platforms.
Note
Varnish Cache Plus should not be confused with Varnish Plus, a product offering by Varnish Software. Varnish Cache Plus is one of the software components available for Varnish Plus customers.
Varnish Cache and Varnish Software Timeline¶
- 2005: Ideas! Verdens Gang (www.vg.no, Norway’s biggest newspaper) were looking for alternative cache solutions
- 2006: Work began: Redpill Linpro was in charge of project management, infrastructure and supporting development. Poul-Henning Kamp did the majority of the actual development.
- 2006: Varnish 1.0 is released
- 2008: Varnish 2.0 is released
- 2008:
varnishtest
is introduced - 2009: The first Varnish User Group Meeting is held in London Roughly a dozen people participate from all around the world
- 2010: Varnish Software is born as a spin-off to Redpill Linpro AS
- 2011: Varnish 3.0 is released
- 2012: The fifth Varnish User Group Meeting is held in Paris Roughly 70 people participate on the User-day and around 30 on the developer-day!
- 2012: The Varnish Book is published
- 2013: Varnish Software chosen as a 2013 Red Herring Top 100 Europe company
- 2013: BOSSIE award winner
- 2013: Varnish Software receives World Summit on Innovation & Entrepreneurship Global Hot 100 award
- 2014: Varnish Plus is launched
- 2014: Varnish 4.0 is released
- 2015: Varnish API Engine is released
- 2015: Gartner names Varnish Software as a 2015 ‘Cool Vendor’ in Web-Scale Platforms
- 2015: Varnish Plus supports SSL/TLS
- 2016: Varnish in the cloud
- 2016: Varnish 5.0 is released
VG, a large Norwegian newspaper, initiated the Varnish project in cooperation with Linpro. The lead developer of the Varnish project, Poul-Henning Kamp, is an experienced FreeBSD kernel hacker. Poul-Henning Kamp continues to bring his wisdom to Varnish in most areas where it counts.
From 2006 throughout 2008, most of the development was sponsored by VG, API, Escenic and Aftenposten, with project management, infrastructure and extra man-power provided by Redpill Linpro. At the time, Redpill Linpro had roughly 140 employees mostly centered around consulting services.
Today Varnish Software is able to fund the core development with income from service agreements, in addition to offering development of specific features on a case-by-case basis. The interest in Varnish continues to increase. An informal study based on the list of most popular web sites in Norway indicates that about 75% or more of the web traffic that originates in Norway is served through Varnish.
Varnish development is governed by the Varnish Governance Board (VGB), which thus far has not needed to intervene. The VGB consists of an architect, a community representative and a representative from Varnish Software.
As of November 2015, the VGB positions are filled by Poul-Henning Kamp (Architect), Rogier Mulhuijzen (Community) and Lasse Karstensen (Varnish Software). On a day-to-day basis, there is little need to interfere with the general flow of development.
What is new in Varnish 4?¶
- Version statement
vcl 4.0;
req.request
is nowreq.method
vcl_fetch
is nowvcl_backend_response
- Directors have been moved to the
vmod_directors
- Hash directors as a client directors
vcl_error
is nowvcl_backend_error
error()
is nowsynth()
, and you must explicitly return it:return (synth(999, "Response"));
- Synthetic responses in
vcl_synth
- Setting headers on synthetic response bodies made in
vcl_synth
are now done onresp.http
instead ofobj.http
. obj.*
invcl_error
replaced byberesp.*
invcl_backend_error
hit_for_pass
objects are created usingberesp.uncacheable
req.*
not available invcl_backend_response
bereq.*
invcl_backend_response
vcl_*
prefix reserved for builtin subroutinesreq.backend.healthy
replaced bystd.healthy(req.backend_hint)
client.port
andserver.port
replaced bystd.port(client.ip)
andstd.port(server.ip)
- Cache invalidation with purges is now done via
return(purge)
invcl_recv
obj.*
is now read-onlyobj.last_use
is retiredvcl_recv
must now returnhash
instead oflookup
vcl_hash
must now returnlookup
instead ofhash
vcl_pass
must now returnfetch
instead ofpass
restart
in the backend is nowretry
, this is now calledreturn(retry)
, and jumps back up tovcl_backend_fetch
- default VCL is now called builtin VCL
- The builtin VCL now honors
Cache-Control: no-cache
(and friends) to indicate uncacheable content from the backend remove
keyword replaced byunset
X-Forwarded-For
is now set beforevcl_recv
session_linxger
has been renamed totimeout_linger
and it is in seconds now (previously was milliseconds)sess_timeout
is renamed totimeout_idle
- Increasing
sess_workspace
is not longer necessary, you may need to increase either workspace_backend or workspace_client thread_pool_purge_delay
is renamed tothread_pool_destroy_delay
and it is in seconds nowthread_pool_add_delay
andthread_pool_fail_delay
are in seconds now- New parameter
vcc_allow_inline_c
to disable inline C in your VCL - New query language to filter logs:
-m
option replaced by-q
The above list tries to summarize the most important changes from Varnish Cache 3 to Varnish Cache 4. For more information, please visit: https://varnish-cache.org/docs/4.1/whats-new/index.html
If you want to migrate your VCL code from Varnish 3 to Varnish 4, you may be interested in looking at the varnish3to4 script. See the VCL Migrator from Varnish 3 to Varnish 4 section for more information.