Magento2 Varnish Cache Warmer

Magento2 module for handling Varnish Cache cleaning and regenerating using admin panel
and CLI. It uses multiple process to run the commands to minimize the time required
to run.

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Prerequisites

For release 1.0

• Magento 2.2
• PHP 7.1

For realease 2.0

• Magento 2.3
• PHP 7.1/7.2

For realease 2.2

• Magento 2.3
• PHP 7.1/7.2/7.3

For realease 2.3

• Magento 2.3/2.4
• PHP 7.1/7.2/7.3/7.4

For release 3.0

• Magento 2.4.6 - 2.4.9
• PHP 8.1 - 8.5

Installing

Download the module

Using composer (suggested)

Simply run

composer require lizardmedia/module-varnish-warmer

Downloading ZIP

Download a ZIP version of the module and unpack it into your project into

app/code/LizardMedia/VarnishWarmer

If you use ZIP file you will need to install the dependencies of the module
manually

composer require guzzlehttp/guzzle:^7.5

Install the module

Run this command

bin/magento module:enable LizardMedia_VarnishWarmer
bin/magento setup:upgrade

Usage

Admin panel

The commands can be run in backround from admin panel using Lizard Media Varnish Warmer
menu tab.

CLI

The commands can be run using Magento built-in CLI:

• lm-varnish:cache-purge-homepage - purges and regenerates homepage
• lm-varnish:cache-purge-general - purges and regenerates homepage and categories
• lm-varnish:cache-purge-wildcard - purges * and regenerates homepage, categories and products
• lm-varnish:cache-purge-all - purges and regenerates homepage, categories and products
• lm-varnish:cache-purge-force - purges * without regenerating. Omits lock check
• lm-varnish:regenerate-products-cache - purges and regenerates products
• lm-varnish:cache-refresh-url - purges and regenerates single URL given as argument

Command options

• store - sets store view ID to get a correct base URL, e.g.

php bin/magento lm-varnish:cache-refresh-url "*" --store=1

Configuration

Stores > Configuration > LizardMedia Varnish > Varnish Cache Warmer

• General
  • number of concurrent regeneration and purge requests
  • exclude 301 redirects from warming
  • HTTP authentication username and password - fill in when the store is behind HTTP basic authentication,
    e.g. on staging
• Purging
  • Enable purging - set to No when Magento already invalidates Varnish by cache tags and the module should
    only warm the cache. Set it to No as well without Varnish: warming fills Magento's built-in full page cache
    the same way, since it also keeps a separate entry per X-Magento-Vary cookie
  • custom purge hosts (comma separated, one per Varnish server) and the Host header sent with them

Cache variants

Every URL is warmed once without a cookie and, when guests of its store view get an X-Magento-Vary cookie,
once more with that cookie. The cookie is calculated the same way the storefront does it, including values other
modules add to the HTTP context with plugins on Magento\Framework\App\Http\Context::getVaryString().
To warm more variants add an after plugin on
LizardMedia\VarnishWarmer\Model\HttpContext\VaryCookieProvider::getVaryCookies().

Upgrading from 2.x

• Remove composer patches applied to this module, everything they fixed is included.
• If purging was commented out with a patch, set lm_varnish/purge/enabled to 0 instead
  (bin/magento config:set lm_varnish/purge/enabled 0 or app/etc/config.php).
• Classes extending LizardMedia\VarnishWarmer\Model\QueueHandler\AbstractQueueHandler or replacing
  VarnishUrlRegenerator need to be removed or rewritten. Cache variants, response codes and HTTP authentication
  are built in now. Queue handlers add PSR-7 requests with queue() and receive them in handleResponse().
• Admin roles without full access need LizardMedia Varnish Warmer > Cache handling to use the admin actions.
• API changes: PurgingConfigProviderInterface::isPurgeEnabled(),
  GeneralConfigProviderInterface::getHttpAuthUsername() and getHttpAuthPassword(),
  VarnishUrlRegeneratorInterface::addUrlToRegenerate(string $url, ?int $storeId = null).
  URL providers return store_id next to request_path.
• Run bin/magento setup:di:compile after the upgrade.

For developers

The number of process used for purging and regenerating Varnish cache can be customized
in the admin panel. The max number of processes is specified in
LizardMedia\VarnishWarmer\Model\Config\GeneralConfigProvider
If you have a powerful server you can modify those numbers to use more resources.

Tests

CI runs the suite against whole Magento releases the projects use (see .github/workflows/tests.yml),
installing Magento packages from the Mage-OS mirror. Locally composer reads them from repo.magento.com,
so it needs your Magento access keys. Pin a release and install (do not commit the resulting composer.json change):

composer config policy.advisories.block false
composer require --no-update magento/product-community-edition:2.4.6-p15
composer update

• vendor/bin/phpunit runs the contract tests: behaviour that must not change between releases.
• vendor/bin/phpunit --group known-bug runs tests describing known defects. They fail until the
  defect is fixed, and the fix removes the test from the known-bug group.

Contributing

Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests to us.

Versioning

We use SemVer for versioning. For the versions available, see the tags on this repository.

Authors

• Maciej Sławik - Initial work - Lizard Media
• Bartosz Kubicki - Fixes & maintenance - Lizard Media

See also the list of contributors who participated in this project.

License

This project is licensed under the MIT License - see the LICENSE.md file for details

2.1.

• fix for incorrect method for purging process
• adding logging for error request
• minor refactor

2.2

• major refactor
• adding missing return types/types hints
• support for PHP 7.3
• fix for case when urls uses store codes inside after domain

2.2.1

• fix for cache purging headers

2.3

• support for PHP 7.4

2.3.1

• fix for magento installation bug (lack of using proxy in console command class)

3.0.0

• support for PHP 8.1 - 8.5 and Magento 2.4.6 - 2.4.9
• HTTP requests sent with Guzzle (already required by Magento) instead of the abandoned react/http-client, so the module installs with composer/composer 2.10+ and react/promise 3; a new request starts as soon as one finishes instead of waiting for the whole batch
• requests time out after 60 seconds (10 seconds to connect) instead of hanging the process
• security: the URL and store view given in the admin panel reach the shell as escaped arguments, every admin action requires the "Cache handling" permission and "purge a group of URLs" runs only the listed processes
• purging can be switched off (Purging > Enable purging) when Magento invalidates Varnish by cache tags and the module should only warm the cache
• every URL is warmed for the cache variant guests of its store view get: without a cookie and with the X-Magento-Vary cookie the storefront gives them
• only URL rewrites of the selected store view are purged and warmed, with its store code in the URL when the store view uses one
• HTTP basic authentication for staging environments (General > HTTP authentication)
• the lock is released when an operation fails, a URL is warmed once regardless of the number of purge hosts, spaces around purge hosts are ignored
• the lock warning in the admin panel and the progress bar no longer fail with an error
• response code of every warmed URL is logged
• queue progress serialized with Magento\Framework\Serialize\Serializer\Json instead of Zend\Json
• BC break: see "Upgrading from 2.x" in README