Angeo LLMs.txt — Magento 2 Module

AI Engine Optimization (AEO) for Magento 2 / Adobe Commerce. Generates
spec-compliant llms.txt, llms-full.txt, and JSONL files so ChatGPT,
Claude, Gemini, Perplexity, and other LLM-powered crawlers can ingest your
catalog efficiently.

Current version: 3.2.0 — performance release. Opt-in single-pass
generation pipeline: one catalog pass per store renders all enabled
formats (default stays legacy; legacy pipeline and the old
ProviderInterface SPI are deprecated and will be removed in 4.0.0).
Builds on 3.1.1 (SQL-level stock filtering, price-index pricing, dedicated
cron group) and 3.1.0 (security & hardening). See
CHANGELOG.md for details and upgrade notes.

What this module does

After install, your storefront serves:

Generation happens via cron (daily by default), CLI, or the admin "Generate
Now" button. The output is streamed to disk with bounded memory, atomically
renamed on completion, and served with proper ETag / Cache-Control headers.

Why this module exists

LLM crawlers can ingest a typical Magento storefront — full theme, JS, image
sprites, navigation chrome — but that's wasteful for everyone. The
llmstxt.org standard defines a clean text format
optimized for AI ingestion: stable links, structured headings, descriptions
in their natural prose form rather than buried in product cards.

This module produces that format for Magento, with care taken for the things
Magento makes hard: multi-store layout, Page Builder content, CMS directive
resolution, customer-group pricing, and very large catalogs.

Installation

composer require angeo/module-llms-txt:^3.0
bin/magento module:enable Angeo_LlmsTxt
bin/magento setup:upgrade
bin/magento setup:di:compile      # only in production mode
bin/magento setup:static-content:deploy adminhtml   # only in production mode
bin/magento cache:flush

Then generate your first batch:

bin/magento angeo:llms:generate

Visit https://your-store.tld/llms.txt.

Configuration reference

All settings live at Stores → Configuration → Angeo → LLMs.txt.

General

Content

Output formats

Content sanitization

Page Builder strategies

The filter parses content with DOMDocument (not regex), so nested Page
Builder containers are handled correctly. Known content-types include:
row, column-group, column, tabs, tab-item, text, heading,
html, image, video, map, divider, spacer, buttons, button-item,
banner, slider, slide, products, block, dynamic-block.

Performance

HTTP caching

Cron

CLI commands

# Generate everything for all eligible stores
bin/magento angeo:llms:generate

# Single store, skip JSONL
bin/magento angeo:llms:generate --store=default --no-jsonl

# Per-store/per-format last-run status
bin/magento angeo:llms:status

# Lint generated files for spec compliance
bin/magento angeo:llms:validate

Extending — custom providers

Drop a new section into llms.txt (e.g. a "Brands" list, a "Recent Posts"
section, etc.) by implementing Angeo\LlmsTxt\Api\ProviderInterface and
registering it via di.xml.

namespace Vendor\Module\Provider\Llms;

use Angeo\LlmsTxt\Api\OutputContextInterface;
use Angeo\LlmsTxt\Model\Provider\AbstractProvider;

class BrandsProvider extends AbstractProvider
{
    public function provide(OutputContextInterface $context): iterable
    {
        yield "## Brands\n\n";
        foreach ($this->brandRepo->getList($context->getStore()->getId()) as $brand) {
            $label = $this->escapeMarkdown($brand->getName());
            yield "- [{$label}]({$brand->getUrl()})\n";
        }
        yield "\n";
    }
}

<!-- etc/di.xml -->
<type name="Angeo\LlmsTxt\Model\Generator\LlmsTxtGenerator">
    <arguments>
        <argument name="providers" xsi:type="array">
            <item name="brands" xsi:type="object">Vendor\Module\Provider\Llms\BrandsProvider</item>
        </argument>
    </arguments>
</type>

The base class gives you escapeMarkdown(), encodeJsonl(), isJsonl(),
isFullTxt(), and isApplicable() overridable to opt out per-format.

Extending — custom sanitizer filters

Insert your own filter between Page Builder and HTML stripping (e.g. to
remove <script> data attributes, redact phone numbers, etc.) by implementing
Angeo\LlmsTxt\Api\SanitizerFilterInterface and re-declaring the pipeline
in di.xml.

<type name="Angeo\LlmsTxt\Model\Sanitizer\Sanitizer">
    <arguments>
        <argument name="filters" xsi:type="array">
            <item name="cms_directive" xsi:type="object">Angeo\LlmsTxt\Model\Sanitizer\Filter\CmsDirectiveFilter</item>
            <item name="page_builder"  xsi:type="object">Angeo\LlmsTxt\Model\Sanitizer\Filter\PageBuilderFilter</item>
            <item name="redact_pii"    xsi:type="object">Vendor\Module\Sanitizer\Filter\PiiRedactionFilter</item>
            <item name="html"          xsi:type="object">Angeo\LlmsTxt\Model\Sanitizer\Filter\HtmlFilter</item>
            <item name="whitespace"    xsi:type="object">Angeo\LlmsTxt\Model\Sanitizer\Filter\WhitespaceFilter</item>
        </argument>
    </arguments>
</type>

Events

Hook in via observers — three events are dispatched per store/format pass:

Migrating from 2.x

• Old files in media/llms/ can be deleted (output now lives in media/angeo/llms/).
• Any custom ProviderInterface implementations must change from returning a string to yielding iterable<string>. See Extending — custom providers.
• Drop any reverse-proxy / Nginx rewrites pointing at the old paths.
• Re-run Stores → Configuration → Angeo → LLMs.txt to set the new fields (Page Builder strategy, customer group, etc.).
• External tooling that called the GET /admin/angeo_llms/generate/index URL must switch to the CLI command (the admin endpoint is now POST + CSRF).

License

MIT — see LICENSE.

Support

• GitHub Issues: https://github.com/angeo-dev/module-llms-txt/issues
• Email: [email protected]

Changelog

All notable changes to Angeo_LlmsTxt are documented in this file.

The format follows Keep a Changelog,
and this project adheres to Semantic Versioning.

[3.2.0] — 2026-06-10

Single-pass generation pipeline (opt-in). Fully backward compatible: the
default mode remains legacy, all pre-3.2 behavior, file paths, events, and
extension points keep working unchanged. Everything superseded is marked
@deprecated and will be removed in 4.0.0.

Added

• Single-pass pipeline (Model/Pipeline/SinglePassGenerator). With
  Stores → Configuration → Angeo LLMs.txt → Performance → Generation Pipeline = Single pass, each store's catalog is iterated once and every
  enabled format (llms.txt, llms-full.txt, llms.jsonl) is rendered from that
  one pass:
  • one frontend emulation per store (legacy: one per format),
  • one url_rewrite warm-up per store (legacy: one per format),
  • each entity loaded and sanitized exactly once (legacy: 2–3× per
    product description),
  • all format files written in parallel streams with atomic rename, under one
    per-store lock (media/angeo/llms/store_{code}.lock).
    Combined with 3.1.1 this gives roughly 3× faster generation on top of the
    3.1.1 gains, with identical output files.
• New @api extension points (implement these going forward):
  • Api\EntityProviderInterface — yields format-agnostic entity records once
    per entity (successor of the format-specific ProviderInterface);
  • Api\Data\EntityRecordInterface + Model\Data\EntityRecord — immutable
    record DTO carrying already-sanitized content;
  • Api\FormatRendererInterface — serializes records into one output format;
  • Model\Output\FilePathResolver — the single source of truth for generated
    file paths (used by both pipelines and the frontend controller);
  • Model\Text\Truncator — shared word-boundary truncation (the Sanitizer
    now delegates to it; behavior is byte-identical).
• Bundled single-pass providers/renderers registered via di.xml
  (SinglePassGenerator → entityProviders, renderers). Third parties add
  their own items the same way.
• Model/Config/Source/GenerationMode + new system.xml field
  angeo_llms/performance/generation_mode (global scope, default legacy).
• Unit tests: TruncatorTest, including the down-truncation invariant that
  guarantees single-pass renderers reproduce legacy truncation byte-for-byte.

Backward compatibility

• generation_mode defaults to legacy — upgrading changes nothing until
  you opt in.
• In single-pass mode the output files, on-disk paths, served URLs, generation
  status records, and the angeo_llms_generation_before/after/failed events
  (dispatched per format) are identical to legacy.
• Custom providers built on the legacy ProviderInterface keep working in
  both modes. In single-pass mode they are detected automatically (anything
  registered on the legacy generators beyond the bundled providers) and
  executed through a compatibility pass that appends their output to the
  corresponding format stream.
• The only semantic difference: the items counter in generation status now
  counts rendered records rather than raw stream chunks.

Deprecated (removal in 4.0.0)

• Api\ProviderInterface and Model\Provider\AbstractProvider — implement
  Api\EntityProviderInterface instead.
• All eight bundled legacy providers under Model\Provider\Llms\* and
  Model\Provider\Jsonl\* — superseded by Model\Pipeline\Provider\* +
  format renderers.
• Model\Generator\AbstractGenerator, LlmsTxtGenerator,
  LlmsFullTxtGenerator, JsonlGenerator — superseded by
  SinglePassGenerator; file-path resolution moved to FilePathResolver.
• The legacy generation mode itself: 4.0.0 ships single-pass as the only
  pipeline and removes everything listed above.

Changed (internal, not @api)

• Service\GenerationService routes by generation mode; new constructor
  dependency (SinglePassGenerator).
• Controller\Index\Index resolves file paths via FilePathResolver instead
  of the deprecated generators (constructor change).
• Model\Sanitizer\Sanitizer accepts an optional Truncator (defaults
  internally — existing instantiations and tests are unaffected).
• AbstractGenerator::getProviders() added so the single-pass pipeline can
  discover third-party legacy providers.

Upgrade notes

1. bin/magento setup:upgrade && bin/magento setup:di:compile
2. Optional but recommended: switch Performance → Generation Pipeline to
   Single pass, run bin/magento angeo:llms:generate, and diff the
   generated files against the legacy output for your data.
3. If you maintain custom providers, plan their migration to
   EntityProviderInterface before 4.0.0.

[3.1.1] — 2026-06-10

Performance release. No public-API changes; drop-in upgrade from 3.1.0.

Performance

• Out-of-stock filtering moved into SQL. Both ProductProviders now use
  StockHelper::addIsInStockFilterToCollection() (a JOIN on
  cataloginventory_stock_status) instead of one StockRegistry round-trip
  per product. On a 100k-SKU catalog with Exclude Out-of-Stock enabled this
  removes ~100,000 queries per format per store.
• Prices come from the price index. Product collections call
  addPriceData($customerGroupId, $websiteId); the final price (group-aware,
  special-/tier-price-aware) is read from the joined
  catalog_product_index_price column instead of invoking the PHP price
  calculation chain per product — which for configurable/bundle products
  lazy-loads child products (another hidden N+1). A per-product fallback to the
  legacy calculation remains for rows missing from the index (e.g. reindex
  pending).
• Dedicated cron group angeo_llms with use_separate_process=1
  (new etc/cron_groups.xml). Long generation runs no longer block
  default-group jobs (transactional emails, scheduled indexers, etc.).
• Default collection_page_size lowered 1000 → 500. Each page holds full
  HTML descriptions of every product in memory; 500 halves the peak without a
  measurable throughput cost. Explicitly configured values are unaffected.
• Duplicate-description sanitization skipped in llms-full.txt: when
  description is byte-identical to short_description (a common merchant
  pattern), the content is sanitized once instead of twice.

Behavior notes

• Exclude Out-of-Stock is now strict: products whose stock status cannot be
  resolved are excluded by the SQL filter, whereas 3.1.0 included them on
  lookup failure ("default in stock"). With a healthy stock index the output
  is identical.
• Prices require the price index to be up to date (bin/magento indexer:reindex catalog_product_price) — standard for any production store; stale index
  rows fall back to the slow per-product calculation rather than emitting a
  wrong price.
• The cron job moved from group default to group angeo_llms. If your
  crontab invokes bin/magento cron:run with explicit --group filters, add
  the new group.
• Internal constructor change (not @api): both ProductProviders now take
  Magento\CatalogInventory\Helper\Stock instead of
  StockRegistryInterface. Recompile DI (setup:di:compile); if you extended
  these concrete classes, update your constructors.

MSI note

Stock filtering still reads the legacy cataloginventory_stock_status table,
which MSI keeps in sync for the default stock. Multi-source/multi-stock setups
that need salable-quantity semantics per stock should override the providers —
now a single JOIN swap instead of a per-product call.

[3.1.0] — 2026-06-10

Security & hardening release following an external security code review.
Upgrading is strongly recommended for all installations, especially those
with the .md mirror feature enabled.

Security

• [HIGH] .md mirror no longer serves disabled or hidden entities
  (information disclosure). Controller/Index/MdMirror now verifies entity
  state before rendering: products must be Enabled, catalog-visible, and
  assigned to the current website; categories must be active; CMS pages must
  be active. Previously a stale url_rewrite row could expose embargoed,
  recalled, or intentionally unpublished content — including price and full
  description — at /{url_key}.md. Hidden entities now return the same 404
  as unknown paths, so their existence is not confirmed.
• [HIGH] .md mirror DoS mitigation. Rendered markdown is now cached in
  the Magento cache (tag ANGEO_LLMS_MD, TTL = configured HTTP Cache-Control
  TTL), so crawls no longer re-trigger entity loads, CMS directive resolution,
  and DOM-based sanitization on every request. Unknown paths are
  negative-cached for 5 minutes to blunt enumeration sweeps; request paths
  longer than 1024 bytes are rejected outright. The cache is flushed
  automatically after every generation run, so mirrors never serve a stale
  catalog state for a full TTL.
• [HIGH] Frontend router no longer hijacks the *.md URL space
  (route hijacking / availability). The router sortOrder moved from 10 to
  70 — after the urlrewrite (20), standard (30), and CMS (60) routers — so any
  real merchant content whose URL ends in .md always wins; this module only
  claims paths that would otherwise 404. The .md branch is additionally
  gated on the md-mirror feature being enabled for the resolved store: when
  the feature is off, the router declines the match instead of swallowing the
  request with a 404.
• [MEDIUM] Template-directive injection surface reduced for product content.
  {{block}} / {{widget}} / {{var}} resolution inside product attribute
  content (descriptions frequently imported from supplier/PIM feeds) is now
  controlled by a separate flag, angeo_llms/sanitizer/resolve_directives_products,
  default OFF. When off, directives found in product content are stripped —
  never resolved and never leaked as source. CMS pages and categories keep the
  existing resolve_directives behavior. On any directive-resolution failure
  the filter now strips directive source instead of returning it raw.
• [MEDIUM] HtmlFilter output-encoding fixes (stored-XSS defense for
  downstream consumers; secret-leak prevention):
  • HTML entities are decoded before the final tag-strip pass, then the
    result is stripped again — <script>…</script> can no longer
    materialize as live markup in the generated output.
  • Unterminated <script> / <style> blocks (and unterminated HTML
    comments) are removed to end-of-input, so inline JS — which can carry
    analytics tokens or API keys — can never leak into llms.txt,
    llms-full.txt, or .md mirrors.
• [MEDIUM] Wholesale-price disclosure warning. The Customer Group for
  Pricing admin field now carries an explicit warning that the generated
  files are public and CDN-cacheable, and that selecting a logged-in / B2B
  group publishes that group's negotiated pricing to the internet.
• [LOW] Admin error messages no longer expose exception internals. The
  "Generate Now" and "Schedule" actions log full exceptions to
  var/log/system.log and show a generic message in the admin UI.
• [LOW] X-Content-Type-Options: nosniff is now sent on all .md mirror
  responses and all 404 responses (previously only on the file endpoint's
  200 responses).
• [LOW] Admin status panel embeds its polling URL via json_encode()
  instead of raw string interpolation inside a <script> block, per Magento
  secure-rendering guidelines.

Fixed

• Large-file serving no longer loads the whole file into PHP memory.
  Controller/Index/Index streams files above 4 MB to the client in 256 KB
  chunks; concurrent requests for a multi-hundred-MB llms-full.txt can no
  longer exhaust the PHP memory limit. Content-Length is now always sent.
• All file serving goes through Magento's Filesystem abstraction —
  no native is_file / filemtime / file_get_contents on raw paths —
  making the endpoint compatible with Adobe Commerce Cloud remote storage
  (AWS S3) drivers.
• Generation status writes are now concurrency-safe.
  GenerationStatusRepository performs a locked read-modify-write (flock on a
  sidecar lock file) followed by an atomic tmp-rename, so parallel
  generators / cron / CLI runs can no longer lose each other's updates or
  leave a truncated status.json.
• "Schedule (Async)" no longer piles up duplicate cron jobs. A new run is
  only queued when no angeo_llms_generate row is already pending or running;
  the admin is informed otherwise.
• Corrected a misleading comment in MdMirror: the rewrite-lookup fallback
  appends the configured .html URL suffix (it never tried a trailing slash).

Changed

• UrlResolver::warmUp() streams url_rewrite rows from the DB cursor
  instead of fetchAll(), roughly halving peak memory on very large rewrite
  tables.
• New public API: AbstractGenerator::getRelativePath() (media-relative path
  of the generated file; preferred over getFilePath() for
  Filesystem-abstraction readers). getFilePath() is retained for backward
  compatibility.
• New well-known shared-context key
  OutputContextInterface::SHARED_ENTITY_TYPE; all bundled providers and the
  .md mirror publish it before sanitizing so filters can apply
  entity-specific policies. Third-party providers are encouraged to do the
  same.
• Admin field comments updated (md-mirror caching behavior, directive
  resolution semantics).

Added

• Config: angeo_llms/sanitizer/resolve_directives_products (default 0).
• Cache tag ANGEO_LLMS_MD for rendered .md mirrors (flush with
  bin/magento cache:clean or automatically on each generation run).
• Unit tests: HtmlFilter security regressions (unterminated script blocks,
  entity-encoded markup resurrection, legitimate < text preservation) and
  CmsDirectiveFilter product-content gating.

Upgrade notes

• Run bin/magento setup:upgrade && bin/magento cache:flush after deploying.
• If you relied on {{widget}} / {{block}} directives inside product
  descriptions being rendered into the generated files, re-enable this
  explicitly at Stores → Configuration → Angeo → LLMs.txt → Content
  Sanitization → Resolve Directives in Product Content after reviewing the
  security note on that field.
• If a customization called AbstractGenerator::getFilePath() to read
  generated files, consider migrating to getRelativePath() plus a
  Filesystem media read-directory for remote-storage compatibility.
• Behavior change: URLs ending in .md that collide with real merchant
  content are now served by that content (the mirror no longer takes
  precedence). URLs of hidden or disabled entities now return 404.

[3.0.5] — 2026-06-04

Admin-config bugfix. Safe drop-in upgrade from 3.0.x.

Fixed

• System Config "Save Config" no longer throws Cannot read properties of undefined (reading 'settings'). The Generate button frontend_model
  template (generate_button.phtml) rendered two <form> elements inside
  the admin system-config form (#config-edit-form). Nested forms are invalid
  HTML: the browser re-parents the inner inputs/buttons onto the outer form, so
  on Save the jQuery validator (jquery.validate.js metadataRules) iterated an
  orphaned submit button that has no rule metadata and crashed, aborting the
  whole submit. The buttons are now plain type="button" elements that POST via
  a JS-built form appended to <body> (outside the config form). CSRF
  protection is unchanged — the form key is still submitted.

Install-blocking bugfix plus PHP 8.5 support. Safe drop-in upgrade from 3.0.x.

Fixed

• setup:upgrade no longer fails XSD validation on etc/adminhtml/system.xml.
  Two <comment> elements (cache_ttl_seconds and schedule) contained raw
  <code> HTML without a CDATA wrapper. system_file.xsd only allows a model
  child inside <comment>, so the literal markup tripped
  Element 'code': This element is not expected. Expected is ( model ) and
  aborted module loading. Both comments are now wrapped in <![CDATA[ … ]]>,
  matching every other HTML-bearing comment in the file.

Changed

• Added PHP 8.5 to the supported range (…||~8.5.0). Intended for Magento
  2.4.9+, which is the first line to support PHP 8.5; on 2.4.8 and earlier,
  PHP 8.4 remains the recommended runtime.

Admin-config bugfix. No functional or API changes — safe drop-in upgrade
from 3.0.x.

Fixed

• System Config "Save Config" no longer throws a JS TypeError. Three
  numeric fields in etc/adminhtml/system.xml declared validation classes
  that are not registered in Magento's mage/validation ruleset
  (validate-greater-than-zero and integer). On 2.4.8-p4 the admin form
  validator (jquery.validate.js metadataRules) looks up
  settings on each rule object; the missing rules resolved to undefined,
  producing Cannot read properties of undefined (reading 'settings') and
  aborting the entire form submit. Replaced with registered rules:
  • collection_page_size: → validate-digits validate-digits-range digits-range-0-1000000
  • product_limit: → validate-digits
  • cache_ttl_seconds: → validate-digits

[3.0.4] — 2026-06-03

Compatibility patch. No functional or API changes — safe drop-in upgrade
from 3.0.x.

Changed

• Lowered the minimum PHP to 8.1 (~8.1.0||~8.2.0||~8.3.0||~8.4.0).
  The module uses no PHP 8.2+ only syntax, so it runs on 2.4.5 / 2.4.6 stores
  that are still on PHP 8.1 as well as on 2.4.7 / 2.4.8 (PHP 8.3 / 8.4).
• Broadened dependency constraints to cover 2.4.5 through 2.4.8. Every
  Magento dependency in require now uses an open lower-bound (>=) pinned to
  the major line that shipped with 2.4.5 — e.g. magento/framework: >=102.0
  and magento/module-url-rewrite: >=102.0. Because these major lines do not
  change between 2.4.5 and 2.4.8, the module installs cleanly across all of
  those minors. This replaces the earlier exact carets (such as the ^101.2
  on module-url-rewrite) that failed on 2.4.8, where that module ships as
  102.x.

[3.0.2] — 2026-06-03

Marketplace-readiness patch. No functional or API changes — safe drop-in
upgrade from 3.0.0.

Fixed

• Replaced md5() with hash('sha256', …) for ETag generation in the
  file-serving controller. The Magento Coding Standard forbids md5(); the
  ETag only needs to be stable and unique, so the switch is behaviour-neutral.
• Removed error-silencing @ operators from filesystem calls
  (fopen / flock / fclose) in the atomic-write lock helper and in the
  validate command. Return values were already checked explicitly, so
  dropping @ changes no behaviour while clearing the coding-standard errors.

Changed

• Dependency constraints pinned to real 2.4.x major lines. require now
  uses caret ranges matching the actual published modules — notably
  magento/module-url-rewrite: ^102.0 (the 101.2 line never existed). This
  resolves a composer require failure on clean 2.4.8 installs.
• Added an explicit version field (3.0.1) to composer.json so the
  package version matches the Marketplace submission form.

[3.0.0] — 2026-05-23

A full rebuild against the architectural review of 2.1.4. This release is
not drop-in compatible — see the Breaking Changes section below for
migration steps.

Breaking changes

• ProviderInterface::provide() signature changed from string to
  iterable<string>. Custom providers contributed by third-party modules
  must now yield chunks rather than return one concatenated string. This is
  the change that lets the generator stream to disk with bounded memory.
• /llms-full.txt now serves a genuinely-different file (full sanitized
  descriptions inline). Previously, this URL silently aliased to /llms.txt,
  which was misleading.
• llms.txt header is now spec-compliant. A single blockquote summary line,
  with currency / locale / base-URL moved to a plain markdown paragraph below.
  The 2.x output used four blockquote lines, which broke llmstxt.org-spec
  parsers.
• Status tracking moved out of core_config_data and into
  var/angeo_llms/status.json. Old status rows under angeo_llms/status/*
  are no longer read. Drop them via bin/magento config:set --lock-env angeo_llms/status/... "" if you want a clean state, but it's harmless to leave them.
• media/llms/ is no longer used as the file output directory; output now
  lives under media/angeo/llms/. Old files can be deleted; remove any reverse-proxy rewrites pointing at the old path.
• Admin "Generate" action moved to POST + CSRF. If you have any external
  tooling that hit the old GET URL, switch to the CLI command instead.
• Module namespace unchanged: still Angeo\LlmsTxt. Composer package
  name unchanged.

Added

• Page Builder element filter with four strategies — preserve, exclude,
  allow, strip — driven by the element's data-content-type attribute.
  Default list of excluded types drops common visual-only elements
  (products carousel, banner, slider, video, map, buttons, block,
  dynamic-block, divider, spacer) so the output focuses on semantic text.
  Configurable per-store at Stores → Configuration → Angeo → LLMs.txt →
  Content Sanitization.
• Streaming generation via PHP generators. Memory stays bounded at one
  collection page (default 1000 products) regardless of catalog size.
• Atomic writes: each file is written to .tmp, then renamed. Readers
  never see a half-written file. Generation locks via a separate .lock file
  with flock(LOCK_EX | LOCK_NB), so concurrent runs cannot corrupt output.
• Cursor pagination by entity_id ASC > $lastId instead of skip/limit, so
  products inserted mid-run can neither be duplicated nor skipped.
• Batch URL resolver loads every URL rewrite for a store in one query
  (vs. the per-product getProductUrl() query that 2.x triggered N times).
• Real llms-full.txt with full sanitized descriptions inline.
• /{url_key}.md mirrors — every product, category, and CMS page exposes
  a clean Markdown rendering at its URL with .md appended. Generated on the
  fly; no extra disk storage.
• CMS directive resolution — {{widget}}, {{block}}, {{var}}, and
  {{store}} directives are now rendered via Magento's standard frontend
  filter before being stripped, instead of leaking as literal text.
• Customer-group-aware pricing — admin can choose which customer group's
  final price (with special-price and group-price applied) gets exposed.
• HTTP caching — ETag, Last-Modified, Cache-Control: public, max-age=,
  X-Robots-Tag: noindex, follow, and 304 responses on conditional GETs.
• Async admin action — Schedule (Async) inserts a cron_schedule row for
  the next tick so admins don't have to wait through a synchronous generation.
• Live admin status panel polling /angeo_llms/status/index every 60s.
• Three CLI commands:
  • bin/magento angeo:llms:generate [--store=…] [--no-jsonl] [--no-llms] [--no-full]
  • bin/magento angeo:llms:status
  • bin/magento angeo:llms:validate [--store=…]
• JSONL JSON-Schema at etc/jsonl-schema.json for downstream pipelines.
• Events: angeo_llms_generation_before, angeo_llms_generation_after,
  angeo_llms_generation_failed — for custom hooks.
• PHPUnit test suite under Test/Unit/.

Changed

• frontend_default_meta_description is now the fallback for the store
  summary, before falling back to the generic stub.
• Multi-store store-code routing handles the last URL path segment, so
  /de/llms.txt works on path-based stores.
• Spec compliance: products go under ## Optional by default (admin
  toggleable) so context-budget-constrained clients can drop them.
• Out-of-stock products excluded by an explicit StockRegistry lookup
  (configurable).
• Logger context is now structured: every log line is prefixed
  [Angeo LlmsTxt] and includes store/format keys.

Fixed

• Pseudo-locking in 2.x: a 'w' open truncates the file before the
  flock() call, so two concurrent generations both saw an empty file and
  the last writer won unpredictably. 3.0 uses a separate .lock file.
• CSRF-exposed admin generate: 2.x used a GET URL; 3.0 requires POST with
  the form key.
• Synchronous admin "Generate" timing out on large catalogs (now async option).
• N+1 URL rewrite queries: now batched.
• Literal {{widget}} text appearing in 2.x output: now resolved.
• Stale files for stores that became inactive or excluded: now cleaned up
  on every generation run.

Removed

• media/llms/ legacy directory (see breaking-changes notes).
• GET endpoint for admin generation.
• Documented-but-non-existent config fields from 2.x README.

[2.1.4] — Pre-rebuild baseline

Last release in the 2.x line. See the architectural review document for
the issues that motivated 3.0.0.