# angeo/module-llms-txt

> Magento 2 module for AI Engine Optimization (AEO). Generates spec-compliant llms.txt and llms-full.txt per llmstxt.org standard, plus streaming JSONL for vector indexing. Multi-store, multi-website, CLI, cron, async admin UI, Page Builder-aware sanitization, customer-group pricing, atomic writes, ETag/Cache-Control, .md mirrors.

`composer require angeo/module-llms-txt`

Canonical URL: https://packagento.com/angeo/module-llms-txt

## At a glance

- **Vendor**: angeo (https://packagento.com/angeo.md)
- **Latest version**: 3.2.0 — released 2026-06-14
- **Pricing**: Free
- **Package type**: Magento 2 module
- **Status**: active, accepting new buyers

## Installation

Packagento is licence-gated, so even free packages need a licence on a project before Composer can resolve them.

1. **Sign in or create an account** at https://packagento.com/customer/account/.

2. **Add the package to your account.** Open https://packagento.com/angeo/module-llms-txt and complete the free checkout. A licence is minted automatically.

3. **Create or pick a project, then activate the licence on it.**
   - Projects represent the Magento installs you deploy to. Manage them at https://packagento.com/projects/.
   - Activate the new licence on the project you'll deploy this package to. Activation is what generates the Composer credentials scoped to that project.

4. **Add the project credentials to your Magento codebase.**

   Grab the project's public + private key from https://packagento.com/projects/ (open the project, then its Credentials tab), and add them to `auth.json`:

   ```json
   {
     "http-basic": {
       "packagento.com": {
         "username": "ppk_live_...",
         "password": "psk_live_..."
       }
     }
   }
   ```

   Add the Packagento Composer repository to `composer.json`:

   ```json
   {
     "repositories": [
       { "type": "composer", "url": "https://packagento.com" }
     ]
   }
   ```

5. **Install and apply.**

   ```bash
   composer require angeo/module-llms-txt:*
   bin/magento setup:upgrade
   bin/magento setup:di:compile
   bin/magento cache:flush
   ```

## What it does

Magento 2 module for AI Engine Optimization (AEO). Generates spec-compliant llms.txt and llms-full.txt per llmstxt.org standard, plus streaming JSONL for vector indexing. Multi-store, multi-website, CLI, cron, async admin UI, Page Builder-aware sanitization, customer-group pricing, atomic writes, ETag/Cache-Control, .md mirrors.

## README

**AI Engine Optimization (AEO) for Magento 2 / Adobe Commerce.** Generates
spec-compliant `llms.txt`, `llms-full.txt`, and JSONL files so ChatGPT,
Claude, Gemini, Perplexity, and other LLM-powered crawlers can ingest your
catalog efficiently.

[![Magento](https://img.shields.io/badge/Magento-2.4.7%2B-orange)]()
[![PHP](https://img.shields.io/badge/PHP-8.2%20%7C%208.3%20%7C%208.4-blue)]()
[![License](https://img.shields.io/badge/license-MIT-green)]()

> **Current version: 3.2.0** — performance release. Opt-in **single-pass
> generation pipeline**: one catalog pass per store renders all enabled
> formats (default stays `legacy`; legacy pipeline and the old
> `ProviderInterface` SPI are deprecated and will be removed in 4.0.0).
> Builds on 3.1.1 (SQL-level stock filtering, price-index pricing, dedicated
> cron group) and 3.1.0 (security & hardening). See
> [CHANGELOG.md](CHANGELOG.md) for details and upgrade notes.

---

### What this module does

After install, your storefront serves:

| URL                          | What it is                                         |
| :--------------------------- | :------------------------------------------------- |
| `https://shop/llms.txt`      | Spec-compliant llmstxt.org file (compact markdown) |
| `https://shop/llms-full.txt` | Same structure, full sanitized descriptions inline |
| `https://shop/llms.jsonl`    | One JSON record per line, for vector indexing      |
| `https://shop/{url-key}.md`  | On-the-fly Markdown mirror of any product/category/CMS page |

Generation happens via cron (daily by default), CLI, or the admin "Generate
Now" button. The output is streamed to disk with bounded memory, atomically
renamed on completion, and served with proper `ETag` / `Cache-Control` headers.

---

### Why this module exists

LLM crawlers can ingest a typical Magento storefront — full theme, JS, image
sprites, navigation chrome — but that's wasteful for everyone. The
[llmstxt.org](https://llmstxt.org) standard defines a clean text format
optimized for AI ingestion: stable links, structured headings, descriptions
in their natural prose form rather than buried in product cards.

This module produces that format for Magento, with care taken for the things
Magento makes hard: multi-store layout, Page Builder content, CMS directive
resolution, customer-group pricing, and very large catalogs.

---

### Installation

```bash
composer require angeo/module-llms-txt:^3.0
bin/magento module:enable Angeo_LlmsTxt
bin/magento setup:upgrade
bin/magento setup:di:compile      # only in production mode
bin/magento setup:static-content:deploy adminhtml   # only in production mode
bin/magento cache:flush
```

Then generate your first batch:

```bash
bin/magento angeo:llms:generate
```

Visit `https://your-store.tld/llms.txt`.

---

### Configuration reference

All settings live at **Stores → Configuration → Angeo → LLMs.txt**.

#### General

| Field             | Default | Notes                                                             |
| :---------------- | :------ | :---------------------------------------------------------------- |
| **Enable**        | Yes     | Master switch.                                                    |
| **Exclude This Scope** | No | Available at website + store scope. Skips generation for this scope. |
| **Store Summary** | —       | One-line summary used as the spec-compliant blockquote. If empty, falls back to *Design → HTML Head → Default Description*. |

#### Content

| Field                                | Default | Notes                                                  |
| :----------------------------------- | :------ | :----------------------------------------------------- |
| **Include Categories**               | Yes     |                                                        |
| **Include CMS Pages**                | Yes     |                                                        |
| **Include Products**                 | Yes     |                                                        |
| **Products under `## Optional`**     | Yes     | Recommended. Lets context-budget-constrained AI clients drop products without losing categories / pages. |
| **Product Limit**                    | 5000    | 0 = unlimited.                                         |
| **Exclude Out-of-Stock Products**    | No      |                                                        |
| **CMS Identifiers to Exclude**       | `no-route, enable-cookies, privacy-policy-cookie-restriction-mode` | Comma- or newline-separated. |
| **Customer Group for Pricing**       | NOT LOGGED IN | Which group's final price (with special / group prices) is exposed. |

#### Output formats

| Field                          | Default | Notes                                                 |
| :----------------------------- | :------ | :---------------------------------------------------- |
| **Generate llms.txt**          | Yes     |                                                       |
| **Generate llms-full.txt**     | No      | 5–50× larger; enable only if you actually want it.    |
| **Generate JSONL**             | Yes     | One record per line; embeds-ready.                    |
| **Serve `/url-key.md` Mirrors**| No      | Per-entity Markdown rendering; on-the-fly, no disk.   |

#### Content sanitization

_(README truncated for .md surface. Full README on https://packagento.com/angeo/module-llms-txt.)_

## Changelog

All notable changes to **Angeo_LlmsTxt** are documented in this file.

The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

---

### [3.2.0] — 2026-06-10

Single-pass generation pipeline (opt-in). **Fully backward compatible**: the
default mode remains `legacy`, all pre-3.2 behavior, file paths, events, and
extension points keep working unchanged. Everything superseded is marked
`@deprecated` and will be removed in **4.0.0**.

#### Added

* **Single-pass pipeline** (`Model/Pipeline/SinglePassGenerator`). With
  `Stores → Configuration → Angeo LLMs.txt → Performance → Generation
  Pipeline = Single pass`, each store's catalog is iterated **once** and every
  enabled format (llms.txt, llms-full.txt, llms.jsonl) is rendered from that
  one pass:
  - one frontend emulation per store (legacy: one per format),
  - one url_rewrite warm-up per store (legacy: one per format),
  - each entity loaded and **sanitized exactly once** (legacy: 2–3× per
    product description),
  - all format files written in parallel streams with atomic rename, under one
    per-store lock (`media/angeo/llms/store_{code}.lock`).
  Combined with 3.1.1 this gives roughly 3× faster generation on top of the
  3.1.1 gains, with identical output files.
* **New `@api` extension points** (implement these going forward):
  - `Api\EntityProviderInterface` — yields format-agnostic entity records once
    per entity (successor of the format-specific `ProviderInterface`);
  - `Api\Data\EntityRecordInterface` + `Model\Data\EntityRecord` — immutable
    record DTO carrying already-sanitized content;
  - `Api\FormatRendererInterface` — serializes records into one output format;
  - `Model\Output\FilePathResolver` — the single source of truth for generated
    file paths (used by both pipelines and the frontend controller);
  - `Model\Text\Truncator` — shared word-boundary truncation (the Sanitizer
    now delegates to it; behavior is byte-identical).
* Bundled single-pass providers/renderers registered via `di.xml`
  (`SinglePassGenerator` → `entityProviders`, `renderers`). Third parties add
  their own items the same way.
* `Model/Config/Source/GenerationMode` + new system.xml field
  `angeo_llms/performance/generation_mode` (global scope, default `legacy`).
* Unit tests: `TruncatorTest`, including the down-truncation invariant that
  guarantees single-pass renderers reproduce legacy truncation byte-for-byte.

#### Backward compatibility

* `generation_mode` defaults to **legacy** — upgrading changes nothing until
  you opt in.
* In single-pass mode the output files, on-disk paths, served URLs, generation
  status records, and the `angeo_llms_generation_before/after/failed` events
  (dispatched per format) are identical to legacy.
* **Custom providers built on the legacy `ProviderInterface` keep working in
  both modes.** In single-pass mode they are detected automatically (anything
  registered on the legacy generators beyond the bundled providers) and
  executed through a compatibility pass that appends their output to the
  corresponding format stream.
* The only semantic difference: the `items` counter in generation status now
  counts rendered records rather than raw stream chunks.

#### Deprecated (removal in 4.0.0)

* `Api\ProviderInterface` and `Model\Provider\AbstractProvider` — implement
  `Api\EntityProviderInterface` instead.
* All eight bundled legacy providers under `Model\Provider\Llms\*` and
  `Model\Provider\Jsonl\*` — superseded by `Model\Pipeline\Provider\*` +
  format renderers.
* `Model\Generator\AbstractGenerator`, `LlmsTxtGenerator`,
  `LlmsFullTxtGenerator`, `JsonlGenerator` — superseded by
  `SinglePassGenerator`; file-path resolution moved to `FilePathResolver`.
* The `legacy` generation mode itself: 4.0.0 ships single-pass as the only
  pipeline and removes everything listed above.

_(Changelog truncated for .md surface. Full history on https://packagento.com/angeo/module-llms-txt.)_

## Recent Versions

| Version | Released |
|---|---|
| 3.2.0 | 2026-06-14 |
| 3.0.5 | 2026-06-04 |
| 3.0.4 | 2026-06-03 |
| 3.0.3 | 2026-06-03 |
| 3.0.2 | 2026-06-03 |
| 3.0.1 | 2026-06-03 |
| 3.0.0 | 2026-05-29 |
| 2.1.4 | 2026-05-06 |
| 2.1.3 | 2026-04-30 |
| 2.1.2 | 2026-04-30 |

Showing 10 of 15 versions. Full release history on https://packagento.com/angeo/module-llms-txt.

## Dependencies

### Require

| Package | Constraint |
|---|---|
| ext-json | * |
| ext-mbstring | * |
| magento/framework | >=102.0 |
| magento/module-backend | >=102.0 |
| magento/module-catalog | >=104.0 |
| magento/module-catalog-inventory | >=100.4 |
| magento/module-catalog-url-rewrite | >=100.4 |
| magento/module-cms | >=104.0 |
| magento/module-config | >=101.2 |
| magento/module-store | >=101.0 |
| magento/module-url-rewrite | >=102.0 |
| php | ~8.1.0\|\|~8.2.0\|\|~8.3.0\|\|~8.4.0\|\|~8.5.0 |

### Require (dev)

| Package | Constraint |
|---|---|
| magento/magento-coding-standard | ^32.0 |
| phpstan/phpstan | ^1.10 |
| phpunit/phpunit | ^10.5 |

### Suggest

| Package | Constraint |
|---|---|
| magento/module-page-builder | Enable to opt-in or opt-out of Page Builder content elements per content-type during sanitization |
| magento/module-shared-catalog | Adobe Commerce: integrate B2B shared catalogs so llms.txt only exposes the allowed catalog |

## Quality

Latest release (3.2.0) fails the Packagento QA pipeline. Verdicts below are per-cell (Magento line × PHP version) for the matrixed tools, and run-once for the static / security tiers.


### Compatibility

Each Magento line is installed on its supported PHP versions, then the module is built (DI compile + static-content deploy). Cells show passed / failed / untested; staircase gaps render as `–`.

| Magento | PHP 8.2 | PHP 8.3 | PHP 8.4 | PHP 8.5 |
|---|---|---|---|---|
| 2.4.7 | Pass | Pass | – | – |
| 2.4.8 | – | Pass | Pass | – |
| 2.4.9 | – | – | Pass | Pass |


### Code Quality

Advisory checks against the module's source. Never affect the Compatibility verdict — a phpcs finding can't make a module incompatible.

#### Static Analysis

Coding standards (phpcs), mess detection (phpmd), copy-pasted code (cpd), PHP cross-version compatibility, composer.json validity. Each runs once for the whole module.

| Tool | Status | Findings | Summary |
|---|---|---|---|
| PHPCS | Fail | 125 | 1 error, 124 warnings (ruleset: Magento2) — 48 auto-fixable with phpcbf |
| PHPMD | Warning | 33 | 33 rule violations (CyclomaticComplexity:8, MissingImport:8, NPathComplexity:6, UnusedFormalParameter:4, ExcessiveMethodLength:3) |
| Cpd | Pass | 0 |  |
| Composer validate | Info | 10 | valid; 10 advisory notes (composer validate --strict) |

#### PHPStan

Type-checks the module against a real Magento install. Re-runs per Magento + PHP version because resolvable symbols differ between releases.

| Magento | PHP 8.2 | PHP 8.3 | PHP 8.4 | PHP 8.5 |
|---|---|---|---|---|
| 2.4.7 | 30 | 30 | – | – |
| 2.4.8 | – | 30 | 30 | – |
| 2.4.9 | – | – | 30 | 30 |


### Tests

Unit and integration suites run per Magento + PHP cell. Test failures speak to the module's behaviour, not its compatibility with a line, so they're reported here separately.

#### Unit Tests

| Magento | PHP 8.2 | PHP 8.3 | PHP 8.4 | PHP 8.5 |
|---|---|---|---|---|
| 2.4.7 | 7 | 7 | – | – |
| 2.4.8 | – | 7 | Error | – |
| 2.4.9 | – | – | 7 | not tested |

#### Integration Tests

| Magento | PHP 8.2 | PHP 8.3 | PHP 8.4 | PHP 8.5 |
|---|---|---|---|---|
| 2.4.7 | N/A | N/A | – | – |
| 2.4.8 | – | N/A | N/A | – |
| 2.4.9 | – | – | N/A | N/A |


### Security

Dependency-advisory audit (composer audit) plus a source malware scan. A malware detection fails the version outright.

| Tool | Status | Findings | Summary |
|---|---|---|---|
| Composer audit | Pass | 0 |  |
| Malware scan | Pass | 0 |  |

## Licence and pricing

Free. A licence is still minted on checkout and bound to your project for Composer access — no payment step.

Refundable within 14 days of first purchase via https://packagento.com/account/refunds/.

## Install via Claude Code or any MCP client

The Packagento MCP server can run the licence + project + Composer steps above in one tool call:

```
purchase_and_install_packages(
  composer_names=["angeo/module-llms-txt"],
  project_id="proj_xxx"
)
```

This handles cart, checkout, licence minting, project activation, and writes auth.json credentials. Connect a client with `claude mcp add packagento https://mcp.packagento.com`. Full setup at https://packagento.com/docs/mcp-setup.

## Vendor

angeo is a Magento 2 vendor on Packagento. See https://packagento.com/angeo.md for their full catalogue.