Skip to content

point crawlers to llms.txt in robots.txt - #268

Merged
SyedFahad7 merged 3 commits into
mainfrom
docs/robots-llms-pointer
Nov 17, 2026
Merged

SyedFahad7 merged 3 commits into
mainfrom
docs/robots-llms-pointer

Conversation

@SyedFahad7

Copy link
Copy Markdown
Collaborator

A robots.txt file at the root of the repo replaces Mintlify's generated one, so this copy keeps the current crawler rules, the Content-Signal line and the sitemap. It then adds comments that point to llms.txt and llms-full.txt, so crawlers that read robots.txt can find those indexes.

Copilot AI balanced review requested due to automatic review settings November 17, 2026 00:16
@mintlify

mintlify Bot commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated
graphify-labs-inc 🟢 Ready View Preview Nov 17, 2026, 12:40 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@graphify-labs-staging graphify-labs-staging Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Graphify code review View in Graphify

Skipped · bceb378a2c0a

Skipped the AI code review because it only changes documentation. The static checks still ran; push a code change to get a full review.

Code graph and static checks

Impact & health

Impact — 0 functions depend on the 0 functions this change touches.

Health — grade A; no new coupling hotspots.

Formal checks — 0 functions this change can affect weren't formally checked in this review. Formal checks are informational only.

Health delta baseline: last indexed commit 445ead8 (diverged from this PR's base — delta is approximate).

Gate & verification

✅ PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.

@SyedFahad7
SyedFahad7 merged commit 45d9691 into main Nov 17, 2026
2 checks passed

This branch was successfully deployed

1 active deployment
staging — d43ad7e9 Deployed Nov 17, 2026 by mintlify[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants