SEO and discoverability

Every published page tells search engines and AI assistants what it is, who wrote it and where its sources are, and the site publishes the files crawlers look for. Authors get a Checks tab that scores each draft for being found and quoted.

What it is

The publish pipeline writes these with every published page. Nothing has to be switched on. The home page gets the head tags and the site's structured data too, without overwriting the ones it sets itself (see Build your home page).

The seo feature adds the site-wide files and the settings for them:

After each publish, IndexNow tells Bing (which feeds ChatGPT search and Copilot), Yandex, Seznam and Naver which pages changed. Google does not take part; the sitemap covers it. For a website walkthrough, see Get found by search engines and AI.

Turn it on or off

The site-wide files are the switchable feature seo. It is on by default and needs no other feature. To turn it off, list it in src/custom/features.ts:

export const CUSTOM_FEATURES: FeatureChoice = {
    off: ['seo'],
};

When it is off, Admin, Settings, Discoverability goes, the regenerateSeoFiles function goes at the next deploy of the whole functions codebase, and neither publishing nor npm run seed writes the sitemap, robots.txt, llms.txt, llms-full.txt, feeds or IndexNow key, and nothing is pinged. Files an earlier publish or seed wrote stay on Hosting until the next website deploy: with seo off it leaves them out of the release, which removes them. So after turning it off, deploy the website once (npm run deploy -- --only hosting --project <alias>), then check that /robots.txt and /sitemap.xml no longer show the old files. Head tags, structured data, Markdown twins, authors, sources and the Checks tab are part of Content and stay. See Feature ids.

Set it up

1. Say who owns the site. In Admin, Settings, About, fill in the site name, the production URL, whether the site is an organization or a person, the logo, a description, the Social and profile links (one URL per line), a contact email and a public phone. They become the Organization or Person node on every page (the phone as telephone, the links as sameAs), and the same details can be printed on the site itself (see Site info), and the production URL is the base of every absolute link in the files above. Empty fields are left out, never guessed.

2. Add authors. Content, Authors. Mark one as the default and new content gets that author. See Authors and tags.

3. Map content types (optional). Add or edit a content type and choose its Structured data type: Article (the default), Blog post, News article, Web page, Product, Service, Event, Person, Recipe or How-to guide. Web page suits pages that are not articles, such as About or a policy; the standard pages use it. Map the type's properties to your custom fields; anything unmapped is left out.

4. Choose the crawler policy. Admin, Settings, Discoverability. Every crawler is allowed until you switch it off. Choose whether to publish /llms.txt and /llms-full.txt and whether to use IndexNow (both on by default). Press Save and apply to site to write the files now, or Save to let the next publish write them.

5. Verify from the outside. Arc CMS keeps no crawler log, so register the site in Google Search Console and Bing Webmaster Tools (which also shows IndexNow submissions).

The crawlers on the settings page come in two groups:

Use it

When the files change

Each publish or unpublish of a content type with public pages rewrites the sitemap, the feeds, robots.txt, llms.txt and (with IndexNow on) the IndexNow key file, and releases them to Hosting together with the pages. IndexNow is pinged only after that release succeeds. Save and apply to site rewrites robots.txt, llms.txt and the key file at once, without a publish.

In the editor

A content type with an FAQ custom field (see Field types reference) adds a FAQPage node from the rows of its first FAQ field that have both a question and an answer. Since 2023 Google shows FAQ results only for some government and health sites, so do not expect them in search; the node is still valid, and AI assistants read it.

What the Checks tab scores

High checks weigh 3, medium 2, low 1. The score is the weight passed out of the weight checked. It reads the default language.

Write for Google and AI assistants

Assistants quote passages, not pages. A passage gets quoted when it answers a question on its own, sits under a heading that matches how people ask, holds something checkable, and comes from a page with a named author, a date and sources. Google rewards the same things. Share this with the people who write:

1. Answer first. The first paragraph under the title, and under each heading, answers in 15 to 70 words. Detail comes after.
2. Write headings as questions. "How much does it cost in India?" beats "Pricing".
3. Use facts, not adjectives. "Supports 12 languages including Hindi and Tamil" can be quoted; "world-class support" cannot.
4. Give it structure. A list for steps, a table for comparisons, an FAQ block for the questions you keep getting.
5. Say who wrote it and when. Pick an author. Press Mark as updated today only after a real revision, never for a typo fix.
6. Cite sources. Add the pages you drew on under Sources.
7. Link to your own pages. Three or more; use the suggestions in the Checks tab.

Do not stuff keywords, rewrite what is already everywhere on the web, mark a page updated without changing it, or write an FAQ of questions nobody asks. Aim for a score above 80, then publish. A day later, test the page with Google's Rich Results Test, and ask an assistant your heading's question to see whether it quotes you. If it does not, sharpen the opening paragraph.

AI assistant visits

When Google Analytics is connected, the dashboard counts visits referred by AI assistants such as ChatGPT and Perplexity. See Analytics.

Extend it

An app extends discoverability through its own content and templates:

• Structured data for your content: map a content type to a schema.org type and its custom fields in the type editor (above). No code.
• Templates: a custom detail template can show the author box, the "Last updated" date, the Sources list and Related items with the same bindings as public/_site/templates/default/detail.html. See Detail page bindings.

New schema.org types, content blocks, crawlers, AI referrers and checks are changes to Arc CMS itself, not to an app (see Keep core generic). Types, blocks and crawlers live in a functions file with a browser copy; change both, since a static page and its in-app twin must agree (a test compares the two lists for types and crawlers):

What it stores

Cloud Functions: the regenerateSeoFiles callable (the seo feature). The rest runs inside the publish queue. See Cloud Functions.

Who can do what

Troubleshooting