Product Deep-Dive

How Our 14 AI Agents Audit Your Website — A Behind-the-Scenes Look

RF
Ross Forrester
··31 min read
14 AI agents organised into three parallel layers — SEO, GEO, and intelligence — auditing a UK small business website

How Our 14 AI Agents Audit Your Website — A Behind-the-Scenes Look

Key Takeaways

  • 14 specialised AI agents run in parallel the moment you submit your URL — simultaneously, not one after another
  • The audit covers three distinct layers: SEO foundations (7 agents), GEO / AI search visibility (5 agents), and competitive intelligence (2 agents)
  • Over 200 individual checks run in 30–45 seconds — a sequential audit of the same depth would take hours
  • Your CMS is identified automatically so platform-specific rules apply to every recommendation
  • The Digital Visibility Score (DVS) combines 60% SEO and 40% GEO into a single trackable number
  • Every report is reviewed by a human before delivery — AI agents find the patterns, humans apply the context
  • A free re-audit at 90 days shows exactly how much your score has moved

You type your URL into the form. You hit submit.

What happens next takes 30–45 seconds. During that window, 14 AI agents launch simultaneously — each focused on a different dimension of your website's visibility. Not one after another. All at once, in parallel.

By the time the progress bar finishes, those agents have run over 200 individual checks. They have examined your crawlability, your page speed, your schema markup, your content quality, whether AI crawlers can access your site, whether your brand exists as a trusted entity across the web, how your competitors score on the same checks, and whether your specific CMS is generating invisible technical problems your existing tools have never flagged.

This article explains exactly what each of those agents does — and why we built the system this way.


Why 14 Agents? Why Not One?

The most obvious question is also the right one to start with.

A conventional SEO audit tool runs as a single sequential process: crawl the site, run checks, output a report. The advantage is simplicity. The disadvantage is depth — no single process can be both broad and deep across every relevant category without taking hours.

Our system runs 14 specialised agents in parallel because specialisation produces better results than generalisation. An agent designed to check AI crawler access does not need to know anything about backlink quality. An agent checking Core Web Vitals does not need to parse schema markup. Keeping each agent focused means it can go deeper on its specific domain than any combined process could.

The parallelism is the practical payoff. Tasks that would take hours if run sequentially complete in under a minute when run simultaneously. You get your Digital Visibility Score — and a human-reviewed report — within minutes of submitting your URL.

Citation Capsule ChatGPT now handles approximately 17% of global search queries. Google AI Overviews appear in at least 16% of all searches. Perplexity crossed 100 million monthly active users in early 2026, growing over 200% year-on-year (SparkToro, January–May 2025: AI-referred web sessions grew +527%). A website audit that does not address AI search visibility is auditing for a world that no longer exists.


Layer One: SEO Foundations (Agents 1–7)

The first seven agents cover the fundamentals that determine how traditional search engines — Google, Bing — crawl, index, and rank your site. These are the established signals. Getting them right is the prerequisite for everything else.

Agent 1: Technical SEO

This agent starts at the bottom of the stack: can search engines reach your site, and can they understand what they find?

It checks your robots.txt for accidental blocks — one of the most common and costly issues we find, because a single misconfigured disallow rule can hide an entire section of your site from Google. It validates your XML sitemap: is it present, is it submitted to Google Search Console, does it include the pages that should be indexed and exclude the ones that should not?

Redirect chains get traced and counted. Every additional redirect hop adds latency and dilutes link equity — no chain should exceed three hops. Canonical tags are checked for correctness across every key page. Canonical errors are common across all CMS platforms and often invisible until they cause ranking problems: canonicals pointing to redirected URLs, canonicals pointing to non-indexable pages, and missing self-referencing canonicals on paginated content are all surfaced here.

Agent 2: Core Web Vitals

Google confirmed that Core Web Vitals — the three user-experience signals Largest Contentful Paint (LCP), Interaction to Next Paint (INP), and Cumulative Layout Shift (CLS) — are direct ranking factors. This agent checks all three.

Critically, it does not rely exclusively on Lighthouse scores. Of pages that score 90 or above in Lighthouse, 43% fail to meet one or more Core Web Vitals thresholds in real-world field data from the Chrome User Experience Report (CrUX), according to analysis by Calibre App. Lighthouse is a lab test run on a simulated 2016 phone on Slow 4G. CrUX is your actual visitors on their actual devices. Where CrUX data is available for your domain, the agent uses that. Where it is not, Lighthouse provides the baseline — flagged with appropriate caveats.

One point worth noting: INP replaced FID (First Input Delay) as a Core Web Vitals metric in March 2024. Many audit tools still report FID. This one does not.

Agent 3: On-Page SEO

Title tags, meta descriptions, heading hierarchy, image alt text, keyword placement, internal linking. This agent runs 34 checks across every on-page signal that affects individual page performance.

Uniqueness is a primary concern: duplicate title tags across two or more pages tell Google you do not have a clear opinion about which page is most relevant for a given query. Multiple H1 tags on a single page suggest poor content architecture. This agent surfaces every instance.

Internal linking patterns are assessed here too. Orphan pages — pages with no internal links pointing to them — are a common crawl-budget drain. Pages buried four or five clicks deep from the homepage rarely rank well regardless of content quality, because Google's crawlers deprioritise deep content when crawl budget is constrained.

Agent 4: Schema Markup

Structured data is the bridge between your content and how search engines — and increasingly, AI systems — understand it. This agent checks for the presence, completeness, and validity of JSON-LD markup across every relevant schema type.

For a local business: LocalBusiness, FAQPage, Review, and BreadcrumbList. For an e-commerce store: Product, Offer, AggregateRating, and BreadcrumbList. For a blog or publisher: Article (or BlogPosting), Author, and Organisation. For most sites: at minimum, Organisation and WebSite schema on the homepage.

Common errors flagged: schema that references images not present on the page, required fields missing from Organisation markup (name and url are both required and frequently absent), and schema generated by CMS plugins that conflicts with schema added manually — a particularly common problem on WordPress sites running both Yoast SEO and a page builder with its own schema output.

Domain authority is a proxy for trust. This agent assesses the quality, relevance, and toxicity of your backlink profile — not just total link count, but the calibre of the sites linking to you.

A hundred low-quality directory links from irrelevant sites can actively harm your rankings if they form a pattern Google associates with manipulation. This agent distinguishes between links that build trust and links that undermine it. Toxic links are flagged separately. If links from penalised or manipulative domains are identified, the report flags them with a recommendation on whether a disavow file is appropriate.

Agent 6: Sitemap and Robots Architecture

Separate from the broader technical SEO agent, this one focuses specifically on the interaction between your sitemap and robots.txt — an area where conflicting signals are surprisingly common and consistently damaging.

A page listed in a sitemap that is also blocked by robots.txt sends contradictory instructions. A page with a noindex tag listed in the sitemap is equally confused. Both patterns waste crawl budget and can suppress rankings on the affected pages. This agent flags every conflicting signal.

Agent 7: Security Headers

Security is both a trust signal and a technical requirement. HTTPS is the baseline: sites still running on HTTP receive a demotion in Chrome and are treated with lower trust by Google's systems. But HTTPS alone is not enough.

This agent checks for HSTS (HTTP Strict Transport Security), which prevents protocol downgrade attacks. It checks for a Content Security Policy header, which signals to Google that you are actively managing your site's security posture. And it checks X-Frame-Options and other signals that contribute to Google's site quality assessment.


Layer Two: GEO — AI Search Visibility (Agents 8–12)

This is the layer that most audit tools do not have. The five GEO agents check your visibility to AI search engines — the platforms now responsible for a growing share of the searches your customers are running.

If you want the full context on why this matters for UK businesses, our article SEO vs GEO: What UK Small Businesses Need to Know in 2026 covers it in detail.

14 Agents — Three Layers of Analysis SEO LAYER — 7 Agents TECHNICAL Crawl, index, robots CORE WEB VITALS LCP, INP, CLS ON-PAGE SEO Titles, H-tags, meta SCHEMA JSON-LD, FAQPage BACKLINKS Authority, toxicity SITEMAP Robots, indexation SECURITY HTTPS, HSTS, CSP GEO LAYER — 5 Agents AI CRAWLER ACCESS GPTBot, ClaudeBot AI CITABILITY Passage structure LLMS.TXT AI instruction file BRAND ENTITY Cross-web signals AI PLATFORMS ChatGPT, Perplexity, AIO INTELLIGENCE LAYER — 2 Agents COMPETITOR BENCHMARKING Gap analysis vs your named competitors PLATFORM-SPECIFIC ANALYSIS WordPress · Shopify · Wix · Squarespace · Webflow DIGITAL VISIBILITY SCORE 60% SEO · 40% GEO · One number · Tracked at 90 days

Agent 8: AI Crawler Access

Before any other GEO check, this agent asks the most fundamental question: can AI search engines actually reach your content?

The main AI crawlers are GPTBot (OpenAI), ClaudeBot (Anthropic), PerplexityBot (Perplexity AI), and Google-Extended (the crawler for Google AI Overviews and Gemini). Industry research shows that 63% of businesses are accidentally blocking at least one of these crawlers — usually through robots.txt rules that were written when only Googlebot and Bingbot existed.

The fix is often a single line in robots.txt. But you cannot fix it if you do not know it is happening.

This agent also checks for Cloudflare bot protection configurations and WAF (Web Application Firewall) rules that might block AI crawlers at the network level — invisible in the robots.txt but just as effective at keeping AI engines out. [Read our full article on this topic: Is Your Website Visible to ChatGPT and AI Search Engines?]

Agent 9: AI Citability

Being accessible to AI crawlers is necessary but not sufficient. AI systems like ChatGPT and Perplexity do not just index your pages — they extract specific passages from them to use as citations in answers to user questions.

This agent assesses whether your content is structured in a way that supports passage-level extraction. The key signals it looks for:

Answer-first structure. The direct answer to a question appears in the opening sentence or paragraph, not buried after extensive background. AI systems favour content that mirrors how they deliver answers — directly and concisely.

Passage independence. Individual paragraphs that can be understood and cited without context from the surrounding article. A paragraph that begins "As mentioned above..." cannot be cited in isolation.

Factual density. Statistics, specifics, dates, and verifiable claims make content more citable. Vague assertions ("many businesses struggle with...") are far less likely to be selected as citation material than precise ones ("63% of businesses are accidentally blocking at least one AI crawler").

Question alignment. Content that explicitly answers the questions people search for. FAQ sections, subheadings framed as questions, and direct definitions of technical terms all raise citability scores.

Agent 10: llms.txt

llms.txt is the AI equivalent of robots.txt — a plain-text instruction file that tells AI systems what your website is about, which pages contain authoritative information, and how you want your business described when summarised.

Placed at yourdomain.com/llms.txt, the file uses a simple markdown format to list your key pages with descriptions, specify which content is most authoritative, and indicate how your business should be categorised. AI systems that respect the standard use it to build a more accurate representation of your site before crawling.

This agent checks whether you have the file, whether it is correctly formatted (common errors include a missing H1 header and broken URL formatting), and whether it accurately represents your current site structure and key services.

For Comprehensive tier audits, we generate a custom llms.txt file as part of the deliverable.

Agent 11: Brand Entity Signals

AI search engines do not decide what to cite based solely on your website. They cross-reference your brand against external signals: is this business consistently described the same way across the web? Does it have a verifiable presence on platforms that AI systems trust?

This agent assesses your brand entity strength across four dimensions:

  • NAP consistency: Is your business Name, Address, and Phone Number consistent across Google Business Profile, Companies House, your website, and major directories? Inconsistency is a strong negative signal for both AI systems and traditional local SEO.
  • Platform presence: LinkedIn company page, Wikipedia or Wikidata entry for established brands, and mentions in authoritative UK publications.
  • Social proof footprint: Reviews on Google, Trustpilot, and industry-specific platforms. AI systems look at recency and the credibility of review sources.
  • Cross-web mention patterns: Is your brand mentioned in consistent contexts by independent sources? Consistent categorisation reinforces entity signals across AI systems.

Citation Capsule Ahrefs research (December 2025) found that brand mentions on authoritative external sources show a 3x stronger correlation with AI citation frequency than backlinks alone. Building brand entity is now as important as building links.

Agent 12: AI Platform-Specific Signals

Google AI Overviews, ChatGPT web search, Perplexity, and Bing Copilot each pull information differently. An optimisation that works for one may not apply to another.

This agent applies platform-specific knowledge to the findings from agents 8–11. For Google AI Overviews: structured data and E-E-A-T signals are primary. For Perplexity: answer-first content structure and citation-ready statistics matter most. For ChatGPT web search: brand mentions across credible, recently-updated sources carry the strongest weight.


Layer Three: Competitive Intelligence (Agents 13–14)

The final two agents apply intelligence that the previous twelve generate. They answer two questions: how do you compare, and why does your specific platform matter?

Agent 13: Competitor Benchmarking

Your Digital Visibility Score only tells you so much in isolation. The competitor benchmarking agent runs the same checks against the competitors you name in the audit form and produces a gap analysis.

The output is specific, not generic. Which keywords they appear for that you do not. What GEO signals they hold that you lack. Where they outperform you technically and by how much. And critically — where the gap is closeable with realistic effort in the next 90 days.

Not every gap is worth closing. A competitor with a ten-year domain history and thousands of backlinks from authoritative publications has advantages that cannot be reversed in 90 days. This agent focuses the gap analysis on areas where meaningful improvement is achievable — not on gaps that are structural and long-term.

Agent 14: Platform-Specific Analysis

This is the agent most audit tools skip entirely.

Once your CMS is identified — from the audit form and from technical signals in your site's HTML — this agent applies a knowledge base of platform-specific issues and constraints to every finding.

Platform-Specific Audit Flow Your Website URL submitted via audit form CMS Detection HTML signals identify platform WordPress Plugin conflicts Duplicate schema Crawl budget PageSpeed (builders) Shopify 4x duplicate URLs Canonical handling Collection structure Product schema Wix No robots.txt edit JS rendering limits Achievable fixes only Constraint flags Squarespace URL folder depth JS-rendered pages Plan-gated fixes Settings map Webflow Strong defaults CMS linking gaps Collection templates Rich text schema Each platform receives recommendations filtered to what is technically achievable on that CMS

WordPress (approximately 43% of all websites): Plugin conflicts causing duplicate schema are common — sites running both Yoast SEO and a page builder like Elementor frequently output conflicting schema. PageSpeed issues often trace to unoptimised images served through page builders. Crawl budget drain from tag and category archives indexing at scale affects many content-heavy WordPress sites.

Shopify: Generates four canonical URL variants per product by default — /products/slug, /collections/[collection]/products/slug, plus two additional variants in some themes. Without careful canonical handling, this creates duplicate content signals across the entire product catalogue. This agent identifies which products are affected and what specific canonical configuration to apply.

Wix: Robots.txt cannot be edited directly on Wix. Any generic recommendation to "update your robots.txt" is technically impossible without a platform migration. This agent only recommends what is actually achievable on Wix — and flags clearly where a specific improvement would require moving to a different platform.

Squarespace: URL folder structures are often suboptimal for SEO, and JavaScript-rendered content affects how Google indexes certain page types. The agent identifies which Squarespace settings can be changed within the current plan and which require a plan upgrade.

Webflow: Strong technical SEO defaults out of the box, but CMS collection pages often lack cross-links to related content. The agent checks collection template linking patterns specifically and flags rich text areas that lack proper semantic structure.

AI website builders (Bolt, Lovable, v0): Sites generated by AI builders frequently have minimal meta tags, absent schema markup, and placeholder content that was never updated. The agent checks for these specific patterns and prioritises foundational fixes.


The Digital Visibility Score: 60% SEO, 40% GEO

Everything those 14 agents find feeds into a single composite number.

How the Digital Visibility Score Is Calculated SEO SCORE 60% Technical SEO · Core Web Vitals · On-Page Schema Markup · Backlinks Sitemap & Robots · Security Headers 7 agents · ~120 checks GEO SCORE 40% AI Crawler Access · AI Citability llms.txt · Brand Entity Signals AI Platform-Specific Signals 5 agents · ~80 checks DIGITAL VISIBILITY SCORE /100 Tracked across your 90-day re-audit

The 40% GEO weighting reflects current AI search market share. Gartner's 2024 research predicted traditional search engine traffic will fall by 50% by 2028 as AI answer engines absorb queries. The DVS weighting will be reviewed as AI search share grows.

Score benchmarks for UK small business sites:

  • Below 40: Significant technical problems in both SEO and GEO. Likely missing basic indexation signals or actively blocking AI crawlers.
  • 40–60: Some SEO work done. Most UK small businesses without active GEO investment land here. Core Web Vitals may be passing, but AI visibility is near zero.
  • 60–75: Active SEO effort and beginning GEO work. Schema present, AI crawlers allowed, but citability structure and brand entity are still weak.
  • 75+: Strong across both channels. Typically sites with consistent SEO investment plus deliberate GEO optimisation.

For a deeper look at how SEO and GEO compare and why both matter, see What Is GEO? The Complete Guide to AI Search Optimisation for UK Businesses.


The 200+ Checks: Category Breakdown

200+ Checks — Category Breakdown Technical SEO — 54 checks GEO / AI Search — 42 checks On-Page SEO — 34 checks Content Quality (E-E-A-T) — 30 checks Platform-Specific — 26 checks Local SEO + Other — 22 checks

Technical SEO (54 checks): Robots.txt analysis, sitemap validity, redirect chain depth, canonical implementation, Core Web Vitals (LCP, INP, CLS from CrUX and Lighthouse), mobile usability, HTTPS and security headers, crawl depth, orphan page detection, internal link architecture.

GEO / AI Search (42 checks): AI crawler access (GPTBot, ClaudeBot, PerplexityBot, Google-Extended), llms.txt presence and formatting, brand entity consistency, passage-level citability structure, schema for AI extractability, platform-specific AI signals (Google AI Overviews, Perplexity, ChatGPT web search).

On-Page SEO (34 checks): Title tag uniqueness and length, meta description coverage, heading hierarchy (H1–H4), image alt text, internal linking patterns, duplicate content detection, keyword placement and density.

Content Quality — E-E-A-T (30 checks): Author credibility signals, first-hand experience indicators, source citations, content depth versus thin content, readability score, FAQ presence, freshness signals, business transparency signals.

Platform-Specific (26 checks): CMS identification, platform-specific known issues, achievable versus non-achievable recommendation filtering, plugin and theme impact assessment (WordPress), canonical handling per product (Shopify), platform constraint flags (Wix, Squarespace).

Local SEO and Other (22 checks): Google Business Profile completeness, NAP consistency across directories, local schema, review signals, backlink quality and toxicity screening, domain authority assessment.


Why We Review Every Report Before Delivery

The 14 agents run automatically. The human review does not.

AI agents are excellent at pattern recognition across structured data. They are less reliable at the contextual judgements that determine whether a finding is actually important for a specific business.

A plumbing company in Manchester and a SaaS startup in London might both have "missing Organisation schema" flagged by the schema agent. The priority is completely different: the plumber needs LocalBusiness schema with service areas, opening hours, and Review schema. The SaaS startup needs SoftwareApplication and FAQPage schema. The generic finding is correct in both cases. The specific response is entirely different — and the wrong response wastes time and effort.

Our review before every delivery:

  1. Catches false positives — the agent flagged something that is intentional or correct for this specific business context
  2. Prioritises findings in the order most likely to move the needle for your type of business, not just by technical severity
  3. Filters non-achievable recommendations — technically accurate advice that cannot be acted on with your current CMS is either removed or moved to a long-term note
  4. Adds specific context — not "fix your LCP" but "your LCP is slow because your hero image is not declared as a priority resource, so it loads after the render-blocking stylesheet in your theme's head section"

This review adds 15–30 minutes to delivery time. We believe it makes the difference between a report you act on and a report that sits in your inbox.


The 90-Day Re-Audit: Proof, Not Sales

Every audit tier includes a free re-audit at 90 days. We run the same 200+ checks on your site again and compare the results side by side against your original report.

The re-audit shows three things:

  1. What has improved — checks that were failing and are now passing, with the score impact quantified
  2. What still needs attention — findings from the original report that have not yet been addressed
  3. Your new Digital Visibility Score — the concrete measure of progress over 90 days

The 90-day window is deliberate. Google's ranking algorithm typically takes 6–12 weeks to fully reflect technical changes. Schema additions can appear in rich results within days. Core Web Vitals improvements take longer to propagate through CrUX field data. Ninety days is the minimum window in which meaningful score improvement becomes measurable across both SEO and GEO signals.

The re-audit is not a sales call. It is the evidence that the recommendations worked.


Key Takeaways

  • 14 specialised AI agents run in parallel — all at once — meaning a 200+ check audit completes in under a minute
  • Three layers: SEO foundations (agents 1–7), GEO and AI search visibility (agents 8–12), competitive intelligence (agents 13–14)
  • The GEO layer is where most audit tools stop short — and where the fastest-growing search traffic is going
  • Platform-specific analysis filters every recommendation to what is achievable on your CMS
  • The Digital Visibility Score (60% SEO, 40% GEO) gives you one number to track across two increasingly distinct search channels
  • Human review before every delivery turns AI pattern recognition into context-appropriate, actionable findings
  • The free 90-day re-audit is proof of progress — not a sales call

Frequently Asked Questions

How long does the audit take from URL submission to report delivery?

The automated audit runs for 30–45 seconds with all 14 agents working simultaneously. Human review adds 15–30 minutes. The full report is delivered by email within a few minutes of form submission for most audits.

Do I need to be technical to understand the report?

No. Every finding includes a plain-English explanation of what the problem is, why it matters for your visibility, and a specific actionable fix. Platform-specific recommendations are filtered to only include what is achievable on your CMS — you will not receive instructions that require developer skills you do not have.

What if I am on Wix or Squarespace and some fixes are not possible?

The platform-specific agent identifies what is achievable on your platform and what is not. Wix users will not receive recommendations to edit robots.txt — because that is technically not possible on Wix. You will only receive recommendations you can act on. Where a specific improvement requires moving to a different platform, the report flags it clearly as a longer-term consideration.

How is the Digital Visibility Score different from other SEO scores?

Most SEO health scores measure only technical SEO and on-page performance — they were built before AI search existed as a meaningful traffic source. The Digital Visibility Score combines SEO (60%) and GEO (40%) into a single number, measuring your website's visibility across both traditional search and AI search engines in one trackable metric.

What happens at the 90-day re-audit?

We run the same 200+ checks on your site again and compare results side by side against your original report. You see exactly which issues have been resolved, how your Digital Visibility Score has changed, and what still needs attention. There is no upsell call — it is evidence of the work you have done.


Ready to see what the 14 agents find on your site? Start your audit — results in under a minute.


how AI website audit works14 AI agents SEO auditbehind the scenes SEO auditDigital Visibility Score explainedAI SEO audit UK

Free Audit

See what our audit finds on your site

14 AI agents, 200+ checks, platform-specific fixes for your exact CMS. Free re-audit at 90 days to measure your progress.

Get Your Free Audit Score

More insights