Telemetry Diagnostic Tool: Technical Architecture Guide
The AEObility Telemetry Diagnostic reviews how clearly a website communicates its services, topics, entities, and commercial relevance. It combines website crawling, semantic comparison, structured-data checks, competitor analysis, and a controlled retrieval simulation to produce a prioritised scorecard. The score helps identify improvement opportunities; it does not predict or guarantee visibility in any external AI platform.
When AI search engines (like Perplexity, ChatGPT, or Google AI Overviews) answer user questions, they extract precise, unambiguous information blocks. If a website's copy is diluted or lacks explicit entity structure, AI models bypass it in favor of clearer competitor sources.
This document provides an open technical specification of AEObility Architecture v2.4 (last tested August 2026). It details our dual vector hashing infrastructure, text-embedding-004 RAG simulation testing, 5-category scoring math, AI Bill ingestion pipeline, and NLWeb/MCP protocols.
The diagnostic does not reproduce or access the proprietary retrieval, ranking, citation, or recommendation systems of ChatGPT, Perplexity, Gemini, Claude, Google, or other external platforms. Its findings are directional diagnostic signals generated from AEObility’s documented test configuration, not predictions or guarantees of visibility.
How raw page copy and search intent pass through parallel semantic, entity, and competitor checks before merging into a single score and handing off context to AI Bill.
A numerical vector representation of text used for semantic comparison in high-dimensional vector spaces.
A mathematical measure of how closely two vector direction angles align invariant to total document length.
A Subject–Predicate–Object relationship statement used to build structured entity knowledge graphs.
An internal stress-test evaluating whether focused content chunks survive retrieval cutoffs for target queries.
Model Context Protocol; a machine interface exposing structured tools and data to compatible AI clients.
The phenomenon where mixing disparate topics into one text block dilutes semantic retrieval focus.
Core Technical Thesis
Modern search and retrieval systems may combine lexical matching, semantic embeddings, passage retrieval, entity signals, source quality, and language models. AEObility tests selected aspects of this broader process through its own diagnostic configuration by evaluating 90–120 token answer blocks validated against Google Gemini's text-embedding-004.
1. Architecture & Vector Map Infrastructure
AEObility separates vector tasks into two layers: a lightweight 384-dimensional character 3-gram hashing vector for local intent classification, and a dense 768-dimensional neural vector for cosine similarity comparison under the selected embedding configuration.
The telemetry engine operates across two complementary vector representation layers: local character N-gram hashing vectors for local intent classification, and dense neural embeddings for high-dimensional cosine similarity analysis under the selected embedding configuration.
Dual-branch routing separates ultra-fast local intent classification (384-dim) from high-dimensional neural similarity (768-dim), reducing API calls and latency.
384-Dim Local Vector Construction
src/lib/search/vectorEngine.tsIn src/lib/search/vectorEngine.ts, text is tokenised into 3-gram character sequences and mapped into a 384-dimensional Float64Array using L2 Euclidean normalisation:
c_i: Character code value at position i in 3-gram sequencek: Sequence length (k = 3)31: Prime hashing seedv_j: Vector magnitude at dimension j768-Dim Dense Proximity Mapping
src/lib/telemetry/proximity.tsIn src/lib/telemetry/proximity.ts, target search intent and crawled site copy are embedded using Google Gemini's text-embedding-004:
a, b: 768-dim dense vectors from text-embedding-004||a||_2: L2 Euclidean magnitude (square root of sum of squared vector elements)We use two math tools: an ultra-fast local checker to instantly classify what search topic your page covers, and a deep 768-dimensional AI model from Google to measure cosine similarity under the selected embedding configuration.
2. RAG Retrieval Simulation & Content Structuring
AEObility currently tests focused answer blocks of approximately 90–120 tokens as an internal content-testing heuristic. Restructuring copy into focused blocks can improve topical isolation within AEObility's retrieval simulation.
In vector retrieval testing, 90–120 tokens (roughly 70–95 words) represent the sweet spot for passage chunking. Longer passages risk combining multiple topics and diluting vector focus. Shorter snippets lack sufficient semantic context for neural embeddings to score high similarity.
Note: 90–120 tokens is an internal evaluation heuristic used by AEObility for diagnostic stress-testing, not a universal retrieval requirement across all AI platforms.What is Content Dilution in Retrieval Tests?
Content Dilution occurs when a single document contains a wide mixture of disparate topics (e.g. backstory, shipping rules, and multiple services). In an internal retrieval test, embedding mixed copy as one block can make the passage less similar to a specific query than a focused passage indexed separately.
In AEObility’s retrieval simulation, a mixed-topic passage may score below the internal simulation pass threshold even when it contains relevant information. Restructuring copy into focused 90–120 token blocks can improve topical isolation within our test environment.
AEObility's Four-Part Mitigation Architecture
AEObility uses text-embedding-004 as its selected dense representation model for semantic comparison. Model identifiers and provider availability may change. This page describes Architecture v2.4, last tested in August 2026.
In rag-sim.ts, content is split into atomic paragraph chunks C1, C2, ..., Ck (k ≤ 5, approximately 90–120 tokens). Each chunk receives an isolated text-embedding-004 vector v_C_i and is evaluated independently.
In graph.ts, unstructured text is refactored into Subject-Predicate-Object (SPO) entity triples to form hyper-focused semantic nodes.
In rag-sim.ts, gemini-3.5-flash generates 3 synthetic query variations from the target search intent. Chunk embeddings are evaluated against these queries to measure simulation survival rates.
AEObility currently treats a cosine-similarity score above 0.62 as an internal simulation pass threshold within this specific retrieval simulation, model configuration, and evaluation design.
Different embedding models produce different baseline similarity score ranges. For instance, Google's text-embedding-004 typically outputs similarity scores between 0.55 – 0.82 for relevant technical content, whereas OpenAI's older text-embedding-ada-002 produced higher baseline numbers (0.75 – 0.92) for similar pairs. Consequently, a threshold of 0.62 is specific to our test setup and cannot be directly compared across different model families.
We break relevant page content into short topic blocks, typically around 90–120 tokens, and test whether each block addresses key buyer questions within our test environment.
3. System APIs & Execution Flow
The diagnostic engine orchestrates Next.js serverless route handlers, Google Gemini APIs, and public SPARQL knowledge bases to execute end-to-end audits.
The diagnostic engine coordinates client-side execution, serverless route handlers, generative embedding APIs, and public SPARQL knowledge bases:
| API / Endpoint | Path / Provider | Input / Output Type | Function Specification |
|---|---|---|---|
| POST /api/diagnostic | src/app/api/diagnostic/route.ts | URL + Intent → Diagnostic JSON | Executes 3-stage async crawl, vector proximity embedding, RAG simulation (Internal pass threshold: 0.62), entity graph extraction, scoring, and strategic insight generation. |
| POST /api/bill | src/app/api/bill/route.ts | Messages → Streamed EventStream | Edge-streamed conversational AI assistant endpoint using OpenAI gpt-4o-mini via Vercel AI SDK. Ingests telemetry payloads. |
| GET & POST /api/search/answer | src/app/api/search/answer/route.ts | Query string or JSON → Answer Object | Grounded NLWeb vector search answer endpoint returning 2-sentence answers and similarity scores. Supports GET pre-flight discovery/query strings (?q=query) and POST JSON vector payloads. |
| GET /api/mcp | src/app/api/mcp/route.ts | HTTP GET → Tool Catalogue JSON | Publishes machine-readable tool catalogue for compatible client agents. |
| text-embedding-004 | Google Gemini API | Text String → 768-dim Float Array | Generates 768-dimensional dense vector embeddings for target search intent and site text. |
| gemini-3.5-flash | Google Gemini API | Prompt → Structured JSON Response | Powers query-variation generation, SPO triple extraction, and Strategic Insight Engine synthesis. |
| Wikidata SPARQL | query.wikidata.org | SPARQL Query → RDF Entity Match | Validates extracted entity subjects against global open knowledge graphs. |
4. 5-Category Weighted Scoring Model & Mathematics
The AI Readiness Score ($0-100$) is calculated from 5 normalized category dimensions using transparent weightings summing to 100%. Use our interactive simulator below to test custom category inputs.
Semantic relevance represents the single largest factor (40%) in the AI Readiness Score because neural retrieval systems prioritize vector similarity above all else when answering user queries. If a website's content is semantically distant from the target search intent, perfect technical code or schema cannot force AI engines to retrieve it.
Think of the 5 categories like a modern business assessment: Semantic Relevance (40%) tests if you actually answer the customer's question; Technical Readiness (20%) verifies your site is fast and clean; Entity Clarity (15%) ensures your brand services are explicitly declared; Competitor Coverage (15%) checks if you cover the topic in sufficient depth; and KG Corroboration (10%) checks if external databases verify your details.
5-Category Score Weight Visual Breakdown
In src/lib/telemetry/config.ts, the AI Readiness Score ($0-100$) is calculated across 5 normalized category dimensions:
In src/lib/telemetry/scoring.ts, each category score (S, T, E, C, K) is bounded to [0, 100] and weighted transparently:
S: Semantic Relevance score (0-100)T: Technical Readiness score (0-100)E: Entity Clarity score (0-100)C: Competitor Coverage score (0-100)K: Knowledge Graph Corroboration score (0-100) — a measure of whether extracted entity information can be matched against open knowledge graphs.5-Category Score Weighting Simulator
Adjust category sub-scores (0-100) to observe real-time weighted normalization.
Semantic Dominance Bounded Score
Relative similarity difference is calculated with a signed delta and mapped to a neutral 50 midpoint to avoid harsh zero floors:
6-Point Schema Completeness Rubric
Evaluates structured data quality across 6 qualitative validation criteria:
- Valid JSON-LD markup present.
- Schema type matches visible content.
- Required/recommended properties populated.
- Entity identifiers (@id) consistent.
- No conflict with page copy.
- Accessible to search crawlers.
Your final score (0-100) is calculated like a weighted report card. Content quality accounts for 40%, technical code 20%, entity data 15%, competitor depth 15%, and external verification 10%.
5. Ingestion Pipeline into AI Bill
Diagnostic results are saved to client localStorage and streamed into AI Bill, which dynamically switches between UI report card generation on Turn 1 and conversational Q&A on Turn 2+.
When a user completes a diagnostic scan on /diagnostic, the resulting telemetry payload is saved to localStorage (aeo_telemetry_latest) and handed off to AI Bill via custom browser events (open_bill_with_query).
Scraped website page copy never enters the AI chat stream directly; only the structured telemetry summary payload is saved locally and handed off to AI Bill to prevent prompt bloat.
Multi-Turn Skill Routing & Report Cards
In src/app/api/bill/route.ts, AI Bill evaluates user turn counts to determine skill routing:
- Turn 1 (Diagnostic Turn): Activates
[ACTIVE SKILL: Telemetry Guide]and enforces a strict report block ([START_TELEMETRY_REPORT] ... [END_TELEMETRY_REPORT]). InBillWidget.tsx,parseTelemetryText()extracts metrics via regex to render interactive UI cards (Clarity Index, Citation Share, First Impression, Blind Spot, Verdict). - Turn 2+ (Follow-up Turn): Activates
[ACTIVE SKILL: Telemetry Consultant], answering follow-up questions conversationally in two or three direct sentences using the audit payload context without re-emitting cards.
6. Machine Interface Protocols: NLWeb & MCP
AEObility publishes discovery link tags in page headers and a machine tool catalogue at /api/mcp for compatible AI agents.
AEObility publishes a machine-readable tool catalogue at /api/mcp and provides an MCP-compatible integration layer for supported clients.
Discovery Head Link Tags
In src/app/layout.tsx, these link relations are published as optional discovery metadata:
MCP Endpoint Tools (/api/mcp)
Exposes machine tool schemas for autonomous agent execution:
- •
get_organization_entity - •
get_founder_entity - •
query_knowledge_hub_node - •
get_service_module
When an external AI agent queries AEObility's machine endpoint, it issues a structured JSON payload:
// Machine Agent Call to GET /api/mcp
{
"tool": "query_knowledge_hub_node",
"arguments": {
"topic": "RAG information dilution",
"format": "JSON-LD"
}
}We publish machine-readable API routes so AI search crawlers can ask our site direct questions and query our business services programmatically.
7. Privacy, Data Handling & Security Controls
Diagnostic audits process copy transiently in memory, store results locally in the user's browser, and enforce strict rate limits and zero data-retention model API policies.
AEObility maintains strict data handling and security boundaries across the telemetry execution lifecycle:
Scope & In-Memory Crawl Processing
URLs and user-entered intents submitted to /api/diagnostic are processed transiently in memory during the execution turn. External pages are parsed strictly for text extraction and schema validation.
Client-Side Retention & User Control
Diagnostic output is stored in the browser's localStorage (aeo_telemetry_latest). Users can clear diagnostic state at any time by clearing site data or invoking client reset methods.
Payload Isolation & AI Bill Handoff
AI Bill receives an abridged scorecard summary payload rather than raw scraped HTML, preventing prompt bloat and isolating third-party copy.
Rate Limiting & Provider Policies
API endpoints enforce rate limits and payload size caps. Requests to external AI models adhere to zero data-retention for model training under commercial enterprise API terms.
System Scope Bounds & Future Roadmap
AEObility Architecture v2.4 provides a controlled, reproducible diagnostic baseline. Transparency requires defining both current scope bounds and planned architectural improvements:
Current Scope Bounds (v2.4)
- • Single-Page Analysis: Audits target URL copy and schema; site-wide crawl checks are evaluated via fan-out sampling.
- • Text & Schema Focus: Evaluates text block embeddings and JSON-LD markup; image/video embeddings are excluded from vector proximity tests.
- • Directional Pass Thresholds: Internal simulation benchmark threshold of 0.62 applies to the Gemini text-embedding-004 configuration.
Planned Architectural Roadmap (v2.5+)
- • Multimodal Passage Embeddings: Stress-testing image captioning and tabular data vector representation.
- • Cross-Model Ensemble Comparison: Evaluating chunk survival rates across multiple open-weights embedding models simultaneously.
- • Direct SPARQL Graph Validation: Automated graph entity assertion checks against custom enterprise knowledge bases.
8. Continue Exploring
This guide explains how AEObility’s Telemetry Diagnostic works. The resources below explore the key ideas behind the system, including retrieval, structured data, semantic search, and the practical steps involved in improving AI visibility.
Telemetry Diagnostic Engine FAQ
What does the AEObility Telemetry Diagnostic measure?
The Telemetry Diagnostic evaluates selected signals associated with machine-readable, retrieval-friendly website content. It uses AEObility’s proprietary crawl, semantic similarity, entity, competitor, and retrieval-simulation methods to identify opportunities relevant to conventional search and AI-mediated discovery.
Does the diagnostic access live platforms like ChatGPT, Gemini, or Claude?
No. The diagnostic does not reproduce or access the proprietary retrieval, ranking, citation, or recommendation systems of ChatGPT, Perplexity, Gemini, Claude, Google, or other external platforms. Its findings are directional diagnostic signals, not predictions or guarantees of visibility.
How is the 0.62 cosine similarity threshold interpreted?
AEObility currently treats a cosine similarity score above 0.62 as an internal simulation pass condition within this specific retrieval simulation, model configuration, and evaluation design. Cosine similarity values are not portable across embedding models, content types, vector indexes, or third-party AI products.
How does the MCP endpoint (/api/mcp) interact with clients?
AEObility publishes a machine-readable tool catalogue at /api/mcp and provides an MCP-compatible integration layer for supported clients to query structured entity nodes.
Ready to inspect your site's AI visibility score?
Run a live telemetry diagnostic on your website to evaluate vector proximity, entity clarity, and RAG retrieval survival.
