WebGraph

Engine walkthrough

You hand it one address. Here is everything that happens next, on a real page.

Fourteen stages in the order they run. Each one says what it takes in, what it hands on, what it does when something goes wrong — and beside it, what its output actually looks like: the bytes each fetch returned, a block with its address and its box, the reading order against the source order, the Markdown written from those blocks, and a fact in the knowledge graph with the evidence it rests on.

The example running through every stage is https://lakshx.in/, captured on 2026-09-22 at engine commit acbc09f. Nothing in it is typed in by hand except the model’s answer in the last stage, marked where it appears. To regenerate it: uv run python tools/walkthrough_sample.py.

Part one

One page

What happens to a single address, in the order the code runs it. The example beside each stage is one real page, measured once and shown as it is.

  1. Stage 01
    one URLHTML, both ways

    Fetch the page both ways, and keep both answers

    Every other engine picks one: plain HTTP, or a real browser. This one takes both, because measuring 24 real sites showed two things are true at once.

    You cannot predict what a plain fetch will miss. Angular's own site loses 32% of its content without a browser, and a page holding 2,078 characters gives no hint that 969 more appear after its scripts run. At the other extreme a React app returns an empty root element and nothing else, and every word a reader sees arrives when the scripts do.

    And a browser is not a strict upgrade. The BBC gives 19,908 characters over plain HTTP and only 9,279 through a browser, because a consent wall replaces the article. Choosing the browser would have thrown away half the page.

    The browser does two more things while it is there. It records the bounding box of every element that holds text, which is what reading order is measured from later. And it marks, on the element itself, anything it laid out and hid: display: none, visibility: hidden, boxes clipped or pushed off screen, and toast containers.

    in: one URLout: plain HTMLout: rendered HTML with boxes and hidden marksout: the final URL
    What it looks likelakshx.in, fetched both ways
    Plain HTTP
    38,559 bytes
    status 200
    text it held
    2,519 chars
    what the server sent
    <!DOCTYPE html><html lang="en" class="inter_1ec0685a-module__ODW2ua__variable poppins_ed9e8a82-module__jm5_za__variable playfair_display_38b6700f-module__B0Xt2q__variable font-sans geist_da832ead-module__aN_Ytq__variable"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="preload" href="/_next/static/media/47fe1b7cd6e6ed85-s.p.3bh2vc0w-r-ll.woff2" as="font" crossorigin="" type="font/woff2"/><link rel="preload" href="/_next/static/media/70e3db2de7f94926-s.p.39pl-v7c3qrze.woff2" as="font" crossorigin="" type="font/woff2"/><link rel="preload" href="/_next/static/media/83afe278b6a6bb3c-s.p.2bn3s6zvc0dyp.woff2" as="font" crossorigin="" type="font/woff2"/><link rel="preload" href="/_next/static/media/caa3a2e1cccd8315-s.p.0wgildi0cnwt9.woff2" as="font" crossorigin="" type="font/woff2"/><link rel="preload" href="/_next/static/media/e2334d715941921e-s.p.3o_v2fun1jzxk.woff2" as="font" crossorigin="" type="font/woff2"/><link rel="preload"…
    Real browser
    48,630 bytes
    148 boxes measured
    text it held
    2,592 chars
    one element after the scripts ran, final URL https://lakshx.in/
    <h1 class="text-balance bg-gradient-to-b from-white via-white to-lakshx-violet-active bg-clip-text font-display text-4xl italic font-bold leading-[1.1] tracking-tight text-transparent drop-shadow-[0_2px_24px_rgba(0,0,0,0.45)] sm:text-5xl sm:leading-[1.08] md:text-6xl" style="opacity: 1; transform: none;" data-wg-id="86" data-wg-brk="1">An agentic IDE, not just autocomplete.</h1>

    This page is rendered on the server, so both fetches hold nearly the same words — 2,519 characters against 2,592. Which way it goes is not knowable before both are in hand, which is the whole reason for taking both. The browser’s copy also carries a measured box for 148 elements, and the data-wg-* attributes are its own marks, written onto the elements for the stages that follow.

    When it goes wrong

    404 or 410
    The only fatal statuses. The page is dropped without rendering, because a browser renders a “not found” page perfectly happily and it would be extracted as though it were an article.
    403 refused
    Not fatal. It escalates to the real browser, which recovers 55% of refusals — not by imitating a browser, but by being one.
    429 or 503
    Means “later”, not “no”. One retry, honouring the wait the server asked for, capped at five seconds. One, because a crawler that retries hard on a 429 is the reason it was sent.
    Browser unavailable
    Falls back to the plain fetch and records that rendering did not happen, so a result is never silently weaker than it claims to be.
    An age gate or region picker
    The browser tries to open it, because a page stuck behind an interstitial never mounts its content at all.
  2. Stage 02
    HTML, both waysstripped tree

    Remove what is not text, and what the reader never saw

    The HTML that comes back is mostly not words. Before any block is built, three kinds of thing go, and the text that follows each of them is kept, because dropping an inline script must not also drop the sentence after it.

    First, tags that never hold prose: script, style, noscript, template, svg, iframe, object, embed, audio, video. An icon drawn as SVG next to a list item is hundreds of characters of path data; the item's few words are what remains.

    Second, what the browser marked hidden. Only the browser's own marks count — a plain fetch marks nothing — and only for display, visibility, clipping, off-screen and toast containers. Opacity is not hiding: a scroll animation starts its text at opacity 0, and that text is on the page. A hidden panel that a control on the page opens (a tab, an accordion, a “show more”) stays, because a reader can reach it. A hidden panel nothing opens — the source a news ticker reads from, a 3,144-word announcements div beside a 726-word page — goes.

    Third, controls that are labels rather than content: a heading's ¶ permalink, a wiki's [edit], screen-reader-only text. These are the one thing a caller can ask to keep, for a use that wants every string in the DOM.

    in: rendered HTMLout: a tree that holds only what a reader saw
    What it looks likewhat goes before a block is built, on lakshx.in
    <script> stripped
    26
    <svg> stripped
    15
    <template> stripped
    1
    marked hidden
    65
    script 26, meta 25, link 11, head 1, title 1, div 1
    toast containers
    1
    dropped
    opacity marks
    13
    kept — an animation, not hiding
    one list item, as the browser holds it
    <li class="flex items-start gap-2.5 text-sm text-ink-navy/70" data-wg-id="157" data-wg-brk="1"><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewbox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-check mt-0.5 h-4 w-4 shrink-0 text-lakshx-violet" aria-hidden="true" data-wg-id="158" data-wg-brk="1"><path d="M20 6 9 17l-5-5" data-wg-id="159"></path></svg>The full IDE — every agent mode, checkpoints &amp; undo, voice mode</li>
    what is left for the block
    The full IDE — every agent mode, checkpoints & undo, voice mode
    what the browser marked, and what the engine did with it
    markelementkept?its text
    notification<div>dropped(empty)
    opacity<h2>keptAn agent you can actually trust with your codebase

    The heading in the last row is the case the rule exists for: it is hidden at the moment the page loads because it fades in as you scroll, and a reader sees every word of it. Opacity is never treated as hiding. Nothing hidden on this page held words a reader could have seen: the display marks are the document head, its metadata, its scripts and one empty element — script 26, meta 25, link 11, head 1, title 1, div 1.

    Full-page mode promises that no visible character is missed, and this stage is where that promise is either kept or broken. So the rule is narrow: an element is dropped for being hidden only when the browser itself measured it as hidden and nothing on the page can open it.

  3. Stage 03
    stripped treeblocks

    Turn what is left into blocks

    A block is the engine's unit of text: one innermost element that actually holds words, with everything the rest of the engine will need to know about it. Not the HTML, not the whole page — one paragraph, one heading, one list item, one image, one table.

    Each block carries its kind (paragraph, heading, list-item, image, table, code, quote), the tag it came from, its XPath in the page, its text, a rich form with links and emphasis when there are any, the box the browser measured for it, the region it sits in (header, nav, footer, or none for the body), and its position in the source.

    The XPath is the address everything else keys on. A fact found later, a quote an answer rests on, a block the content view dropped — each points back to a block by its XPath, so anything the engine says about a page can be traced to the element it came from.

    in: the stripped treeout: typed blocks, each with an address, a box and a region
    What it looks like45 blocks on lakshx.in
    the element
    <span class="inline-flex items-center gap-1.5 rounded-full border border-white/25 bg-white/10 px-4 py-1.5 text-sm text-white backdrop-blur-sm" style="opacity: 1; transform: none;" data-wg-id="84" data-wg-brk="1"><span aria-hidden="true" class="text-lakshx-violet-active" data-wg-id="85" data-wg-brk="1">✦</span>India’s #1 Agentic Coding IDE</span>
    the block
    {
     "kind": "paragraph",
     "tag": "div",
     "xpath": "/html/body/main/div[1]/div/div[3]/div/span",
     "text": "✦India’s #1 Agentic Coding IDE",
     "rich_text": null,
     "rect": {
      "x": 598,
      "y": 267,
      "w": 245,
      "h": 34
     },
     "region": "main",
     "href": null,
     "alt": null,
     "level": null,
     "dom_index": 3
    }
    one block of each kind, and one from the footer
    kindtagregionbox (x, y, w, h)text
    paragraphdivmain598, 267, 245, 34✦India’s #1 Agentic Coding IDE
    headingh1main384, 325, 672, 130An agentic IDE, not just autocomplete.
    list-itemlimain289, 2572, 378, 40The full IDE — every agent mode, checkpoints & undo, voice mode
    imageimgmain—
    paragraphdivmain1220, 32, 180, 34[Changelog](https://lakshx.in/changelog) [Docs](https://lakshx.in/docs)
    paragraph
    27
    heading
    9
    list-item
    8
    image
    1
    in main
    40
    in footer
    4
    in nav
    1
    • Forum threads hidden from crawlersDiscourse serves an empty shell and puts the whole thread in a fallback block that is normally thrown away. 19 of 112 forum pages produced nothing at all before this.
    • EquationsMathematical markup is converted to LaTeX before anything can strip it. It used to be deleted outright, which on a scientific page removes the very thing the page is about.
    • Dropdowns and form controlsA 200-country selector is 400 unlinked words and beats a real article on any density measure. On one page the engine returned the country list instead of the terms and conditions.
    • TablesA real data table keeps its grid, merged cells and all. A table with one row, one column, or mostly empty cells is page furniture wearing table markup, and is read as ordinary content instead.
    • Icon-only links and inline imagesA link whose only content is an icon takes its name from its aria-label or title, so “LinkedIn” survives where the icon does not. A named inline image is kept as an image block; a 1×1 lazy-loading placeholder is not.
  4. Stage 04
    blocksblocks in order

    Put the blocks in reading order

    Source order is not reading order. Any page using CSS to rearrange its layout hands you its content in the wrong sequence, and you cannot tell from the text alone.

    So the boxes the browser measured are used to reconstruct the order a person reads in: the page is repeatedly split into columns and rows (an XY-cut), and blocks are read top to bottom within each. Blocks with no measurable box — a collapsed panel, anything behind a disclosure — are placed next to their neighbours in the source.

    Right-to-left pages are detected from the document and read column-wise from the right. That used to default to left-to-right for every caller but the CLI, which read Arabic multi-column pages backwards while reporting a measured order.

    in: blocks with boxesout: blocks in the order a person reads themout: how the order was obtained
    What it looks likegeometric-anchored on lakshx.in
    4 consecutive blocks in reading order, with where each sat in the source and on the screen
    readssource #xykindtext
    10——image(an image)
    214033paragraphLakshX
    32122032paragraphChangelog Docs
    43598267paragraph✦India’s #1 Agentic Coding IDE
    method
    geometric-anchored
    boxes measured
    148
    blocks with no box
    1
    anchored to their neighbours
    order differs from source
    0 places

    These blocks sit in 3 columns across the top of the page, and the engine reads them left to right: the brand, then the links beside it. Here the measured order agrees with the source order everywhere, which is the common case and not one you can count on — the measurement is what makes it a fact rather than an assumption. The method is geometric-anchored rather than a fully measured one because 1 block has no box at all — here the hero image, which the browser laid out with no measurable size — so it is anchored beside its neighbours in the source, and the page is told which it got.

    It always says which it did: measured order, measured with some blocks anchored, or plain source order when there was no browser. A consumer is told the confidence rather than left to assume it.

  5. Stage 05
    blocks in orderone document

    Merge the two fetches, losing nothing

    Both fetches have been through the three stages above separately. Now they are one document. The rendered document leads, because its order is measured. A block only the plain fetch had is not thrown away and not guessed into place: it is inserted after the nearest block both documents contain, which is the same rule that anchors an unmeasured block within one document.

    Two exceptions keep the merge honest. A plain-fetch block that the browser marked hidden is not something the render lost, it is something the render hid, and it stays out — php.net's manual table of contents is a hundred links under display: none that this merge used to put straight back. And a block under a client-side template directive the render never took is a branch the framework chose not to build.

    The result reports what each side contributed, which turns “nothing was lost” from a promise into a measurement.

    in: two ordered documentsout: one documentout: characters and blocks each fetch contributed
    What it looks likethe two fetches of lakshx.in, merged
    strategy
    union
    plain fetch held
    2,519 chars
    browser held
    2,592 chars
    merged document
    2,592 chars
    blocks only the browser had
    3
    blocks only the plain fetch had
    0
    blocks after merge
    45
    blocks only the browser had
    Download for macOS
    Upgrade to Pro
    Have a promo code?

    The two fetches agree on the prose; what only the browser had is the interface the page mounts around it. Nothing here was lost either way — on a site whose plain fetch is a consent wall, the numbers run the other way and the plain fetch is the side holding the article.

    When it goes wrong

    The plain fetch was a wall
    A login page, a challenge, a consent gate. The wall's text is left out and the result says what was refused.
    Both fetches were walls
    The page is refused with the wall's own evidence rather than extracted as an article about cookies.
  6. Stage 06
    one documentcontent view

    Say what kind of page it is, then decide what counts as the content

    A small gradient-boosted model reads 126 features of the document — its markup statistics, its declared URL, its landmarks, how its text is distributed — and names one of seven page types: article, documentation, service, forum, collection, listing, product. Below a confidence of 0.5 it says unknown rather than guess.

    The type picks a policy, and the policy runs four steps in order. Drop what the page itself labels navigation and footer. Narrow to the main region when the page declares a trustworthy one. Remove the furniture this site repeats on every page, learned in the crawl. Then draw a boundary around the real content.

    in: one documentout: the page type and its confidenceout: the content viewout: which steps fired
    What it looks likethe router and the content view on lakshx.in
    page type
    service
    confidence 0.997
    blocks kept
    34
    blocks dropped
    11
    steps that fired
    landmarks, main-content
    what the content view dropped, and which step dropped it
    dropped byregionkindtext
    the boundarymainimage(no text — a decorative image)
    the boundarymainparagraphLakshX
    the boundarymainparagraphChangelog Docs
    the boundarymainparagraph✦India’s #1 Agentic Coding IDE
    the boundarymainparagraphUpgrade to Pro
    the boundarymainparagraphHave a promo code?
    the page's own landmarksfooterparagraphLakshX
    the page's own landmarksfooterparagraphAn agentic coding IDE — plans, edits, and runs commands across your r…
    the page's own landmarksnavparagraphDocs Changelog Terms of Service Privacy Policy Refund Policy
    the page's own landmarksfooterparagraph© 2026 LakshX. All rights reserved.
    the page's own landmarksfooterparagraphMade for developers who ship.

    Two different mechanisms. 5 of these sit inside the page’s own <footer> and <nav>, so the first step removes them without reading a word. The other 6 are inside <main> — this page puts its top bar there and declares no <header> at all — so no landmark marks them, and it is the last step, the boundary around the prose, that leaves them out: a brand, its two header links, a badge above the headline, and two buttons in the pricing card.

    The full-page view still has all 45, which is the point of computing a view rather than editing the list.

    This never destroys anything. The complete block list stays complete, and “the content” is a separate, narrower view computed from it — so the full page and the article are both available, and the reduction is always reversible.

  7. Stage 07
    content viewMarkdown

    Write the blocks out as Markdown

    Markdown is written from the blocks, not from the HTML. Each block becomes the Markdown its kind calls for — a heading of its level, a paragraph, a list item, an image with its alt text and address, a table with its grid, a fenced code block, an equation kept as TeX — in the reading order the blocks are already in, with the links and emphasis a block's rich text recorded.

    Two documents can be written from the same blocks. The full page writes every block, footer and all. The content view writes only the blocks the previous stage kept. On the example page the difference is the header, the navigation and the footer.

    This is the text a model reads when it is asked about the page, and the blocks are what its answer is checked against, so the two must say the same thing. They do by construction, because one is written from the other.

    in: blocks in reading orderout: Markdown, full page or content viewout: plain text
    What it looks likethe sample blocks above, written out
    kindtext
    paragraph✦India’s #1 Agentic Coding IDE
    headingAn agentic IDE, not just autocomplete.
    list-itemThe full IDE — every agent mode, checkpoints & undo, voice mode
    image
    paragraph[Changelog](https://lakshx.in/changelog) [Docs](https://lakshx.in/docs)
    Markdown
    ✦India’s #1 Agentic Coding IDE
    
    # An agentic IDE, not just autocomplete.
    
    - The full IDE — every agent mode, checkpoints & undo, voice mode
    
    ![](https://lakshx.in/_next/image?url=%2Fhero-bg.jpg&w=3840&q=75)
    
    [Changelog](https://lakshx.in/changelog) [Docs](https://lakshx.in/docs)
    
    full page
    458 words
    3,164 chars
    content view
    402 words
    2,550 chars
    the difference
    56 words
    11 blocks of header, nav, footer

Part two

A whole site

What a crawl adds around that: a queue, a loop, the furniture every page shares, and at the end two graphs -- one observed, one inferred.

  1. Stage 08
    one URLroot page

    Look at the front door, once

    The home page is fetched both ways, as above. In the same pass the crawl reads the site's robots file if asked to honour it, collects every sitemap that file advertises, and identifies the technology behind the site from 237 fingerprint rules.

    in: one URLout: the root pageout: sitemap URLsout: technology profile

    When it goes wrong

    Site unreachable
    The crawl stops here and says why. Nothing else can proceed.
    Redirected elsewhere
    The new address becomes the root, and the redirect is reported rather than followed silently.
    No sitemap published
    Fine. The crawl discovers pages by following links instead.
  2. Stage 09
    root pagequeue

    Build the queue, and clean every address going into it

    Sitemap pages seed a queue. The unglamorous part matters most: four addresses differing only by a trailing slash, a fragment, and a tracking parameter are one page. A crawler that treats them as four spends its budget four times and puts four copies of every fact in your graph.

    So it strips about two dozen tracking parameters, normalises ports and index filenames, and skips addresses that are clearly not pages. PDFs are deliberately kept, because a PDF is a document worth reading.

    in: sitemap URLsout: a de-duplicated queue
  3. Stage 10
    queuepage events

    Crawl and extract at the same time

    Discovery and extraction are interleaved, not sequential. Pages are taken from the queue in batches, every page goes through the seven stages above, and every page that comes back has its links read and added to the queue.

    So the crawl reaches everything reachable rather than only what the sitemap listed, and the first result arrives in seconds instead of after a full enumeration.

    Once a page's links have been read, its raw markup is discarded. On a long article that markup is the single largest thing in memory and nothing needs it again.

    in: the queueout: one result event per page, as it finishes
    What it looks likewhat the crawl of lakshx.in emitted
    stage
    3
    analysis
    1
    discovery
    1
    frontier
    1
    page
    12
    fetching
    11
    done
    1

    One event per page as it finishes, not one batch at the end. No page on this site was served under two addresses; where that happens the second is reported as an alias rather than extracted again.

    When it goes wrong

    A page fails
    Recorded against that page. The crawl carries on. One bad page never stops a run.
    Two addresses serve one page
    The second is reported as an alias of the first, not extracted twice. A privacy-policy link that the server answers with the home page is one home page.
    Budget reached
    Stops cleanly at the page limit you set, or runs until the queue empties if you set none.
  4. Stage 11
    page eventsrepeated blocks

    Learn what the site repeats

    Once six or more pages are in, the engine compares them and finds the blocks that appear on all of them. That is the site's furniture: its header, its footer, its cookie notice, its sidebar. No single page can reveal this; it only shows up across a site. In a crawl this is the third of the four content steps above.

    A guard refuses to call more than half of any page furniture. On a documentation site where every page genuinely resembles its neighbours, that rule is the difference between removing the navigation and removing the documentation.

    in: six or more extracted pagesout: the blocks this site repeats everywhere
  5. Stage 12
    page eventsobserved graph

    Build the observed graph: what the site says about itself

    No model is involved here. As each page arrives, its blocks are cut into sections — a heading owns everything beneath it until the next heading of equal or higher level, which is the author's own idea of where a topic starts and stops. This is only possible because reading order was recovered first: on a multi-column page, source order does not say which paragraph sits under which heading.

    Every link between pages is recorded with the anchor text it was clicked through, and a link that appears on every page of a site is one edge carrying the number of times it was seen. Entities come only from what pages declare in JSON-LD or microdata, or from a subject several pages agree on by their anchors. Aliases join the addresses that turned out to be one page.

    A section keeps the XPath and character span of every block it was joined from, so anything read out of a section can be traced back to a block.

    in: every extracted pageout: pagesout: sections with block spansout: labelled linksout: declared entitiesout: aliases
    What it looks likethe observed graph after 12 pages of lakshx.in
    pages
    12
    sections
    111
    links
    143
    declared entities
    1
    from open-graph
    aliases
    0
    sections of the home page, in reading order
    idlevelheadingblockschars
    lakshx.in/#s00(before the first heading)3111
    lakshx.in/#s11An agentic IDE, not just autocomplete.4436
    lakshx.in/#s22An agent you can actually trust with your codebase1181
    lakshx.in/#s33Pick your autonomy level1242
    lakshx.in/#s43Every edit is reversible1203
    lakshx.in/#s53Any provider, or the free hosted model1184
    entities the pages declared about themselves — no model involved
    typenamepages
    open-graphLakshX — India's #1 Agentic Coding IDE12
    links, with the anchor text they were clicked through
    fromtoanchorsseen
    lakshx.in/docslakshx.in/docs/installationInstallation · Install it · Next Installation3
    lakshx.in/docslakshx.in/docs/sign-inSign In & Models · sign in or add a model · configure your own provider key3
    lakshx.in/docslakshx.in/docs/chatThe Chat Panel · chat panel · The Chat PanelTalk to the agent, attach files, @-mention, and steer a run.3
    lakshx.in/docslakshx.in/docs/modesAgent Modes · safety modes · Agent ModesReview, Approve, Auto, and Royal — how much you let the agent do.3
    lakshx.in/changeloglakshx.in/Home · Download · home3
    lakshx.in/termslakshx.in/privacyPrivacy Policy3
  6. Stage 13
    observed graphinferred graph

    Build the inferred graph: what a model reads out of each section

    Off by default; on with WEBGRAPH_KG=1 and a provider key. A model is given one section at a time — its page, its heading path, the entities already known on the page, and the section's blocks numbered [b0], [b1], … — and asked to state as data what the section says: entities with a type, their attributes, and relations between them. Every one of those must carry the block it came from and a quote copied verbatim from that block.

    The model's answer is not trusted. Each quote is located in the block it names — and if not there, in the section's other blocks, because models mis-number more often than they invent. A quote found becomes evidence: page, section, block XPath, character span. A quote not found is discarded, and an entity or relation with no evidence left goes with it. The counts of what was refused are kept.

    Across pages, entities are merged by type and normalised name, by the aliases the model listed, and by near-duplicate names. The result exports to Neo4j and answers questions by walking a few hops and then reading the sections the walk reached.

    in: one section at a timeout: entities, attributes, relationsout: evidence for every oneout: what was refused
    What it looks likeone section of lakshx.in, followed through

    The section “Free” — 7 blocks, 355 characters. The same characters at every step: the elements, the blocks, the Markdown, the model’s numbered input, and the evidence a verified fact points at.

    three of its elements
    <p class="mt-1 text-sm text-ink-navy/50" data-wg-id="152" data-wg-brk="1">For BYOK users, and anyone getting started</p>
    
    <div class="mt-6 flex items-baseline gap-1" data-wg-id="153" data-wg-brk="1"><span class="font-heading text-4xl font-bold tracking-tight text-ink-navy" data-wg-id="154" data-wg-brk="1">$0</span><span class="text-sm text-ink-navy/50" data-wg-id="155" data-wg-brk="1">forever</span></div>
    
    <li class="flex items-start gap-2.5 text-sm text-ink-navy/70" data-wg-id="157" data-wg-brk="1"><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewbox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-ch…
    the same three blocks (text, address, box)
    {
     "text": "For BYOK users, and anyone getting started",
     "xpath": "/html/body/main/div[2]/section[2]/div/div[2]/div[1]/p",
     "rect": {
      "x": 289,
      "y": 2456,
      "w": 378,
      "h": 20
     }
    }
    {
     "text": "$0 forever",
     "xpath": "/html/body/main/div[2]/section[2]/div/div[2]/div[1]/div",
     "rect": {
      "x": 289,
      "y": 2500,
      "w": 378,
      "h": 40
     }
    }
    {
     "text": "The full IDE — every agent mode, checkpoints & undo, voice mode",
     "xpath": "/html/body/main/div[2]/section[2]/div/div[2]/div[1]/ul/li[1]",
     "rect": {
      "x": 289,
      "y": 2572,
      "w": 378,
      "h": 40
     }
    }
    the section as Markdown
    For BYOK users, and anyone getting started
    
    \$0 forever
    
    - The full IDE — every agent mode, checkpoints & undo, voice mode
    - Bring your own API key from any supported provider — fully unlimited, forever
    - Or sign in free with Google for the hosted LakshX model, up to a starter trial amount
    - No credit card required
    
    [Download for free](https://lakshx.in/#top)
    
    what the model is told (start of the rules)
    You read one section of a web page and state, as data, what it says.
    
    Rules:
    1. Only what the section states. Nothing from your own knowledge, nothing implied.
    2. Every mention, attribute and relation carries a `block` (the [bN] marker) and a `quote`
       copied VERBATIM from that block: same words, same spelling, same punctuation, 3 to 40
       words. Anything whose quote is not found verbatim in the block is discarded, so copy,
       never paraphrase.
    3. Entity types: use one of Organization, Person, Product, Service, Course, Event, Place,
       Document, Offer, Topic, ContactPoint, Role when it fits; otherwise the most specific
       PascalCase noun. Prices, dates, durations, counts, phone numbers and…
    what the model is given (start of the input)
    Page: LakshX — India's #1 Agentic Coding IDE
    URL: https://lakshx.in/
    Heading path: Free
    
    Known entities on this page (use these names exactly):
    - (none)
    
    Section blocks:
    
    [b0] For BYOK users, and anyone getting started
    
    [b1] $0 forever
    
    [b2] The full IDE — every agent mode, checkpoints & undo, voice mode
    
    [b3] Bring your own API key from any supported provider — fully unlimited, forever
    
    [b4] Or sign in free with Google for the hosted LakshX model, up to a starter trial amount
    
    [b5] No credit card required
    
    [b6] [Download for free](https://lakshx.in/#top)
    
    Return the JSON object.
    the model's answer — written by hand for this walkthrough; the verification below is the real engine
    {
     "entities": [
      {
       "name": "LakshX Free",
       "type": "Offer",
       "aliases": [
        "Free"
       ],
       "attributes": [
        {
         "key": "price",
         "value": "$0 forever",
         "block": "b1",
         "quote": "$0 forever"
        }
       ],
       "mentions": [
        {
         "block": "b3",
         "quote": "Bring your own API key from any supported provider"
        }
       ]
      },
      {
       "name": "hosted LakshX model",
       "type": "Product",
       "aliases": [],
       "attributes": [],
       "mentions": [
        {
         "block": "b4",
         "quote": "sign in free with Google for the hosted LakshX model"
        }
       ]
      },
      {
       "name": "Enterprise plan",
       "type": "Offer",
       "aliases": [],
       "attributes": [],
       "mentions": [
        {
         "block": "b5",
         "quote": "includes a dedicated support engineer"
        }
       ]
      }
     ],
     "relations": [
      {
       "subject": "LakshX Free",
       "predicate": "includes",
       "object": "hosted LakshX model",
       "fact": "The free plan can use the hosted LakshX model after a free Google sign-in, up to a starter trial amount.",
       "evidence": [
        {
         "block": "b4",
         "quote": "sign in free with Google for the hosted LakshX model, up to a starter trial amount"
        }
       ]
      }
     ]
    }
    accepted: LakshX Free (Offer), with an attribute
    {
     "mention": {
      "quote": "Bring your own API key from any supported provider",
      "block_xpath": "/html/body/main/div[2]/section[2]/div/div[2]/div[1]/ul/li[2]",
      "span": [0, 50],
      "section_id": "https://lakshx.in/#s8"
     },
     "attributes": [
      {
       "key": "price",
       "value": "$0 forever",
       "block_xpath": "…/div[2]/div[1]/div",
       "span": [0, 10],
       "quote": "$0 forever"
      }
     ]
    }
    accepted: LakshX Free → includes → hosted LakshX model, 1 piece of evidence
    {
     "fact": "The free plan can use the hosted LakshX model after a free Google sign-in, up to a starter trial amount.",
     "evidence": [
      {
       "block_xpath": "…/div[1]/ul/li[3]",
       "span": [3, 85]
      }
     ]
    }
    refused
    {
     "quote_not_found": 1,
     "entity_without_evidence": 1
    }

    2 of the 3 entities kept their evidence: page, section, block XPath, and the character span inside the block. AAn attribute is a fact about a thing and is verified the same way — the one above points at the characters $0 forever in the block that holds them. The remaining quote was invented, so “Enterprise plan” has nothing to stand on and is not in the graph; the counts beside it are what the engine keeps of that.

    Markdown is what the model reads; blocks are what its answer is checked against. A fact in this graph is never a sentence the model wrote — it is a quote the page contains, with an address a reader can open.

  7. Stage 14
    inferred graphresults

    Hand it back

    Per page: plain text, structure-preserving Markdown in both views, the typed blocks, the page type, which reduction steps fired, and what each fetch contributed.

    Per site: the observed graph, the inferred graph when it was built, and the aliases and failures recorded along the way.

    in: every extracted pageout: text and Markdownout: blocksout: the two graphs

    One last check across the whole crawl

    Three or more pages identical
    A warning. If many different addresses return the same text, the crawl is not extracting a site — it is extracting one gate, over and over. Two identical pages is a coincidence; three is a pattern.

The three ideas everything else follows from

Lose nothing first, narrow afterwards

Every stage that removes something produces a new view rather than editing the original. That is why a mistake in content selection costs you a worse summary and never costs you the page.

A failure is a recorded fact, not an exception

An unreachable host, a refused page, a browser that would not start — each becomes a value attached to that page while the crawl continues. Across thousands of pages, anything else means one bad page ends the run.

Everything points back to a block

A Markdown line, a section, a link label, a fact a model read out: each keeps the XPath of the block it came from. Nothing the engine says about a page is more than one step from the element that says it.

Where the engine places against every public benchmark, including the ones it loses, is on the benchmarks page.