Engine walkthrough
You hand it one address. Here is everything that happens next, on a real page.
Fourteen stages in the order they run. Each one says what it takes in, what it hands on, what it does when something goes wrong — and beside it, what its output actually looks like: the bytes each fetch returned, a block with its address and its box, the reading order against the source order, the Markdown written from those blocks, and a fact in the knowledge graph with the evidence it rests on.
The example running through every stage is https://lakshx.in/, captured on 2026-09-22 at engine commit acbc09f. Nothing in it is typed in by hand except the model’s answer in the last stage, marked where it appears. To regenerate it: uv run python tools/walkthrough_sample.py.
Part one
One page
What happens to a single address, in the order the code runs it. The example beside each stage is one real page, measured once and shown as it is.
- Stage 01one URLHTML, both ways
Fetch the page both ways, and keep both answers
Every other engine picks one: plain HTTP, or a real browser. This one takes both, because measuring 24 real sites showed two things are true at once.
You cannot predict what a plain fetch will miss. Angular's own site loses 32% of its content without a browser, and a page holding 2,078 characters gives no hint that 969 more appear after its scripts run. At the other extreme a React app returns an empty root element and nothing else, and every word a reader sees arrives when the scripts do.
And a browser is not a strict upgrade. The BBC gives 19,908 characters over plain HTTP and only 9,279 through a browser, because a consent wall replaces the article. Choosing the browser would have thrown away half the page.
The browser does two more things while it is there. It records the bounding box of every element that holds text, which is what reading order is measured from later. And it marks, on the element itself, anything it laid out and hid: display: none, visibility: hidden, boxes clipped or pushed off screen, and toast containers.
in: one URLout: plain HTMLout: rendered HTML with boxes and hidden marksout: the final URLWhat it looks likelakshx.in, fetched both ways - Plain HTTP
- 38,559 bytes
- status 200
- text it held
- 2,519 chars
what the server sent<!DOCTYPE html><html lang="en" class="inter_1ec0685a-module__ODW2ua__variable poppins_ed9e8a82-module__jm5_za__variable playfair_display_38b6700f-module__B0Xt2q__variable font-sans geist_da832ead-module__aN_Ytq__variable"><head><meta charSet="utf-8"/><meta name="viewport" content="width=device-width, initial-scale=1"/><link rel="preload" href="/_next/static/media/47fe1b7cd6e6ed85-s.p.3bh2vc0w-r-ll.woff2" as="font" crossorigin="" type="font/woff2"/><link rel="preload" href="/_next/static/media/70e3db2de7f94926-s.p.39pl-v7c3qrze.woff2" as="font" crossorigin="" type="font/woff2"/><link rel="preload" href="/_next/static/media/83afe278b6a6bb3c-s.p.2bn3s6zvc0dyp.woff2" as="font" crossorigin="" type="font/woff2"/><link rel="preload" href="/_next/static/media/caa3a2e1cccd8315-s.p.0wgildi0cnwt9.woff2" as="font" crossorigin="" type="font/woff2"/><link rel="preload" href="/_next/static/media/e2334d715941921e-s.p.3o_v2fun1jzxk.woff2" as="font" crossorigin="" type="font/woff2"/><link rel="preload"…
- Real browser
- 48,630 bytes
- 148 boxes measured
- text it held
- 2,592 chars
one element after the scripts ran, final URL https://lakshx.in/<h1 class="text-balance bg-gradient-to-b from-white via-white to-lakshx-violet-active bg-clip-text font-display text-4xl italic font-bold leading-[1.1] tracking-tight text-transparent drop-shadow-[0_2px_24px_rgba(0,0,0,0.45)] sm:text-5xl sm:leading-[1.08] md:text-6xl" style="opacity: 1; transform: none;" data-wg-id="86" data-wg-brk="1">An agentic IDE, not just autocomplete.</h1>
This page is rendered on the server, so both fetches hold nearly the same words — 2,519 characters against 2,592. Which way it goes is not knowable before both are in hand, which is the whole reason for taking both. The browser’s copy also carries a measured box for 148 elements, and the data-wg-* attributes are its own marks, written onto the elements for the stages that follow.
When it goes wrong
- 404 or 410
- The only fatal statuses. The page is dropped without rendering, because a browser renders a “not found” page perfectly happily and it would be extracted as though it were an article.
- 403 refused
- Not fatal. It escalates to the real browser, which recovers 55% of refusals — not by imitating a browser, but by being one.
- 429 or 503
- Means “later”, not “no”. One retry, honouring the wait the server asked for, capped at five seconds. One, because a crawler that retries hard on a 429 is the reason it was sent.
- Browser unavailable
- Falls back to the plain fetch and records that rendering did not happen, so a result is never silently weaker than it claims to be.
- An age gate or region picker
- The browser tries to open it, because a page stuck behind an interstitial never mounts its content at all.
- Stage 02HTML, both waysstripped tree
Remove what is not text, and what the reader never saw
The HTML that comes back is mostly not words. Before any block is built, three kinds of thing go, and the text that follows each of them is kept, because dropping an inline script must not also drop the sentence after it.
First, tags that never hold prose: script, style, noscript, template, svg, iframe, object, embed, audio, video. An icon drawn as SVG next to a list item is hundreds of characters of path data; the item's few words are what remains.
Second, what the browser marked hidden. Only the browser's own marks count — a plain fetch marks nothing — and only for display, visibility, clipping, off-screen and toast containers. Opacity is not hiding: a scroll animation starts its text at opacity 0, and that text is on the page. A hidden panel that a control on the page opens (a tab, an accordion, a “show more”) stays, because a reader can reach it. A hidden panel nothing opens — the source a news ticker reads from, a 3,144-word announcements div beside a 726-word page — goes.
Third, controls that are labels rather than content: a heading's ¶ permalink, a wiki's [edit], screen-reader-only text. These are the one thing a caller can ask to keep, for a use that wants every string in the DOM.
in: rendered HTMLout: a tree that holds only what a reader sawWhat it looks likewhat goes before a block is built, on lakshx.in - <script> stripped
- 26
- <svg> stripped
- 15
- <template> stripped
- 1
- marked hidden
- 65
- script 26, meta 25, link 11, head 1, title 1, div 1
- toast containers
- 1
- dropped
- opacity marks
- 13
- kept — an animation, not hiding
one list item, as the browser holds it<li class="flex items-start gap-2.5 text-sm text-ink-navy/70" data-wg-id="157" data-wg-brk="1"><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewbox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-check mt-0.5 h-4 w-4 shrink-0 text-lakshx-violet" aria-hidden="true" data-wg-id="158" data-wg-brk="1"><path d="M20 6 9 17l-5-5" data-wg-id="159"></path></svg>The full IDE — every agent mode, checkpoints & undo, voice mode</li>
what is left for the blockThe full IDE — every agent mode, checkpoints & undo, voice mode
what the browser marked, and what the engine did with itmark element kept? its text notification <div> dropped (empty) opacity <h2> kept An agent you can actually trust with your codebase The heading in the last row is the case the rule exists for: it is hidden at the moment the page loads because it fades in as you scroll, and a reader sees every word of it. Opacity is never treated as hiding. Nothing hidden on this page held words a reader could have seen: the display marks are the document head, its metadata, its scripts and one empty element — script 26, meta 25, link 11, head 1, title 1, div 1.
Full-page mode promises that no visible character is missed, and this stage is where that promise is either kept or broken. So the rule is narrow: an element is dropped for being hidden only when the browser itself measured it as hidden and nothing on the page can open it.
- Stage 03stripped treeblocks
Turn what is left into blocks
A block is the engine's unit of text: one innermost element that actually holds words, with everything the rest of the engine will need to know about it. Not the HTML, not the whole page — one paragraph, one heading, one list item, one image, one table.
Each block carries its kind (paragraph, heading, list-item, image, table, code, quote), the tag it came from, its XPath in the page, its text, a rich form with links and emphasis when there are any, the box the browser measured for it, the region it sits in (header, nav, footer, or none for the body), and its position in the source.
The XPath is the address everything else keys on. A fact found later, a quote an answer rests on, a block the content view dropped — each points back to a block by its XPath, so anything the engine says about a page can be traced to the element it came from.
in: the stripped treeout: typed blocks, each with an address, a box and a regionWhat it looks like45 blocks on lakshx.in the element<span class="inline-flex items-center gap-1.5 rounded-full border border-white/25 bg-white/10 px-4 py-1.5 text-sm text-white backdrop-blur-sm" style="opacity: 1; transform: none;" data-wg-id="84" data-wg-brk="1"><span aria-hidden="true" class="text-lakshx-violet-active" data-wg-id="85" data-wg-brk="1">✦</span>India’s #1 Agentic Coding IDE</span>
the block{ "kind": "paragraph", "tag": "div", "xpath": "/html/body/main/div[1]/div/div[3]/div/span", "text": "✦India’s #1 Agentic Coding IDE", "rich_text": null, "rect": { "x": 598, "y": 267, "w": 245, "h": 34 }, "region": "main", "href": null, "alt": null, "level": null, "dom_index": 3 }one block of each kind, and one from the footerkind tag region box (x, y, w, h) text paragraph div main 598, 267, 245, 34 ✦India’s #1 Agentic Coding IDE heading h1 main 384, 325, 672, 130 An agentic IDE, not just autocomplete. list-item li main 289, 2572, 378, 40 The full IDE — every agent mode, checkpoints & undo, voice mode image img main — paragraph div main 1220, 32, 180, 34 [Changelog](https://lakshx.in/changelog) [Docs](https://lakshx.in/docs) - paragraph
- 27
- heading
- 9
- list-item
- 8
- image
- 1
- in main
- 40
- in footer
- 4
- in nav
- 1
- Forum threads hidden from crawlersDiscourse serves an empty shell and puts the whole thread in a fallback block that is normally thrown away. 19 of 112 forum pages produced nothing at all before this.
- EquationsMathematical markup is converted to LaTeX before anything can strip it. It used to be deleted outright, which on a scientific page removes the very thing the page is about.
- Dropdowns and form controlsA 200-country selector is 400 unlinked words and beats a real article on any density measure. On one page the engine returned the country list instead of the terms and conditions.
- TablesA real data table keeps its grid, merged cells and all. A table with one row, one column, or mostly empty cells is page furniture wearing table markup, and is read as ordinary content instead.
- Icon-only links and inline imagesA link whose only content is an icon takes its name from its aria-label or title, so “LinkedIn” survives where the icon does not. A named inline image is kept as an image block; a 1×1 lazy-loading placeholder is not.
- Stage 04blocksblocks in order
Put the blocks in reading order
Source order is not reading order. Any page using CSS to rearrange its layout hands you its content in the wrong sequence, and you cannot tell from the text alone.
So the boxes the browser measured are used to reconstruct the order a person reads in: the page is repeatedly split into columns and rows (an XY-cut), and blocks are read top to bottom within each. Blocks with no measurable box — a collapsed panel, anything behind a disclosure — are placed next to their neighbours in the source.
Right-to-left pages are detected from the document and read column-wise from the right. That used to default to left-to-right for every caller but the CLI, which read Arabic multi-column pages backwards while reporting a measured order.
in: blocks with boxesout: blocks in the order a person reads themout: how the order was obtainedWhat it looks likegeometric-anchored on lakshx.in 4 consecutive blocks in reading order, with where each sat in the source and on the screenreads source # x y kind text 1 0 — — image (an image) 2 1 40 33 paragraph LakshX 3 2 1220 32 paragraph Changelog Docs 4 3 598 267 paragraph ✦India’s #1 Agentic Coding IDE - method
- geometric-anchored
- boxes measured
- 148
- blocks with no box
- 1
- anchored to their neighbours
- order differs from source
- 0 places
These blocks sit in 3 columns across the top of the page, and the engine reads them left to right: the brand, then the links beside it. Here the measured order agrees with the source order everywhere, which is the common case and not one you can count on — the measurement is what makes it a fact rather than an assumption. The method is geometric-anchored rather than a fully measured one because 1 block has no box at all — here the hero image, which the browser laid out with no measurable size — so it is anchored beside its neighbours in the source, and the page is told which it got.
It always says which it did: measured order, measured with some blocks anchored, or plain source order when there was no browser. A consumer is told the confidence rather than left to assume it.
- Stage 05blocks in orderone document
Merge the two fetches, losing nothing
Both fetches have been through the three stages above separately. Now they are one document. The rendered document leads, because its order is measured. A block only the plain fetch had is not thrown away and not guessed into place: it is inserted after the nearest block both documents contain, which is the same rule that anchors an unmeasured block within one document.
Two exceptions keep the merge honest. A plain-fetch block that the browser marked hidden is not something the render lost, it is something the render hid, and it stays out — php.net's manual table of contents is a hundred links under display: none that this merge used to put straight back. And a block under a client-side template directive the render never took is a branch the framework chose not to build.
The result reports what each side contributed, which turns “nothing was lost” from a promise into a measurement.
in: two ordered documentsout: one documentout: characters and blocks each fetch contributedWhat it looks likethe two fetches of lakshx.in, merged - strategy
- union
- plain fetch held
- 2,519 chars
- browser held
- 2,592 chars
- merged document
- 2,592 chars
- blocks only the browser had
- 3
- blocks only the plain fetch had
- 0
- blocks after merge
- 45
blocks only the browser hadDownload for macOS Upgrade to Pro Have a promo code?
The two fetches agree on the prose; what only the browser had is the interface the page mounts around it. Nothing here was lost either way — on a site whose plain fetch is a consent wall, the numbers run the other way and the plain fetch is the side holding the article.
When it goes wrong
- The plain fetch was a wall
- A login page, a challenge, a consent gate. The wall's text is left out and the result says what was refused.
- Both fetches were walls
- The page is refused with the wall's own evidence rather than extracted as an article about cookies.
- Stage 06one documentcontent view
Say what kind of page it is, then decide what counts as the content
A small gradient-boosted model reads 126 features of the document — its markup statistics, its declared URL, its landmarks, how its text is distributed — and names one of seven page types: article, documentation, service, forum, collection, listing, product. Below a confidence of 0.5 it says unknown rather than guess.
The type picks a policy, and the policy runs four steps in order. Drop what the page itself labels navigation and footer. Narrow to the main region when the page declares a trustworthy one. Remove the furniture this site repeats on every page, learned in the crawl. Then draw a boundary around the real content.
in: one documentout: the page type and its confidenceout: the content viewout: which steps firedWhat it looks likethe router and the content view on lakshx.in - page type
- service
- confidence 0.997
- blocks kept
- 34
- blocks dropped
- 11
- steps that fired
- landmarks, main-content
what the content view dropped, and which step dropped itdropped by region kind text the boundary main image (no text — a decorative image) the boundary main paragraph LakshX the boundary main paragraph Changelog Docs the boundary main paragraph ✦India’s #1 Agentic Coding IDE the boundary main paragraph Upgrade to Pro the boundary main paragraph Have a promo code? the page's own landmarks footer paragraph LakshX the page's own landmarks footer paragraph An agentic coding IDE — plans, edits, and runs commands across your r… the page's own landmarks nav paragraph Docs Changelog Terms of Service Privacy Policy Refund Policy the page's own landmarks footer paragraph © 2026 LakshX. All rights reserved. the page's own landmarks footer paragraph Made for developers who ship. Two different mechanisms. 5 of these sit inside the page’s own <footer> and <nav>, so the first step removes them without reading a word. The other 6 are inside <main> — this page puts its top bar there and declares no <header> at all — so no landmark marks them, and it is the last step, the boundary around the prose, that leaves them out: a brand, its two header links, a badge above the headline, and two buttons in the pricing card.
The full-page view still has all 45, which is the point of computing a view rather than editing the list.
This never destroys anything. The complete block list stays complete, and “the content” is a separate, narrower view computed from it — so the full page and the article are both available, and the reduction is always reversible.
- Stage 07content viewMarkdown
Write the blocks out as Markdown
Markdown is written from the blocks, not from the HTML. Each block becomes the Markdown its kind calls for — a heading of its level, a paragraph, a list item, an image with its alt text and address, a table with its grid, a fenced code block, an equation kept as TeX — in the reading order the blocks are already in, with the links and emphasis a block's rich text recorded.
Two documents can be written from the same blocks. The full page writes every block, footer and all. The content view writes only the blocks the previous stage kept. On the example page the difference is the header, the navigation and the footer.
This is the text a model reads when it is asked about the page, and the blocks are what its answer is checked against, so the two must say the same thing. They do by construction, because one is written from the other.
in: blocks in reading orderout: Markdown, full page or content viewout: plain textWhat it looks likethe sample blocks above, written out kind text paragraph ✦India’s #1 Agentic Coding IDE heading An agentic IDE, not just autocomplete. list-item The full IDE — every agent mode, checkpoints & undo, voice mode image paragraph [Changelog](https://lakshx.in/changelog) [Docs](https://lakshx.in/docs) Markdown✦India’s #1 Agentic Coding IDE # An agentic IDE, not just autocomplete. - The full IDE — every agent mode, checkpoints & undo, voice mode  [Changelog](https://lakshx.in/changelog) [Docs](https://lakshx.in/docs)
- full page
- 458 words
- 3,164 chars
- content view
- 402 words
- 2,550 chars
- the difference
- 56 words
- 11 blocks of header, nav, footer
Part two
A whole site
What a crawl adds around that: a queue, a loop, the furniture every page shares, and at the end two graphs -- one observed, one inferred.
- Stage 08one URLroot page
Look at the front door, once
The home page is fetched both ways, as above. In the same pass the crawl reads the site's robots file if asked to honour it, collects every sitemap that file advertises, and identifies the technology behind the site from 237 fingerprint rules.
in: one URLout: the root pageout: sitemap URLsout: technology profileWhen it goes wrong
- Site unreachable
- The crawl stops here and says why. Nothing else can proceed.
- Redirected elsewhere
- The new address becomes the root, and the redirect is reported rather than followed silently.
- No sitemap published
- Fine. The crawl discovers pages by following links instead.
- Stage 09root pagequeue
Build the queue, and clean every address going into it
Sitemap pages seed a queue. The unglamorous part matters most: four addresses differing only by a trailing slash, a fragment, and a tracking parameter are one page. A crawler that treats them as four spends its budget four times and puts four copies of every fact in your graph.
So it strips about two dozen tracking parameters, normalises ports and index filenames, and skips addresses that are clearly not pages. PDFs are deliberately kept, because a PDF is a document worth reading.
in: sitemap URLsout: a de-duplicated queue - Stage 10queuepage events
Crawl and extract at the same time
Discovery and extraction are interleaved, not sequential. Pages are taken from the queue in batches, every page goes through the seven stages above, and every page that comes back has its links read and added to the queue.
So the crawl reaches everything reachable rather than only what the sitemap listed, and the first result arrives in seconds instead of after a full enumeration.
Once a page's links have been read, its raw markup is discarded. On a long article that markup is the single largest thing in memory and nothing needs it again.
in: the queueout: one result event per page, as it finishesWhat it looks likewhat the crawl of lakshx.in emitted - stage
- 3
- analysis
- 1
- discovery
- 1
- frontier
- 1
- page
- 12
- fetching
- 11
- done
- 1
One event per page as it finishes, not one batch at the end. No page on this site was served under two addresses; where that happens the second is reported as an alias rather than extracted again.
When it goes wrong
- A page fails
- Recorded against that page. The crawl carries on. One bad page never stops a run.
- Two addresses serve one page
- The second is reported as an alias of the first, not extracted twice. A privacy-policy link that the server answers with the home page is one home page.
- Budget reached
- Stops cleanly at the page limit you set, or runs until the queue empties if you set none.
- Stage 11page eventsrepeated blocks
Learn what the site repeats
Once six or more pages are in, the engine compares them and finds the blocks that appear on all of them. That is the site's furniture: its header, its footer, its cookie notice, its sidebar. No single page can reveal this; it only shows up across a site. In a crawl this is the third of the four content steps above.
A guard refuses to call more than half of any page furniture. On a documentation site where every page genuinely resembles its neighbours, that rule is the difference between removing the navigation and removing the documentation.
in: six or more extracted pagesout: the blocks this site repeats everywhere - Stage 12page eventsobserved graph
Build the observed graph: what the site says about itself
No model is involved here. As each page arrives, its blocks are cut into sections — a heading owns everything beneath it until the next heading of equal or higher level, which is the author's own idea of where a topic starts and stops. This is only possible because reading order was recovered first: on a multi-column page, source order does not say which paragraph sits under which heading.
Every link between pages is recorded with the anchor text it was clicked through, and a link that appears on every page of a site is one edge carrying the number of times it was seen. Entities come only from what pages declare in JSON-LD or microdata, or from a subject several pages agree on by their anchors. Aliases join the addresses that turned out to be one page.
A section keeps the XPath and character span of every block it was joined from, so anything read out of a section can be traced back to a block.
in: every extracted pageout: pagesout: sections with block spansout: labelled linksout: declared entitiesout: aliasesWhat it looks likethe observed graph after 12 pages of lakshx.in - pages
- 12
- sections
- 111
- links
- 143
- declared entities
- 1
- from open-graph
- aliases
- 0
sections of the home page, in reading orderid level heading blocks chars lakshx.in/#s0 0 (before the first heading) 3 111 lakshx.in/#s1 1 An agentic IDE, not just autocomplete. 4 436 lakshx.in/#s2 2 An agent you can actually trust with your codebase 1 181 lakshx.in/#s3 3 Pick your autonomy level 1 242 lakshx.in/#s4 3 Every edit is reversible 1 203 lakshx.in/#s5 3 Any provider, or the free hosted model 1 184 entities the pages declared about themselves — no model involvedtype name pages open-graph LakshX — India's #1 Agentic Coding IDE 12 links, with the anchor text they were clicked throughfrom to anchors seen lakshx.in/docs lakshx.in/docs/installation Installation · Install it · Next Installation 3 lakshx.in/docs lakshx.in/docs/sign-in Sign In & Models · sign in or add a model · configure your own provider key 3 lakshx.in/docs lakshx.in/docs/chat The Chat Panel · chat panel · The Chat PanelTalk to the agent, attach files, @-mention, and steer a run. 3 lakshx.in/docs lakshx.in/docs/modes Agent Modes · safety modes · Agent ModesReview, Approve, Auto, and Royal — how much you let the agent do. 3 lakshx.in/changelog lakshx.in/ Home · Download · home 3 lakshx.in/terms lakshx.in/privacy Privacy Policy 3 - Stage 13observed graphinferred graph
Build the inferred graph: what a model reads out of each section
Off by default; on with WEBGRAPH_KG=1 and a provider key. A model is given one section at a time — its page, its heading path, the entities already known on the page, and the section's blocks numbered [b0], [b1], … — and asked to state as data what the section says: entities with a type, their attributes, and relations between them. Every one of those must carry the block it came from and a quote copied verbatim from that block.
The model's answer is not trusted. Each quote is located in the block it names — and if not there, in the section's other blocks, because models mis-number more often than they invent. A quote found becomes evidence: page, section, block XPath, character span. A quote not found is discarded, and an entity or relation with no evidence left goes with it. The counts of what was refused are kept.
Across pages, entities are merged by type and normalised name, by the aliases the model listed, and by near-duplicate names. The result exports to Neo4j and answers questions by walking a few hops and then reading the sections the walk reached.
in: one section at a timeout: entities, attributes, relationsout: evidence for every oneout: what was refusedWhat it looks likeone section of lakshx.in, followed through The section “Free” — 7 blocks, 355 characters. The same characters at every step: the elements, the blocks, the Markdown, the model’s numbered input, and the evidence a verified fact points at.
three of its elements<p class="mt-1 text-sm text-ink-navy/50" data-wg-id="152" data-wg-brk="1">For BYOK users, and anyone getting started</p> <div class="mt-6 flex items-baseline gap-1" data-wg-id="153" data-wg-brk="1"><span class="font-heading text-4xl font-bold tracking-tight text-ink-navy" data-wg-id="154" data-wg-brk="1">$0</span><span class="text-sm text-ink-navy/50" data-wg-id="155" data-wg-brk="1">forever</span></div> <li class="flex items-start gap-2.5 text-sm text-ink-navy/70" data-wg-id="157" data-wg-brk="1"><svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewbox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" class="lucide lucide-ch…
the same three blocks (text, address, box){ "text": "For BYOK users, and anyone getting started", "xpath": "/html/body/main/div[2]/section[2]/div/div[2]/div[1]/p", "rect": { "x": 289, "y": 2456, "w": 378, "h": 20 } } { "text": "$0 forever", "xpath": "/html/body/main/div[2]/section[2]/div/div[2]/div[1]/div", "rect": { "x": 289, "y": 2500, "w": 378, "h": 40 } } { "text": "The full IDE — every agent mode, checkpoints & undo, voice mode", "xpath": "/html/body/main/div[2]/section[2]/div/div[2]/div[1]/ul/li[1]", "rect": { "x": 289, "y": 2572, "w": 378, "h": 40 } }the section as MarkdownFor BYOK users, and anyone getting started \$0 forever - The full IDE — every agent mode, checkpoints & undo, voice mode - Bring your own API key from any supported provider — fully unlimited, forever - Or sign in free with Google for the hosted LakshX model, up to a starter trial amount - No credit card required [Download for free](https://lakshx.in/#top)
what the model is told (start of the rules)You read one section of a web page and state, as data, what it says. Rules: 1. Only what the section states. Nothing from your own knowledge, nothing implied. 2. Every mention, attribute and relation carries a `block` (the [bN] marker) and a `quote` copied VERBATIM from that block: same words, same spelling, same punctuation, 3 to 40 words. Anything whose quote is not found verbatim in the block is discarded, so copy, never paraphrase. 3. Entity types: use one of Organization, Person, Product, Service, Course, Event, Place, Document, Offer, Topic, ContactPoint, Role when it fits; otherwise the most specific PascalCase noun. Prices, dates, durations, counts, phone numbers and…
what the model is given (start of the input)Page: LakshX — India's #1 Agentic Coding IDE URL: https://lakshx.in/ Heading path: Free Known entities on this page (use these names exactly): - (none) Section blocks: [b0] For BYOK users, and anyone getting started [b1] $0 forever [b2] The full IDE — every agent mode, checkpoints & undo, voice mode [b3] Bring your own API key from any supported provider — fully unlimited, forever [b4] Or sign in free with Google for the hosted LakshX model, up to a starter trial amount [b5] No credit card required [b6] [Download for free](https://lakshx.in/#top) Return the JSON object.
the model's answer — written by hand for this walkthrough; the verification below is the real engine{ "entities": [ { "name": "LakshX Free", "type": "Offer", "aliases": [ "Free" ], "attributes": [ { "key": "price", "value": "$0 forever", "block": "b1", "quote": "$0 forever" } ], "mentions": [ { "block": "b3", "quote": "Bring your own API key from any supported provider" } ] }, { "name": "hosted LakshX model", "type": "Product", "aliases": [], "attributes": [], "mentions": [ { "block": "b4", "quote": "sign in free with Google for the hosted LakshX model" } ] }, { "name": "Enterprise plan", "type": "Offer", "aliases": [], "attributes": [], "mentions": [ { "block": "b5", "quote": "includes a dedicated support engineer" } ] } ], "relations": [ { "subject": "LakshX Free", "predicate": "includes", "object": "hosted LakshX model", "fact": "The free plan can use the hosted LakshX model after a free Google sign-in, up to a starter trial amount.", "evidence": [ { "block": "b4", "quote": "sign in free with Google for the hosted LakshX model, up to a starter trial amount" } ] } ] }accepted: LakshX Free (Offer), with an attribute{ "mention": { "quote": "Bring your own API key from any supported provider", "block_xpath": "/html/body/main/div[2]/section[2]/div/div[2]/div[1]/ul/li[2]", "span": [0, 50], "section_id": "https://lakshx.in/#s8" }, "attributes": [ { "key": "price", "value": "$0 forever", "block_xpath": "…/div[2]/div[1]/div", "span": [0, 10], "quote": "$0 forever" } ] }accepted: LakshX Free → includes → hosted LakshX model, 1 piece of evidence{ "fact": "The free plan can use the hosted LakshX model after a free Google sign-in, up to a starter trial amount.", "evidence": [ { "block_xpath": "…/div[1]/ul/li[3]", "span": [3, 85] } ] }refused{ "quote_not_found": 1, "entity_without_evidence": 1 }2 of the 3 entities kept their evidence: page, section, block XPath, and the character span inside the block. AAn attribute is a fact about a thing and is verified the same way — the one above points at the characters $0 forever in the block that holds them. The remaining quote was invented, so “Enterprise plan” has nothing to stand on and is not in the graph; the counts beside it are what the engine keeps of that.
Markdown is what the model reads; blocks are what its answer is checked against. A fact in this graph is never a sentence the model wrote — it is a quote the page contains, with an address a reader can open.
- Stage 14inferred graphresults
Hand it back
Per page: plain text, structure-preserving Markdown in both views, the typed blocks, the page type, which reduction steps fired, and what each fetch contributed.
Per site: the observed graph, the inferred graph when it was built, and the aliases and failures recorded along the way.
in: every extracted pageout: text and Markdownout: blocksout: the two graphsOne last check across the whole crawl
- Three or more pages identical
- A warning. If many different addresses return the same text, the crawl is not extracting a site — it is extracting one gate, over and over. Two identical pages is a coincidence; three is a pattern.
The three ideas everything else follows from
Lose nothing first, narrow afterwards
Every stage that removes something produces a new view rather than editing the original. That is why a mistake in content selection costs you a worse summary and never costs you the page.
A failure is a recorded fact, not an exception
An unreachable host, a refused page, a browser that would not start — each becomes a value attached to that page while the crawl continues. Across thousands of pages, anything else means one bad page ends the run.
Everything points back to a block
A Markdown line, a section, a link label, a fact a model read out: each keeps the XPath of the block it came from. Nothing the engine says about a page is more than one step from the element that says it.
Where the engine places against every public benchmark, including the ones it loses, is on the benchmarks page.