POST /api/extract

POST /api/extract maps a page's structured data - JSON-LD, microdata, Open Graph, __NEXT_DATA__, RSC flight data, Nuxt and __INITIAL_STATE__ payloads - onto a JSON Schema and returns one fact per matched path. Every fact says where it came from and how confident the engine is; a fact without a source cannot be checked by whoever consumes it.

Request

{
  "url": "https://shop.example.com/p/desk-lamp",
  "schema": {
    "type": "object",
    "properties": {
      "name": { "type": "string" },
      "offers": {
        "type": "object",
        "properties": {
          "price": { "type": "number" },
          "priceCurrency": { "type": "string" }
        }
      }
    }
  },
  "render": false,
  "rtl": false
}

ExtractRequest has no html, fetch, renderOptions or include_hidden_text: this route always fetches, with the engine's defaults. A schema that is not an object or lacks properties is 422 with schema must be a JSON Schema object with 'properties'.

Matching

Keys are matched cheapest and most certain first: exact key, then an alias you declared with the x-webgraph-aliases schema extension, then a normalised key (priceAmount, price_amount and Price Amount all match price_amount), then a known schema.org alias, then a bounded descent into nested objects, nearest match first. Confidence falls with each step because each is a weaker claim about what the page meant.

Response

{
  "page": { "url": "https://shop.example.com/p/desk-lamp", "reading_order": "dom-fallback", "reading_order_measured": false, "blocks": 212, "frameworks": ["next"], "requires_render": false, "payloads": ["json-ld", "open-graph", "next-data"], "content_hash": "…", "dom_order_differs": false },
  "facts": {
    "name": { "value": "Arc desk lamp", "confidence": 0.95, "extractor": "structured-data", "modality": "dom-json", "source": "json-ld:exact", "source_xpath": "/html/head/script[3]" },
    "offers.price": { "value": 89.0, "confidence": 0.9215, "extractor": "structured-data", "modality": "dom-json", "source": "json-ld:nested", "source_xpath": "/html/head/script[3]" },
    "offers.priceCurrency": { "value": "EUR", "confidence": 0.9215, "extractor": "structured-data", "modality": "dom-json", "source": "json-ld:nested", "source_xpath": "/html/head/script[3]" }
  },
  "schema_choice": null
}

page is the same PageInfo the text route returns. facts is keyed by dotted schema path, sorted; array items appear as plans.0.price.

FactOut

No schema: the engine chooses

With schema omitted the router types the page and the engine uses the built-in schema for that type - article, product, forum, service, collection/listing, documentation. It then reads only the structured-data node that describes the page: not the site's Organization, not its breadcrumbs. Without that gate a category page reports the shop's name as its own product 46% of the time. Fields no node about the page supplied are filled, at a discount, from the page's WebPage wrapper and then from its Open Graph tags, and each such fact's source says so.

schema_choice explains the choice:

The gate runs only on an auto-chosen schema. A caller who wrote a schema may be reaching for the Organization on purpose.

Refusing to guess

Nothing here infers semantics. If no key matches a path, no fact is emitted; a missing value is recoverable downstream and a wrong one silently poisons everything built on it. Likewise, a page the router types as unknown gets an empty schema and no facts rather than a vocabulary picked at random, and schema_choice.fields is then []. "No price" therefore means different things depending on schema_choice.page_type, which is why it is returned.

JavaScript shells

A page whose static HTML is a shell - markup with no readable text until a script runs - is a refusal for /api/text, which would have nothing to say. It is a document for this route: the hydration payload (__NEXT_DATA__, a flight stream, JSON-LD) is complete in the shell and is the whole point of the page. With render: false the route tries a browser once if Playwright is installed; if the page stays empty, the facts are read from the shell's payload and page.blocks may be 0.

curl -s http://127.0.0.1:8000/api/extract \
  -H 'Content-Type: application/json' \
  -d '{"url": "https://shop.example.com/p/desk-lamp",
       "schema": {"type": "object", "properties": {"name": {"type": "string"}}}}'