6 min read

AI engine

packages/ai (@commerce/ai) is the one owner of AI in the platform. Features never import the Vercel AI SDK  directly; they go through a small interface:

The SDK is Apache-2.0 (credited in THIRD_PARTY_NOTICES.md). The root entry is server only (the client-imports-server sweep lists it): it reads the platform keys and pulls in the SDK.

Providers and keys

AI runs on one platform key; merchants never bring their own. Set one of:

AI_PROVIDER picks explicitly (gateway, openrouter, off, or mock); unset, the gateway whose key is set wins, Vercel first. Both gateways take the same provider/model ids, so one model id works on either. No key means AI is off: ai.status answers configured: false and the admin hides every AI button.

Model routing (one config)

Every call goes to one of two tiers in MODEL_TIERS (packages/ai/src/features.ts):

tierFor(feature, quality) decides; features marked writes: true move to strong at “best”. A strong call costs about three times as much, so it counts as TIER_CALL_WEIGHT.strong (3) requests against the monthly allowance. The quality is a column on ai_settings (writing_quality, migration 0086).

AI_PROVIDER=mock swaps in a built-in fake model: canned product copy, and for the assistant a deterministic planner (packages/ai/src/mock-plan.ts) that turns a message into tool calls, so “Create discount DIWALI10: 10% off collection Mice, 20 Oct to 5 Nov” really becomes that proposal, and “Raise prices 5% in collection Mice”, “Add 8 products to new collection Glasspads”, “Write this product in Hindi” (on a product page) and “Draft a reply to order #2250” each produce their card. A purchase document read (documentReader) answers a fixed supplier invoice, Bill of Entry or service bill (packages/ai/src/purchase-documents/fixtures.ts). Use it for local demos and screenshots with no key. It is refused when NODE_ENV=production.

Cost on the fast tier (Claude Haiku 5.5, $0.10 in / $0.50 out per million tokens for prompts up to 100K tokens): a product write is roughly 5k tokens in and 800 out (about $0.0009, ₹0.08); an assistant message roughly 9k in over two to four steps and 500 out (about $0.0012, ₹0.1).

How one call runs

Every AI procedure goes through packages/api/src/services/ai/guard.ts (the feature, plan and quota part is core’s admitAiCall in @commerce/core/ai-runtime, shared with the worker’s bulk runs):

1. canDetached(permission) checks membership and the permission in a short withStore transaction, then runs the handler outside a transaction. A model call takes seconds; a streamed answer runs after the middleware returns. Neither may hold a Postgres connection idle in a transaction, so every read opens its own withStore(context.db, storeId, ...).
2. startAiCall(context, feature):
   • AI configured? (aiModel(feature)), else PRECONDITION_FAILED / ai_off
   • per-store burst limit, 20 calls per 5 minutes across all AI features, on the shared rate-limit store, else TOO_MANY_REQUESTS / rate_limit
   • feature on for the store (ai_settings.features), else PRECONDITION_FAILED / feature_off
   • monthly quota for the store’s plan (ai_usage summed for the IST month), else TOO_MANY_REQUESTS / quota. Stores in AI_UNLIMITED_STORES (slugs) are on unlimited
   • counts the call (calls + TIER_CALL_WEIGHT[tier]) and returns the AiRuntime (store, member, plan, feature)
3. The engine runs with that runtime. Tools get the runtime from the engine, never from the model’s input, so a tool always reads the caller’s store under RLS.
4. finishAiCall adds the reported tokens. For streams it runs from onUsage, which fires even when the browser goes away mid-answer.

Usage and settings live in two tenant tables (migration 0083_ai_engine): ai_settings (plan, feature switches) and ai_usage (store, month, feature, calls, input and output tokens). No prompt or answer is ever stored or logged.

Plans

The merchant API never changes the plan; billing (or a platform admin) sets ai_settings.plan.

Approvals for risky tools

A tool with risk: 'write' is never run by the model. streamAnswer passes it to the SDK’s toolApproval as user-approval: the stream yields an approval event (tool name and input) and stops.

A tool with risk: 'propose' runs, but changes nothing in the store: it validates the model’s input and stores a proposal. The AI assistant’s write tools are all propose tools; the change itself runs only from the merchant’s Approve, outside the model (see AI assistant).

What data goes to the model

• Write with AI: the product’s title, type, brand, existing tags and description, the merchant’s notes, and up to three of the product’s photos (the 640px WebP variant from the image pipeline when there is one). Photos are read from the store’s own storage by media id under RLS, so another store’s id simply finds nothing.
• AI assistant: the message, the last four turns of the conversation (with one line per change card), one line about the page (resolved under RLS from the record id), and tool results. Tools return aggregates, product and order facts; the only personal detail is an order’s customer first name (orderDetails, for reply drafts), and the answer says so. Customer notes go through with phone numbers and emails hidden.
• Bulk Write with AI: the same as Write with AI, per product, from the worker.
• Suggest keywords (ai.suggestKeywords, and the keywords field of a bulk run): the product’s title, brand, type, category breadcrumb, description, price range, its current product.seo_keywords and, when a source is registered with setSearchQueriesSource (packages/core/src/seo/search-queries.ts, the Search Console connection), up to 20 queries that brought visitors to /products/<handle>. Runs under the productWriter switch (bulk: bulkWriter). The chosen keywords (primary first) then go to Write with AI, which puts the primary one in the search title and once in the search description. The storefront never prints them (no meta keywords: search engines ignore it).

Add an AI feature (one slice)

Say you are adding “Reply to a review”.

1. Feature and model: add 'reviewReply' to AI_FEATURES and a row to FEATURE_MODELS (packages/ai/src/features.ts). Missing switches default to on, so no migration.

2. Prompt and schema: packages/ai/src/review-reply/schema.ts (zod output, client-safe) and prompt.ts (instructions, content builder). Export them from client.ts / index.ts. Treat merchant text as data, never instructions.

3. Service: packages/api/src/services/ai/review-reply.ts:

   export async function replyToReview(context: AiCallContext, input: Input) {
     const call = await startAiCall(context, 'reviewReply')
     const review = await withStore(context.db, call.runtime.storeId, (tx) => loadReview(tx, input.id))
     const { output, usage } = await generateStructured({
       runtime: call.runtime,
       model: call.model,
       schema: ReviewReplySchema,
       instructions: reviewReplyInstructions(),
       content: reviewReplyContent(review),
     })
     await finishAiCall(context.db, call, usage)
     return output
   }

   For a streamed, tool-using feature copy assistant/assistant.ts: streamAnswer plus tools built with defineAiTool, each opening its own withStore for runtime.storeId. To give the assistant a new tool instead of a new feature, see AI assistant.

4. Procedure: add it to packages/api/src/routers/ai.ts with canDetached(permissions.X), and give ai.status a boolean for the button (map the feature to its permission in settings.service.ts). Add the procedure to the cross-store leak test’s SKIP list (it reaches the model provider).

5. Admin: copy in apps/admin/src/data/ai.ts plus the Hindi overlay in data/hi/ai.ts; show the button only when ai.status says so. Suggest, never save: the merchant applies.

6. Settings → AI: the new switch appears from AI_FEATURES once its label and help are in aiContent.settings.features.

7. Tests: engine-level with textModel / scriptedModel; API-level with AI_PROVIDER=mock against the test database (packages/api/test/ai.test.ts shows quota, permission, switch and tool tests). Never call a real provider in a test.

8. Docs: a line in the merchant guide’s “AI features” page.

Local testing

# .env (root), no key needed:
AI_PROVIDER=mock
# or a real gateway:
AI_GATEWAY_API_KEY=...        # or OPENROUTER_API_KEY=...

Restart the admin, then press ⌘J / Ctrl+J on any page (the AI bar), open a product (Write with AI on the Basic details card), select products on the list (Write with AI in the selection bar; run the worker so the run is picked up), and Settings → AI for usage, switches, writing quality and Activity.