SICSS 2026 Synthetic Shoppers Challenge · Public research contest

# Participant code guide — `sicss2026/`

Short map of the Python tools for the Synthetic Shoppers challenge. You do not need Python or these CLIs — see challenge-api-contract.html for a language-agnostic description of inputs, outputs, and HTTP APIs.

Fastest Python path: install → edit prompts → run `shop-vl-predict` (or `--mode text`) → `shop-evaluate` on train → submit test JSONL.

The download kit includes `sicss2026/` + `pyproject.toml` — no need to clone the full instructor repo.

---

## Choose your inference setup

You can run inference with your own Hugging Face or OpenAI account (BYOK).

Use your Hugging Face or OpenAI account — workshop quotas do not apply; you pay the provider directly.

    export HF_TOKEN=hf_...
    export INFERENCE_BASE_URL=https://router.huggingface.co/v1
    export OPENAI_API_KEY=...   # alternative to HF_TOKEN

Pass any model id your provider supports, e.g.:

    poetry run shop-vl-predict batch \
      --mode vision \
      --model meta-llama/Llama-3.2-11B-Vision-Instruct \
      --input data/public/synthetic_shoppers_test.csv \
      --output preds.jsonl

If you prefer, you can also generate predictions without any API at all (local model, rules, spreadsheet) — as long as your submission file matches the contract in `docs/challenge-api-contract.md`.

---

## Install

Requirements: Python 3.11+.

From the kit root (`pyproject.toml`, `sicss2026/`, `data/`, …):

    poetry install
    # or: python3 -m venv .venv && source .venv/bin/activate && pip install -e .

Command | Purpose
--- | ---
`shop-vl-predict` | Predictions via vision or text (`--mode`)
`shop-evaluate` | Local training self-check
`shop-cua-scenario` | Optional browser agent demo (advanced)

`shop-evaluate` needs no API key.

---

## Quick start

### Vision (screenshot + VL model)

    poetry run shop-vl-predict batch \
      --mode vision \
      --input data/public/synthetic_shoppers_test.csv \
      --output test_predictions.jsonl

### Text-only (HTML page text + text model)

    poetry run shop-vl-predict batch \
      --mode text \
      --input data/public/synthetic_shoppers_test.csv \
      --output test_predictions.jsonl

Text mode reads `scenarios/R00xx.html`, strips tags to plain text, and sends that to the text model.

### Self-check on training labels

    poetry run shop-evaluate -p my_train_preds.jsonl

---

## Module map

File | Role
--- | ---
`vl_predict.py` | CLI: vision or text prediction, workshop or BYOK
`html_text.py` | HTML → plain text for `--mode text`
`kit_paths.py` | Default paths for kit vs full repo
`evaluate_predictions.py` | `shop-evaluate` CLI
`evaluation.py` | Shared metrics (MAE, Brier, persona MAE, design-effect)
`prediction_parser.py` | Parse model output → probability + reason

---

## `shop-vl-predict` details

Vision flow: persona prompt + PNG screenshot → chat completion → JSON prediction.

Text flow: persona prompt + extracted page text from `scenarios/R00xx.html` → chat completion → JSON prediction.

Flag | Default
--- | ---
`--mode` | `vision`
`--model` | Qwen3-VL (vision) or Qwen3-30B (text)
`--screenshots-dir` | `screenshots/` in kit
`--scenarios-dir` | `scenarios/` in kit
`--persona-prompts` | `exercises/prompts/`

---

## Not using Python?

Read challenge-api-contract.html:

- Required submission fields
- OpenAI-compatible inference examples (curl)
- Training eval and submission endpoints
- BYOK setup

Build in R, Julia, JavaScript, or a notebook — as long as you upload a valid 38-row file.

---

## Persona prompts (`exercises/prompts/`)

- `value_optimizer.txt`
- `trust_first_verifier.txt`
- `convenience_need_solver.txt`
- `inspiration_impulse_explorer.txt`

Rewrite these so each segment matches training behavior across categories and Design A/B.

---

## Suggested workflow

1. Install (or use your own stack per API contract)

2. Explore training CSV + HTML/screenshots per persona

3. Edit the four prompt files

4. Predict on train → `shop-evaluate` → iterate

5. Predict on test → submit at https://sicss2026.derlem.com/submit

6. Check https://sicss2026.derlem.com/ranking

---

## Optional: CUA browser agent

`shop-cua-scenario` is an optional demo for browser-driven agents. If you want a ready-to-use UI, see the upstream CUA Space: https://huggingface.co/spaces/smolagents/computer-use-agent