Here is a cleaner version you can give to students.

# SICSS 2026 Synthetic Shoppers Challenge

## Overview

You are working with data from a simulated e-commerce interface experiment.

The site contains one representative product from each of the following categories:

* electronics
* fashion
* home
* grocery
* cosmetics
* books
* sports
* baby products

Each product scenario is shown in two product-detail-page designs:

* **Design A:** a simple product page with minimal product information
* **Design B:** a richer product page with more information, social proof, delivery/return details, and urgency cues

The company has segmented its users into four behavioral groups. However, you are given only the short segment labels and labeled training examples. You are **not** given the full behavioral definitions of the segments.

Your goal is to build prompt-based shopping agents that mimic these segments as well as possible.

**Dataset:** **128 scenarios** total — **90 labeled training** rows and **38 unlabeled test** rows you must predict and submit.

## Your Task

Build one prompt-based agent for each user segment.

Each agent should inspect the product page (HTML in the browser, or the provided screenshot) as if it were a shopper from that segment and estimate the probability that the shopper would purchase the product.

For each test scenario, your agent should output:

```json
{
  "record_id": "R0001",
  "persona_id": "trust_first_verifier",
  "purchase_probability": 0.72,
  "reason": "The product has strong review and return-policy signals, which make the purchase feel relatively safe for this segment."
}
```

The output should include:

1. `record_id`
2. `persona_id` (stable segment key, e.g. `trust_first_verifier`)
3. `purchase_probability`, a number between 0 and 1
4. `reason`, a short explanation grounded in the page content and the inferred segment behavior

## Data Provided

You will receive:

1. A labeled training file (`synthetic_shoppers_train.csv`, **90 rows**)
   This includes examples of how each segment behaved in previous product-page scenarios.

2. An unlabeled test file (`synthetic_shoppers_test.csv`, **38 rows**)
   This includes the scenarios for which you must submit purchase-probability predictions.

3. HTML pages and PNG screenshots for each scenario
   Each row has `html_path` and `screenshot_path` on the live mock shop. The recommended VL tool sends the **screenshot** plus your persona prompt to the model.

The training and test files include only high-level identifiers such as:

```text
record_id
persona_id
persona_name
category
design_variant
html_path
screenshot_path
```

Use `persona_id` in submissions. The `persona_name` column is a human-readable label only (e.g. `Trust-First Verifier`).

Live mock shop: https://sicss2026.derlem.com — open `html_path` for each row (e.g. `/scenarios/R0001.html`).

**Participant kit (data + code):** [download zip](https://sicss2026.derlem.com/downloads/latest) — includes CSVs, scenario HTML, screenshots, the `sicss2026/` Python tools with `pyproject.toml`, the [code guide](https://sicss2026.derlem.com/docs/participant-code-guide.html), and the language-agnostic [API contract](https://sicss2026.derlem.com/docs/challenge-api-contract.html).

You may use **any tools or models** you prefer. The bundled Python CLIs are optional conveniences; see the API contract for curl examples and BYOK setup.

## Background & contact

This site was originally built for a **2-hour SICSS Istanbul 2026 guest session** held at **Boğaziçi University** on **11 July 2026**. It is now left open for researchers who want to work on this problem.

SICSS Istanbul 2026: https://sicss.io/2026/istanbul/

Contact: onurgu@pt.bogazici.edu.tr · onurgu@boun.edu.tr

## Tools (optional)

**Python helpers (in the kit):**

```bash
# Vision — default
poetry run shop-vl-predict batch \
  --mode vision \
  --input data/public/synthetic_shoppers_test.csv \
  --output test_predictions.jsonl

# Text-only — reads scenario HTML as plain text
poetry run shop-vl-predict batch \
  --mode text \
  --input data/public/synthetic_shoppers_test.csv \
  --output test_predictions.jsonl
```

**Inference (BYOK):** set `HF_TOKEN` or `OPENAI_API_KEY`, and point `INFERENCE_BASE_URL` at your provider (e.g. `https://router.huggingface.co/v1`). Any model your account supports works. Details: [API contract](https://sicss2026.derlem.com/docs/challenge-api-contract.html).

Starter persona prompts: `exercises/prompts/*.txt`.

**Optional (browser agent UI):** https://huggingface.co/spaces/smolagents/computer-use-agent

## Important Constraint

Your agent must ground its decision in information visible on the product page (HTML or screenshot).

It should not invent product facts, assume missing discounts, or rely only on generic category stereotypes. The best agents will infer the segment behavior from the labeled training examples and then apply that inferred behavior consistently to new pages.

## Goal

Your goal is not to produce a binary buy / do-not-buy decision.

Your goal is to estimate a calibrated purchase probability.

For example:

```text
0.10 = very unlikely to purchase
0.50 = uncertain / could go either way
0.90 = very likely to purchase
```

## Evaluation

Your submissions will be evaluated against hidden purchase probabilities generated by the experiment.

The main evaluation criterion is how close your predicted probabilities are to the hidden probabilities.

We will use metrics such as:

* **Mean Absolute Error:** how far your probabilities are from the hidden probabilities on average
* **Brier Score:** how well-calibrated your probabilities are
* **Persona-level error:** whether your agent captures the average behavior of each segment
* **Design-effect error:** whether your agent captures how each segment responds differently to Design A and Design B

The most important question is not only whether your predictions are accurate, but whether your agents reproduce meaningful behavioral differences between segments and between interface designs.

The public **leaderboard** (https://sicss2026.derlem.com/ranking) ranks submissions by **overall MAE** (lower is better) and shows **per-persona MAE** (Value, Trust, Convenience, Inspiration), plus **Brier score** and **design-effect MAE**.

## Suggested Workflow

### 1. Explore the training examples

Look at how each segment behaves across:

* product categories
* Design A vs. Design B
* price, discount, reviews, delivery, return, trust, and urgency cues visible in the HTML pages

Try to infer what each short segment label means behaviorally.

### 2. Write one prompt per segment

For each segment, write a prompt that describes how that shopper should think and decide.

The prompt should be specific enough to guide behavior, but general enough to work across categories and page designs.

### 3. Test your agents on training examples

Run your agents on the training scenarios and compare their predicted probabilities with the provided labels.

Revise your prompts based on the errors.

### 4. Predict and submit the test scenarios

Run your final agents on all **38** unlabeled test scenarios.

Submit one purchase probability and one short reason for **every** test `record_id`.

1. Go to **https://sicss2026.derlem.com/submit**
2. Choose your predictions file (`.jsonl`, `.json`, or `.csv`)
3. Enter your **display name** (shown on the leaderboard)
4. Click **Submit file**

Your file must include all 38 test rows. Submitting again **replaces** your previous entry (only your latest submission counts).

Self-check on training rows locally before submitting:

```bash
poetry run shop-evaluate --predictions my_train_preds.jsonl
```

After submitting, scores appear on the **leaderboard**: https://sicss2026.derlem.com/ranking

### 5. Reflect

After submitting your predictions, consider:

* Which segment was easiest to model?
* Which segment was hardest?
* Did Design B increase purchase probability for all segments, or only some?
* Did your agent actually inspect the page, or did it rely too much on generic assumptions?
* Are these agents useful behavioral models, or are they just stereotypes?

## Submission Format

Submit a **JSONL**, **JSON** (array of objects), or **CSV** file with one row per test scenario (**38 rows**).

Required fields:

```text
record_id
persona_id
purchase_probability
reason
```

Example CSV:

```csv
record_id,persona_id,purchase_probability,reason
R0001,trust_first_verifier,0.72,"Strong reviews and a visible return policy increase confidence for this segment."
R0002,value_optimizer,0.34,"The discount is modest and the price remains relatively high, so this segment is unlikely to purchase."
```
