{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ade733f1",
   "metadata": {},
   "source": [
    "# Extended follow-up experiments (wave 2)\n",
    "\n",
    "Secondary RQs answering open questions from the geo / CA / Q27–Q28 / covariate memos.\n",
    "\n",
    "**CLI equivalent:** `ca-personas followup-experiments --join inner --seed 42`\n",
    "\n",
    "See [`docs/secondary_rq_followup_experiments.md`](../docs/secondary_rq_followup_experiments.md) and [`docs/research_memo_agenda.md`](../docs/research_memo_agenda.md).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a28a800d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import pandas as pd\n",
    "from ca_personas.followup_experiments import run_followup_experiments_pipeline\n",
    "\n",
    "artifacts = run_followup_experiments_pipeline(\n",
    "    join_how=\"inner\",\n",
    "    output_dir=Path(\"outputs/followup_experiments\"),\n",
    "    figures_dir=Path(\"memos/figures\"),\n",
    "    random_state=42,\n",
    ")\n",
    "overview = pd.read_csv(artifacts[\"overview\"])\n",
    "overview"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bfc584e",
   "metadata": {},
   "source": [
    "## Per-experiment results cards"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50591bbe",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "from IPython.display import display, Markdown\n",
    "\n",
    "root = Path(\"outputs/followup_experiments\")\n",
    "for key in overview[\"spec_key\"]:\n",
    "    card = json.loads((root / key / f\"{key}_results_card.json\").read_text())\n",
    "    auc = (card.get(\"cv_metrics\") or {}).get(\"roc_auc\")\n",
    "    display(Markdown(f\"### `{key}` — n={card.get('sample_n')}, AUC={auc:.3f}\" if auc else f\"### `{key}`\"))\n",
    "    if card.get(\"summary_delta\"):\n",
    "        display(pd.Series(card[\"summary_delta\"]).to_frame(\"value\"))\n",
    "    comp = root / key / f\"{key}_comparison.csv\"\n",
    "    if comp.exists():\n",
    "        display(pd.read_csv(comp))\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}