{ "cells": [ { "cell_type": "markdown", "id": "590963d6", "metadata": {}, "source": [ "# Geo-Memo Follow-Ups: Which Covariates Predict Regular Transit?\n", "\n", "### Head-to-head Random Forests for car access, employment, and ride-share\n", "\n", "**Course:** PSYCH 755 · University of Wisconsin–Madison \n", "**Project:** CA persona / PRCA research framework \n", "\n", "This notebook tests the candidate predictors named in [`memos/geo_predicts_transit.md`](../memos/geo_predicts_transit.md):\n", "\n", "1. Car license / access (`Q20` / `Q21`)\n", "2. Employment status\n", "3. Ride-share frequency (`Q28` / `Q29`)\n", "4. Joint mobility bundle (all of the above)\n", "\n", "Benchmarks from companion memos: **geo AUC ≈ 0.551**, **CA AUC ≈ 0.590**, chance = 0.500.\n", "\n", "Companion memo: [`memos/transit_covariate_followups.md`](../memos/transit_covariate_followups.md) \n", "CLI: `ca-personas covariate-transit-rf --join inner --seed 42`\n" ] }, { "cell_type": "markdown", "id": "0d737b1c", "metadata": {}, "source": [ "## 1. Setup\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "6fb9eed7", "metadata": { "execution": { "iopub.execute_input": "2026-07-25T15:40:50.204144Z", "iopub.status.busy": "2026-07-25T15:40:50.204011Z", "iopub.status.idle": "2026-07-25T15:40:51.208836Z", "shell.execute_reply": "2026-07-25T15:40:51.207900Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Data source: ../sibling_data File A/B/C\n", "Prolific: ['/tmp/sibling_data/PRCAProlificExport_FileA.csv', '/tmp/sibling_data/PRCAProlificExport_FileB.csv']\n", "Qualtrics: /tmp/sibling_data/PRCAQualtricsExport_FileC.csv\n" ] } ], "source": [ "from __future__ import annotations\n", "\n", "from pathlib import Path\n", "import sys\n", "\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import pandas as pd\n", "\n", "ROOT = Path.cwd().resolve()\n", "if ROOT.name == \"notebooks\":\n", " ROOT = ROOT.parent\n", "if str(ROOT / \"src\") not in sys.path:\n", " sys.path.insert(0, str(ROOT / \"src\"))\n", "\n", "from ca_personas.load import load_full_cohort\n", "from ca_personas.paths import default_prolific_paths, default_qualtrics_path, sibling_data_available\n", "from ca_personas.transit_covariate_rf import (\n", " FEATURE_SPECS,\n", " plot_comparison_memo_figure,\n", " plot_family_memo_figure,\n", " run_all_followup_analyses,\n", " run_feature_family_analysis,\n", " save_feature_family_artifacts,\n", " save_followup_bundle,\n", ")\n", "\n", "plt.rcParams.update({\"axes.grid\": True, \"grid.alpha\": 0.25})\n", "plt.rcParams[\"figure.dpi\"] = 120\n", "\n", "if sibling_data_available():\n", " PROLIFIC = default_prolific_paths()\n", " QUALTRICS = default_qualtrics_path()\n", " source = \"../sibling_data File A/B/C\"\n", "else:\n", " staged = Path(\"/tmp/sibling_data\")\n", " PROLIFIC = [\n", " staged / \"PRCAProlificExport_FileA.csv\",\n", " staged / \"PRCAProlificExport_FileB.csv\",\n", " ]\n", " QUALTRICS = staged / \"PRCAQualtricsExport_FileC.csv\"\n", " source = \"/tmp/sibling_data File A/B/C\"\n", "\n", "SEED = 42\n", "N_SPLITS = 5\n", "N_PERM = 30\n", "print(\"Data source:\", source)\n", "print(\"Prolific:\", [str(p) for p in PROLIFIC])\n", "print(\"Qualtrics:\", QUALTRICS)\n", "\n", "import json\n", "\n" ] }, { "cell_type": "markdown", "id": "7417320f", "metadata": {}, "source": [ "## 2. Load cohort & run all families\n" ] }, { "cell_type": "code", "execution_count": 2, "id": "a902bcc1", "metadata": { "execution": { "iopub.execute_input": "2026-07-25T15:40:51.210592Z", "iopub.status.busy": "2026-07-25T15:40:51.210387Z", "iopub.status.idle": "2026-07-25T15:41:16.716697Z", "shell.execute_reply": "2026-07-25T15:41:16.715939Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Matched analytic rows: 241\n", "{'n_prolific_raw': 262, 'n_prolific_unique': 262, 'n_qualtrics_raw': 273, 'n_qualtrics_with_pid': 255, 'n_qualtrics_complete_ca': 260, 'n_joined': 252, 'n_matched_both': 252, 'n_analytic': 241, 'n_dropped_missing_pid': 0, 'n_dropped_incomplete_ca': 11, 'n_dropped_unscorable_ca': 0, 'n_dropped_unjoined': 31, 'n_prolific_only': 10, 'n_qualtrics_only': 21, 'n_qualtrics_missing_pid': 18, 'waves': {'B': 153, 'A': 99}, 'notes': ['Normalized 19 DATA_EXPIRED Student status values to missing.', 'Full Prolific waves (File A/B) omit Ethnicity / Nationality / Language; demos tier uses Age, Sex, Country of residence, and Student status.', 'Analytic sample = Prolific∩Qualtrics with complete scorable PRCA group + interpersonal items.', 'Merge coverage (pre-CA filter): 252 matched Prolific∩Qualtrics; 21 Qualtrics-only (incl. 18 blank Q0 test rows; disregard); 10 Prolific-only (disregard).']}\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
spec_keylabelnn_regularprevalenceroc_aucaverage_precisionbalanced_accuracyf1brier
4q28_daysRide-share days (Q28)241.0101.00.4190870.7620930.6888730.7303750.7022220.189406
5q27_q28Transit intensity + ride-share days (Q27/Q28)239.0101.00.4225940.7612640.6896060.6960470.6666670.192595
6mobility_bundleCar + ride-share + employment bundle143.056.00.3916080.7467160.6052580.7166460.6666670.197594
2rideshareRide-share frequency (Q28/Q29)233.099.00.4248930.7451380.6548220.7310790.7085200.197184
0car_accessCar license & access (Q20/Q21)149.058.00.3892620.6073320.5150210.6394470.4761900.226675
8ca_benchmarkGroup + interpersonal CA (CA memo benchmark)241.0101.00.4190870.590000NaNNaNNaNNaN
3q27_intensityTransit intensity on use days (Q27)239.0101.00.4225940.5885710.5133990.6225430.4666670.230955
7geo_benchmarkLat/long (geo memo benchmark)241.0101.00.4190870.551000NaNNaNNaNNaN
1employmentEmployment status241.0101.00.4190870.5280760.4244490.5595120.5857140.247240
9chanceChance / prevalenceNaNNaNNaN0.500000NaN0.500000NaNNaN
\n", "
" ], "text/plain": [ " spec_key label n \\\n", "4 q28_days Ride-share days (Q28) 241.0 \n", "5 q27_q28 Transit intensity + ride-share days (Q27/Q28) 239.0 \n", "6 mobility_bundle Car + ride-share + employment bundle 143.0 \n", "2 rideshare Ride-share frequency (Q28/Q29) 233.0 \n", "0 car_access Car license & access (Q20/Q21) 149.0 \n", "8 ca_benchmark Group + interpersonal CA (CA memo benchmark) 241.0 \n", "3 q27_intensity Transit intensity on use days (Q27) 239.0 \n", "7 geo_benchmark Lat/long (geo memo benchmark) 241.0 \n", "1 employment Employment status 241.0 \n", "9 chance Chance / prevalence NaN \n", "\n", " n_regular prevalence roc_auc average_precision balanced_accuracy \\\n", "4 101.0 0.419087 0.762093 0.688873 0.730375 \n", "5 101.0 0.422594 0.761264 0.689606 0.696047 \n", "6 56.0 0.391608 0.746716 0.605258 0.716646 \n", "2 99.0 0.424893 0.745138 0.654822 0.731079 \n", "0 58.0 0.389262 0.607332 0.515021 0.639447 \n", "8 101.0 0.419087 0.590000 NaN NaN \n", "3 101.0 0.422594 0.588571 0.513399 0.622543 \n", "7 101.0 0.419087 0.551000 NaN NaN \n", "1 101.0 0.419087 0.528076 0.424449 0.559512 \n", "9 NaN NaN 0.500000 NaN 0.500000 \n", "\n", " f1 brier \n", "4 0.702222 0.189406 \n", "5 0.666667 0.192595 \n", "6 0.666667 0.197594 \n", "2 0.708520 0.197184 \n", "0 0.476190 0.226675 \n", "8 NaN NaN \n", "3 0.466667 0.230955 \n", "7 NaN NaN \n", "1 0.585714 0.247240 \n", "9 NaN NaN " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "participants, report = load_full_cohort(\n", " prolific_paths=PROLIFIC,\n", " qualtrics_path=QUALTRICS,\n", " join_how=\"inner\",\n", ")\n", "print(\"Matched analytic rows:\", len(participants))\n", "print(report)\n", "\n", "bundle = run_all_followup_analyses(\n", " participants,\n", " n_splits=N_SPLITS,\n", " n_perm_repeats=N_PERM,\n", " random_state=SEED,\n", ")\n", "comparison = bundle[\"comparison\"].copy()\n", "display(comparison)\n", "\n" ] }, { "cell_type": "markdown", "id": "994c0c59", "metadata": {}, "source": [ "## 3. Comparison figure\n" ] }, { "cell_type": "code", "execution_count": 3, "id": "949d8c20", "metadata": { "execution": { "iopub.execute_input": "2026-07-25T15:41:16.718865Z", "iopub.status.busy": "2026-07-25T15:41:16.718706Z", "iopub.status.idle": "2026-07-25T15:41:18.164835Z", "shell.execute_reply": "2026-07-25T15:41:18.164219Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Wrote /workspace/memos/figures/transit_covariate_followups_memo.png\n", "car_access AUC= 0.607 n= 149 → /workspace/memos/figures/car_access_predicts_transit_memo.png\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "employment AUC= 0.528 n= 241 → /workspace/memos/figures/employment_predicts_transit_memo.png\n", "rideshare AUC= 0.745 n= 233 → /workspace/memos/figures/rideshare_predicts_transit_memo.png\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "q27_intensity AUC= 0.589 n= 239 → /workspace/memos/figures/q27_intensity_predicts_transit_memo.png\n", "q28_days AUC= 0.762 n= 241 → /workspace/memos/figures/q28_days_predicts_transit_memo.png\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "q27_q28 AUC= 0.761 n= 239 → /workspace/memos/figures/q27_q28_predicts_transit_memo.png\n", "mobility_bundle AUC= 0.747 n= 143 → /workspace/memos/figures/mobility_bundle_predicts_transit_memo.png\n" ] } ], "source": [ "fig_path = ROOT / \"memos\" / \"figures\" / \"transit_covariate_followups_memo.png\"\n", "plot_comparison_memo_figure(comparison, output_path=fig_path)\n", "print(\"Wrote\", fig_path)\n", "\n", "for key, analysis in bundle[\"analyses\"].items():\n", " p = ROOT / \"memos\" / \"figures\" / f\"{key}_predicts_transit_memo.png\"\n", " plot_family_memo_figure(analysis, output_path=p)\n", " print(key, \"AUC=\", f\"{analysis['metrics']['roc_auc']:.3f}\", \"n=\", analysis[\"summary\"][\"sample\"][\"n\"], \"→\", p)\n", "\n" ] }, { "cell_type": "markdown", "id": "aeb1d0dd", "metadata": {}, "source": [ "## 4. Per-family association highlights\n" ] }, { "cell_type": "code", "execution_count": 4, "id": "a3bf8369", "metadata": { "execution": { "iopub.execute_input": "2026-07-25T15:41:18.167004Z", "iopub.status.busy": "2026-07-25T15:41:18.166862Z", "iopub.status.idle": "2026-07-25T15:41:18.191587Z", "shell.execute_reply": "2026-07-25T15:41:18.191310Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "=== Car license & access (Q20/Q21) ===\n", "Car license & access (Q20/Q21) Random Forest CV ROC-AUC = 0.607. Stronger discrimination than the geo benchmark (≈0.55). Exceeds the CA-score benchmark (≈0.59).\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
featurelevelnn_regularn_not_regularpct_regular
1Q20Yes12947820.364341
0Q20No201190.550000
4Q21Yes12338850.308943
2Q21No251960.760000
3Q21Not Sure1101.000000
\n", "
" ], "text/plain": [ " feature level n n_regular n_not_regular pct_regular\n", "1 Q20 Yes 129 47 82 0.364341\n", "0 Q20 No 20 11 9 0.550000\n", "4 Q21 Yes 123 38 85 0.308943\n", "2 Q21 No 25 19 6 0.760000\n", "3 Q21 Not Sure 1 1 0 1.000000" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "=== Employment status ===\n", "Employment status Random Forest CV ROC-AUC = 0.528. Comparable to or weaker than the geo benchmark (≈0.55). Does not exceed the CA-score benchmark (≈0.59).\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
featurelevelnn_regularn_not_regularpct_regular
0Employment statusFull-Time14867810.452703
1Employment statusOther6219430.306452
2Employment statusPart-Time3115160.483871
\n", "
" ], "text/plain": [ " feature level n n_regular n_not_regular pct_regular\n", "0 Employment status Full-Time 148 67 81 0.452703\n", "1 Employment status Other 62 19 43 0.306452\n", "2 Employment status Part-Time 31 15 16 0.483871" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "=== Ride-share frequency (Q28/Q29) ===\n", "Ride-share frequency (Q28/Q29) Random Forest CV ROC-AUC = 0.745. Stronger discrimination than the geo benchmark (≈0.55). Exceeds the CA-score benchmark (≈0.59).\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
featurelevelnn_regularn_not_regularpct_regular
1Q282-4 days a month6534310.523077
4Q28Never629530.145161
0Q280-1 days a month4711360.234043
2Q284-8 days a month4229130.690476
3Q288 or more days a month171610.941176
5Q291-2 rides in a typical day206821240.398058
6Q293-4 rides in a typical day171070.588235
7Q295-6 rides in a typical day9630.666667
8Q297 or more rides in a typical day1101.000000
\n", "
" ], "text/plain": [ " feature level n n_regular n_not_regular \\\n", "1 Q28 2-4 days a month 65 34 31 \n", "4 Q28 Never 62 9 53 \n", "0 Q28 0-1 days a month 47 11 36 \n", "2 Q28 4-8 days a month 42 29 13 \n", "3 Q28 8 or more days a month 17 16 1 \n", "5 Q29 1-2 rides in a typical day 206 82 124 \n", "6 Q29 3-4 rides in a typical day 17 10 7 \n", "7 Q29 5-6 rides in a typical day 9 6 3 \n", "8 Q29 7 or more rides in a typical day 1 1 0 \n", "\n", " pct_regular \n", "1 0.523077 \n", "4 0.145161 \n", "0 0.234043 \n", "2 0.690476 \n", "3 0.941176 \n", "5 0.398058 \n", "6 0.588235 \n", "7 0.666667 \n", "8 1.000000 " ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "=== Transit intensity on use days (Q27) ===\n", "Transit intensity on use days (Q27) Random Forest CV ROC-AUC = 0.589. Stronger discrimination than the geo benchmark (≈0.55). Does not exceed the CA-score benchmark (≈0.59).\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
featurelevelnn_regularn_not_regularpct_regular
0Q271-2 rides in a typical day190661240.347368
1Q273-4 rides in a typical day4531140.688889
2Q275-6 rides in a typical day3301.000000
3Q277 or more rides in a typical day1101.000000
\n", "
" ], "text/plain": [ " feature level n n_regular n_not_regular \\\n", "0 Q27 1-2 rides in a typical day 190 66 124 \n", "1 Q27 3-4 rides in a typical day 45 31 14 \n", "2 Q27 5-6 rides in a typical day 3 3 0 \n", "3 Q27 7 or more rides in a typical day 1 1 0 \n", "\n", " pct_regular \n", "0 0.347368 \n", "1 0.688889 \n", "2 1.000000 \n", "3 1.000000 " ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "=== Ride-share days (Q28) ===\n", "Ride-share days (Q28) Random Forest CV ROC-AUC = 0.762. Stronger discrimination than the geo benchmark (≈0.55). Exceeds the CA-score benchmark (≈0.59).\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
featurelevelnn_regularn_not_regularpct_regular
4Q28Never7011590.157143
1Q282-4 days a month6534310.523077
0Q280-1 days a month4711360.234043
2Q284-8 days a month4229130.690476
3Q288 or more days a month171610.941176
\n", "
" ], "text/plain": [ " feature level n n_regular n_not_regular pct_regular\n", "4 Q28 Never 70 11 59 0.157143\n", "1 Q28 2-4 days a month 65 34 31 0.523077\n", "0 Q28 0-1 days a month 47 11 36 0.234043\n", "2 Q28 4-8 days a month 42 29 13 0.690476\n", "3 Q28 8 or more days a month 17 16 1 0.941176" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "=== Transit intensity + ride-share days (Q27/Q28) ===\n", "Transit intensity + ride-share days (Q27/Q28) Random Forest CV ROC-AUC = 0.761. Stronger discrimination than the geo benchmark (≈0.55). Exceeds the CA-score benchmark (≈0.59).\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
featurelevelnn_regularn_not_regularpct_regular
0Q271-2 rides in a typical day190661240.347368
1Q273-4 rides in a typical day4531140.688889
2Q275-6 rides in a typical day3301.000000
3Q277 or more rides in a typical day1101.000000
8Q28Never6911580.159420
5Q282-4 days a month6534310.523077
4Q280-1 days a month4611350.239130
6Q284-8 days a month4229130.690476
7Q288 or more days a month171610.941176
\n", "
" ], "text/plain": [ " feature level n n_regular n_not_regular \\\n", "0 Q27 1-2 rides in a typical day 190 66 124 \n", "1 Q27 3-4 rides in a typical day 45 31 14 \n", "2 Q27 5-6 rides in a typical day 3 3 0 \n", "3 Q27 7 or more rides in a typical day 1 1 0 \n", "8 Q28 Never 69 11 58 \n", "5 Q28 2-4 days a month 65 34 31 \n", "4 Q28 0-1 days a month 46 11 35 \n", "6 Q28 4-8 days a month 42 29 13 \n", "7 Q28 8 or more days a month 17 16 1 \n", "\n", " pct_regular \n", "0 0.347368 \n", "1 0.688889 \n", "2 1.000000 \n", "3 1.000000 \n", "8 0.159420 \n", "5 0.523077 \n", "4 0.239130 \n", "6 0.690476 \n", "7 0.941176 " ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "=== Car + ride-share + employment bundle ===\n", "Car + ride-share + employment bundle Random Forest CV ROC-AUC = 0.747. Stronger discrimination than the geo benchmark (≈0.55). Exceeds the CA-score benchmark (≈0.59).\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
featurelevelnn_regularn_not_regularpct_regular
13Employment statusFull-Time8137440.456790
14Employment statusOther4514310.311111
15Employment statusPart-Time175120.294118
1Q20Yes12345780.365854
0Q20No201190.550000
4Q21Yes11837810.313559
2Q21No241860.750000
3Q21Not Sure1101.000000
9Q28Never489390.187500
6Q282-4 days a month4224180.571429
5Q280-1 days a month329230.281250
7Q284-8 days a month14860.571429
8Q288 or more days a month7610.857143
10Q291-2 rides in a typical day13350830.375940
11Q293-4 rides in a typical day7340.428571
12Q295-6 rides in a typical day3301.000000
\n", "
" ], "text/plain": [ " feature level n n_regular \\\n", "13 Employment status Full-Time 81 37 \n", "14 Employment status Other 45 14 \n", "15 Employment status Part-Time 17 5 \n", "1 Q20 Yes 123 45 \n", "0 Q20 No 20 11 \n", "4 Q21 Yes 118 37 \n", "2 Q21 No 24 18 \n", "3 Q21 Not Sure 1 1 \n", "9 Q28 Never 48 9 \n", "6 Q28 2-4 days a month 42 24 \n", "5 Q28 0-1 days a month 32 9 \n", "7 Q28 4-8 days a month 14 8 \n", "8 Q28 8 or more days a month 7 6 \n", "10 Q29 1-2 rides in a typical day 133 50 \n", "11 Q29 3-4 rides in a typical day 7 3 \n", "12 Q29 5-6 rides in a typical day 3 3 \n", "\n", " n_not_regular pct_regular \n", "13 44 0.456790 \n", "14 31 0.311111 \n", "15 12 0.294118 \n", "1 78 0.365854 \n", "0 9 0.550000 \n", "4 81 0.313559 \n", "2 6 0.750000 \n", "3 0 1.000000 \n", "9 39 0.187500 \n", "6 18 0.571429 \n", "5 23 0.281250 \n", "7 6 0.571429 \n", "8 1 0.857143 \n", "10 83 0.375940 \n", "11 4 0.428571 \n", "12 0 1.000000 " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "for key, analysis in bundle[\"analyses\"].items():\n", " print(\"\\n===\", FEATURE_SPECS[key][\"label\"], \"===\")\n", " print(analysis[\"summary\"][\"verdict\"][\"interpretation\"])\n", " display(analysis[\"associations\"])\n", "\n" ] }, { "cell_type": "markdown", "id": "a95c9b6a", "metadata": {}, "source": [ "## 5. Export artifacts\n" ] }, { "cell_type": "code", "execution_count": 5, "id": "d99dc4fa", "metadata": { "execution": { "iopub.execute_input": "2026-07-25T15:41:18.192977Z", "iopub.status.busy": "2026-07-25T15:41:18.192871Z", "iopub.status.idle": "2026-07-25T15:41:18.235690Z", "shell.execute_reply": "2026-07-25T15:41:18.235373Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Results card: /workspace/outputs/transit_covariate_rf/followup_results_card.json\n", "{\n", " \"secondary_rq\": \"Among the geo-memo follow-up candidates (car access/license, employment, ride-share frequency), which features best predict regular public-transit use?\",\n", " \"benchmarks\": {\n", " \"geo_auc\": 0.551,\n", " \"ca_auc\": 0.59,\n", " \"chance_auc\": 0.5\n", " },\n", " \"best_spec\": \"q28_days\",\n", " \"best_auc\": 0.7620933521923621,\n", " \"comparison\": [\n", " {\n", " \"spec_key\": \"q28_days\",\n", " \"label\": \"Ride-share days (Q28)\",\n", " \"n\": 241.0,\n", " \"n_regular\": 101.0,\n", " \"prevalence\": 0.4190871369294606,\n", " \"roc_auc\": 0.7620933521923621,\n", " \"average_precision\": 0.688872869993286,\n", " \"balanced_accuracy\": 0.7303748231966054,\n", " \"f1\": 0.7022222222222222,\n", " \"brier\": 0.18940626566184446\n", " },\n", " {\n", " \"spec_key\": \"q27_q28\",\n", " \"label\": \"Transit intensity + ride-share days (Q27/Q28)\",\n", " \"n\": 239.0,\n", " \"n_regular\": 101.0,\n", " \"prevalence\": 0.4225941422594142,\n", " \"roc_auc\": 0.7612641698952504,\n", " \"average_precision\": 0.689606135030495,\n", " \"balanced_accuracy\": 0.6960467785909026,\n", " \"f1\": 0.6666666666666666,\n", " \"brier\": 0.19259516438231897\n", " },\n", " {\n", " \"spec_key\": \"mobility_bundle\",\n", " \"label\": \"Car + ride-share + employment bundle\",\n", " \"n\": 143.0,\n", " \"n_regular\": 56.0,\n", " \"prevalence\": 0.3916083916083916,\n", " \"roc_auc\": 0.7467159277504106,\n", " \"average_precision\": 0.6052583001385728,\n", " \"balanced_accuracy\": 0.7166461412151067,\n", " \"f1\": 0.6666666666666666,\n", " \"brier\": 0.19759361412167553\n", " },\n", " {\n", " \"spec_key\"\n" ] } ], "source": [ "out = ROOT / \"outputs\" / \"transit_covariate_rf\"\n", "paths = save_followup_bundle(bundle, out)\n", "print(\"Results card:\", paths[\"results_card\"])\n", "print(json.dumps(bundle[\"results_card\"], indent=2)[:1500])\n", "\n" ] }, { "cell_type": "markdown", "id": "ee47e973", "metadata": {}, "source": [ "## 6. Interpretation notes\n", "\n", "- Ride-share days (`Q28`) carries the strongest single-family signal among the geo-memo candidates.\n", "- The joint bundle is essentially ride-share-dominated on the complete-case subset that also has car items.\n", "- Car access improves on geography/CA but is limited by missingness on `Q20`/`Q21`.\n", "- Employment status alone is near chance and weaker than the geo RF.\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }