Memo: Transit focus — regular use & intensity with mobility held out

Research memorandum — TF1 / TF2 secondary focus

Author

Jack J. Burleson

Researcher: Jack J. Burleson
Date: 2026–07-28

1 Question

Given only a user’s demographics, employment, and geography (and, in a second arm, their CA scores (McCroskey, 1970)), can models estimate (1) whether they are a regular public-transit rider and (2) how much public transit they use — when every transit / ride-share / car survey answer is held out of the features and persona prompt?

2 Why this focus

Primary persona work asks whether transit cues help predict CA. Here the twin asks whether non-transit context predicts transit. That is the natural LLM task if a digital twin must guess ridership without seeing Q26–Q29.

3 Design (seed = 42)

Hold-out: Q26, Q27, Q28, Q29, Q20, Q21, Q19
TF1 features: Age, Sex, Country, Student, Employment, lat/long (± group & interpersonal CA)
TF2 features: same profile + CA → Q26 ordinal; Q27 among reporters
Model: balanced Random Forest ((Breiman, 2001)), stratified 5-fold CV
Sample: complete-case n = 224 (of 241 analytic) for profile specs

CLI: ca-personas transit-focus · artifacts outputs/transit_focus/ · LLM prompts under persona_prompts/.

4 Results summary

Table Table 1 reports the transit-focus regular-use and intensity results.

Table 1.. Transit focus regular and intensity results.

Spec	n	Metric	Value
Profile → regular transit	224	ROC-AUC	0.662
Profile + CA → regular transit	224	ROC-AUC	0.672
Profile + CA → `Q26` days	224	Balanced accuracy / ordinal MAE	0.283 / 1.22
Profile + CA → `Q27` rides	222	Balanced accuracy / ordinal MAE	0.371 / 0.38

TF1. Non-mobility profile alone recovers regular ridership well above chance (0.662). Adding CA scores helps only modestly (+0.010). Permutation importance is led by Age, then longitude, latitude, employment — consistent with the Wave-2 demographics memo (demographics_predict_transit.qmd; Age+Sex+Student AUC 0.618) but stronger once place and work are included. The profile-only result also parallels the CA → transit memo (ca_scores_predict_transit.qmd) and geography memo (geo_predicts_transit.qmd) in that non-mobility context alone is a weak-to-moderate ridership signal.

TF2. Fine-grained Q26 recovery is weak in absolute terms (balanced accuracy 0.28) though it beats a majority classifier. Q27 is heavily imbalanced (most respondents report 1–2 rides/day); the forest does not beat majority accuracy.

5 LLM twin path

Transit-focus persona tiers (tf_demos → tf_employment → tf_geo → tf_geo_ca) reuse narrative demos/employment/geo blocks and ask for JSON {regular_transit, q26_days, confidence} with mobility stems banned from the prompt. Not yet run on GPU providers; tabular numbers above are the classical ceiling for that task.

6 Remaining uncertainties

Will live LLMs match or underperform the 0.66–0.67 AUC tabular floor when transit is held out?
Should car access (Q20/Q21) ever be allowed as “context” rather than held out? Current design treats it as mobility and excludes it.
Ordinal models (ordered logit / ordinal RF) may improve Q26 MAE relative to flat multiclass RF.

Sources: src/ca_personas/transit_focus.py · ca-personas transit-focus · write-up · github.com/Exios66/psych755-jjb

References

Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32. https://doi.org/10.1023/A:1010933404324

McCroskey, J. C. (1970). Measures of communication-bound anxiety. Speech Monographs, 37(4), 269–277. https://doi.org/10.1080/03637757009375677

--- title: "Memo: Transit focus — regular use & intensity with mobility held out" subtitle: "Research memorandum — TF1 / TF2 secondary focus" author: Jack J. Burleson bibliography: ../references.bib --- **Researcher:** Jack J. Burleson **Date:** 2026–07-28 --- ## Question Given only a user’s **demographics, employment, and geography** (and, in a second arm, their **CA scores** [@mccroskey1970]), can models estimate (1) whether they are a **regular public-transit rider** and (2) **how much** public transit they use — when every transit / ride-share / car survey answer is **held out** of the features and persona prompt? ## Why this focus Primary persona work asks whether *transit cues help predict CA*. Here the twin asks whether *non-transit context predicts transit*. That is the natural LLM task if a digital twin must guess ridership without seeing `Q26`–`Q29`. ## Design (seed = 42) - **Hold-out:** `Q26`, `Q27`, `Q28`, `Q29`, `Q20`, `Q21`, `Q19` - **TF1 features:** Age, Sex, Country, Student, Employment, lat/long (± group & interpersonal CA) - **TF2 features:** same profile + CA → `Q26` ordinal; `Q27` among reporters - **Model:** balanced **Random Forest** ([@breiman2001]), stratified 5-fold CV - **Sample:** complete-case n = **224** (of 241 analytic) for profile specs CLI: `ca-personas transit-focus` · artifacts `outputs/transit_focus/` · LLM prompts under `persona_prompts/`. ## Results summary Table @tbl-tf-1 reports the transit-focus regular-use and intensity results. | Spec | n | Metric | Value | |---|---:|---|---:| | Profile → regular transit | 224 | ROC-AUC | **0.662** | | Profile + CA → regular transit | 224 | ROC-AUC | **0.672** | | Profile + CA → `Q26` days | 224 | Balanced accuracy / ordinal MAE | **0.283** / **1.22** | | Profile + CA → `Q27` rides | 222 | Balanced accuracy / ordinal MAE | **0.371** / **0.38** | : Transit focus regular and intensity results. {#tbl-tf-1} **TF1.** Non-mobility profile alone recovers regular ridership well above chance (0.662). Adding CA scores helps only modestly (+0.010). Permutation importance is led by **Age**, then **longitude**, latitude, employment — consistent with the Wave-2 demographics memo ([`demographics_predict_transit.qmd`](demographics_predict_transit.qmd); Age+Sex+Student AUC 0.618) but stronger once place and work are included. The profile-only result also parallels the CA → transit memo ([`ca_scores_predict_transit.qmd`](ca_scores_predict_transit.qmd)) and geography memo ([`geo_predicts_transit.qmd`](geo_predicts_transit.qmd)) in that non-mobility context alone is a weak-to-moderate ridership signal. **TF2.** Fine-grained `Q26` recovery is weak in absolute terms (balanced accuracy 0.28) though it beats a majority classifier. `Q27` is heavily imbalanced (most respondents report 1–2 rides/day); the forest does not beat majority accuracy. ## LLM twin path Transit-focus persona tiers (`tf_demos` → `tf_employment` → `tf_geo` → `tf_geo_ca`) reuse narrative demos/employment/geo blocks and ask for JSON `{regular_transit, q26_days, confidence}` with mobility stems banned from the prompt. Not yet run on GPU providers; tabular numbers above are the classical ceiling for that task. ## Remaining uncertainties 1. Will live LLMs match or underperform the 0.66–0.67 AUC tabular floor when transit is held out? 2. Should car access (`Q20`/`Q21`) ever be allowed as “context” rather than held out? Current design treats it as mobility and excludes it. 3. Ordinal models (ordered logit / ordinal RF) may improve `Q26` MAE relative to flat multiclass RF. *Sources:* `src/ca_personas/transit_focus.py` · `ca-personas transit-focus` · [write-up](../docs/secondary_rq_transit_focus.md) · [github.com/Exios66/psych755-jjb](https://github.com/Exios66/psych755-jjb) *Related memos:* [demographics → transit](demographics_predict_transit.qmd) · [CA → transit](ca_scores_predict_transit.qmd) · [geography → transit](geo_predicts_transit.qmd)