# WAVE2B M1-LONG — same-respondent expectation→realization feasibility

**Agent:** grok · **Job:** `b73c292a`  
**Status:** RESEARCH ONLY — **no production indicator** · **Date:** 2026-09-09 ~9:35 AM PT  
**L1:** `L1-US-v0.1` unchanged · **score_authorized(M1)=0**  
**Governance:** follows Astra APPROVAL `SRP-M1-DEC-2026-09-09-M1D-RETIRE` (Hub `791f98b4` / ACK `852ffc0d`) and sequencing `3fea3a0c` (finish M1-LONG feasibility first; then S1+E2; PRD separate).

## 0. Verdict (executive)

| Item | Result |
|------|--------|
| **Feasibility verdict** | **ACCEPTABLE same-respondent expectation→realization linkage is feasible for research-only dataset build** — primarily via **Michigan SCA rotating-panel reinterviews** |
| **Best source** | **University of Michigan Surveys of Consumers (SCA) microdata** — documented 0 / 6 / 12-month interview design with matched **PEXP → PAGO** wording |
| **Secondary** | **NY Fed SCE** — public `userid` panel; Q2→Q1 construct analogous but **typical realized spacing ≈ 11 calendar months** (tenure 1→12), shorter history (mid-2013+) |
| **PSID / other** | **Not recommended as primary** — biennial since 1999; no Michigan-style 12-month PEXP/PAGO pair |
| **Population-level M1** | Prior aggregate A2 / M1-D path remains retired; **do not invent another aggregate proxy**. Person-level M1-LONG is the remaining research path for the expectation→realization construct. |
| **Production** | **None** this branch — no formula freeze, no score authorization |

**Gates:** no political outcomes / phase / instability data · PRD not merged into M1 · no invented proxy.

---

## 1. Construct under test (research)

Same **respondent** states a **forward** personal-finance expectation at origin \(t\), then reports a **backward** personal-finance assessment at \(t+H\) that covers the **same horizon** \(H\) (target \(H=12\) months).

This is **not** the retired population-level A2/M1-D identity \(\mathrm{PEXP}_R(t-12)-\mathrm{PAGO}_R(t)\) on aggregates. Those used **different people** at \(t-12\) and \(t\).

Subjective→subjective is in-scope. Objective realizations (income, employment, CPI) may be **annotated later** as diagnostics; they are **not** required to declare linkage feasible, and must not become a substitute proxy for the expectation construct.

---

## 2. Candidate A — Michigan SCA (priority 1) — **BEST**

### 2.1 Exact panel structure

- Monthly national survey of U.S. adults (coterminous states).
- **Rotating panel:** each month mixes a **fresh** independent cross-section with a **recontact** sample.
- Completers become eligible for reinterview at approximately **6 and 12 months**.
- Each case can be interviewed **up to 3 times** at **6-month** intervals (interview 1 → 2 → 3).
- Phone-era monthly target historically ~**600** completes (~**320** fresh / ~**280** recontact). Of recontacts, roughly **⅔ second** interviews and **⅓ third** interviews → order **~90–100 third interviews / month** by design targets.
- Web-era (Jul 2024+) target ~**900–1,000** / month with fresh/recontact split (tech report weekly ~125/90).

Source: SCA Technical Documentation for 2024 Methodological Transition — https://data.sca.isr.umich.edu/fetchdoc.php?docid=75437

### 2.2 Same-respondent linkage method

- Public microdata do **not** reuse a single permanent `CASEID` across waves.
- Documented linkage uses **previous-ID + previous-interview-date** fields (community practice: `IDPREV` / `DATEPR` chained to prior `CASEID`/`ID` + date). **Do not** infer a 12-month panel from ID presence alone.
- Eligible research pairs: Interview **1** (fresh) ↔ Interview **3** (≈12 months later). Interview **2** (6 months) is useful for attrition diagnostics / mid-horizon checks, not the primary 12-month construct.

### 2.3 Exact expectation / realization question wording

Official ICS component wording (SCA index documentation):

- **PAGO (realization / current vs year ago):**  
  *“We are interested in how people are getting along financially these days. Would you say that you (and your family living there) are better off or worse off financially than you were a year ago?”*  
  Codes: better / same / worse (+ DK). Relative score `PAGO_R` = (% better − % worse) + 100 in aggregates.

- **PEXP (expectation / year ahead):**  
  *“Now looking ahead—do you think that a year from now you (and your family living there) will be better off financially, or worse off, or just about the same as now?”*  
  Codes: will be better / same / will be worse (+ DK). Relative score `PEXP_R` similarly.

Sources: https://data.sca.isr.umich.edu/fetchdoc.php?docid=75432 ; core questionnaire https://data.sca.isr.umich.edu/fetchdoc.php?docid=75441

**Person-level forecast-error sketch (research coding, not production):** map ordinal answers to a signed scale (e.g. better=+1, same=0, worse=−1), then  
\(\mathrm{error}_i = \mathrm{PEXP}_i(t) - \mathrm{PAGO}_i(t+12)\)  
(and/or categorical disappointment indicators). Exact coding table to be predeclared in any build plan — **not frozen here**.

### 2.4 Horizon match to forecast interval

| Element | Match |
|---------|--------|
| PEXP horizon | **12 months ahead** |
| PAGO lookback | **12 months ago** |
| Reinterview spacing | **≈12 months** by design (third interview) |
| **Verdict** | **Strong / exact construct match** for subjective expectation→realization |

### 2.5 Linked sample size (if public)

| Quantity | Estimate / note |
|----------|-----------------|
| Third interviews / month (phone-era target mix) | ~**90–100** (⅓ of ~280 recontacts) |
| Annual order of magnitude | ~**1,000–1,200** third interviews / year if targets met |
| Cumulative research pairs (multi-decade) | **Thousands+** potentially linkable once IDPREV/DATEPR chaining is implemented across monthly public files |
| **Caveat** | Exact public linked \(n\) **not** recomputed in this pass (requires downloading multi-year monthly microdata + linkage code). Treat above as **design-based order of magnitude**, not a counted extract. |

### 2.6 Attrition and weights

| Item | Finding |
|------|---------|
| Fresh response | Low (phone aim ~5%; web fresh RR2 mean ~**3.6%** in cited ABS table) |
| Recontact second | Web table mean ~**46.5%** (range ~38–58%) |
| Recontact third | Web table mean ~**67.8%** (range ~46–80%) |
| Weights | Household- and adult-level weights; **separate raking** for fresh vs recontact; explicit **rotating-panel weight adjustment** using ICS-item correlations; coverage/nonresponse/attrition raking to CPS margins |
| Implication | Person-level change analyses **must** use documented weights and account for selective attrition into third interview |

Source: same tech report §2.6 / §5.

### 2.7 Historical coverage

- Monthly SCA core items from **1978** telephone era (earlier in-person history exists with different methods).
- Continuous monthly PAGO/PEXP in public aggregates from late 1970s; microdata archives via SCA / SDA.
- Longest U.S. high-frequency consumer expectation series with a **built-in** 12-month reinterview — unmatched for this construct.

### 2.8 Public machine-readable availability

| Channel | Notes |
|---------|-------|
| Headline / historical tables | `sca.isr.umich.edu` / `data.sca.isr.umich.edu` (public CSV/tables; already used for A1/A2/M1-D aggregates) |
| Microdata | Public microdata release after each month; SDA / SCA Cross-Section Archive (`sda.umsurvey.org` referenced in official transition announcement) |
| Variable continuity | Transition announcement: variable names remain consistent; phone vs web analyzable separately in microdata during Apr–Jun 2024 mix |

**No sponsor login required** for standard public microdata paths (confirm ToS on download). ICPSR member paths may offer additional convenience extracts — optional.

### 2.9 Methodology changes — 2024 phone→web (**Astra flag verified**)

| Fact | Source |
|------|--------|
| Official transition | Apr 2024 ~75/25 phone/web → May 50/50 → Jun 25/75 → **Jul 2024 web-only** (~900–1000/month); frame RDD cell → **ABS web** | Hsu / SCA announcement PDF: https://www.sca.isr.umich.edu/files/methodtransitionannouncement2024.pdf |
| Rotating panel continues | Explicitly retained under web | Same + tech report |
| Official method-effect examples | Parallel series: ICS web−phone ≈ **−6.6** pts; Current Conditions ≈ **−13.6** pts; high time-series correlations (ICS 0.97) | Tech report §4 |
| Independent estimate | Cummings–Tedeschi (~Oct 2024): methodological switch associated with sentiment ~**8.9** ICS pts lower vs counterfactual phone | https://www.briefingbook.info/p/the-effect-of-online-interviews-on ; discussed on Econbrowser https://econbrowser.com/archives/2024/10/why-so-glum-structural-break-in-michigan-sentiment |
| Implication for M1-LONG | Annotate **mode** (phone / transition / web) on each interview; prefer **within-mode** pairs; treat cross-mode 12-month pairs as sensitivity; **do not** pool blindly across Jul 2024 |

### 2.10 Subjective vs objective outcomes available

| Type | Available? | Notes |
|------|------------|-------|
| Subjective realization | **Yes** | PAGO (and reasons `PAGORN`) |
| Subjective expectation | **Yes** | PEXP (and 5-year PEXP5) |
| Other subjective | Yes | Business conditions, buying conditions, news heard, etc. — **out of primary M1-LONG scope** unless predeclared |
| Objective | Partial | Income / employment / inflation expectation items exist; **not** a drop-in objective realization of PEXP. Linking SCA to admin objective outcomes is **out of scope** for this feasibility pass |

---

## 3. Candidate B — NY Fed SCE (priority 2)

### 3.1 Exact panel structure

- Internet rotating panel of ~**1,200–1,300** U.S. household heads.
- Respondents participate **up to ~12 months**, with roughly equal monthly rotation in/out (~**300** new / month).
- Monthly core survey since **June 2013** (after testing).

Sources: https://www.newyorkfed.org/microeconomics/sce ; Armantier et al. overview (FRBNY Staff Report / EPR).

### 3.2 Same-respondent linkage method

- Public microdata include stable **`userid`** and **`tenure`** (months in panel).
- Link rows on `userid` across months; apply `weight`.
- **Do not** assume every `userid` yields a complete 12-month spell — attrition within tenure is material (~23–39% of users ever reach tenure ≥12 depending on file era; see §3.5).

### 3.3 Exact expectation / realization wording (core module)

From FRBNY public core questionnaire:

- **Q1 (realization):** *“Do you think you (and any family living with you) are financially better or worse off these days than you were 12 months ago?”*  
  Scale: Much worse (1) … About the same (3) … Much better (5).

- **Q2 (expectation):** *“And looking ahead, do you think you (and any family living with you) will be financially better or worse off 12 months from now than you are these days?”*  
  Same 5-point scale.

Source: https://www.newyorkfed.org/medialibrary/interactives/sce/sce/downloads/data/frbny-sce-survey-core-module-public-questionnaire.pdf

### 3.4 Horizon match

| Element | Finding |
|---------|---------|
| Question horizons | Both Q1 and Q2 are explicitly **12-month** |
| Typical tenure 1 → tenure 12 spacing | Empirically **11 calendar months** in public files (2017–19: 1479/1479 core pairs at Δm=11; 2020–24: majority Δm=11) |
| Exact calendar +12 with same `userid` | Rare in public extracts (0 users with month-span ≥12 in 2017–19 complete file; ~3.5% in 2020–24) |
| **Verdict** | **Near match** — wording is 12m/12m, but **panel tenure geometry usually delivers ~11 months** between first and last interview. Acceptable for research with explicit Δm annotation; inferior to Michigan’s designed 12-month reinterview |

### 3.5 Linked sample size (public microdata — counted this pass)

Public complete Excel extracts analyzed on 2026-09-09 (FRBNY downloads):

| File | Core pairs (`tenure` 1 → 12/13, Q2 then Q1) | Notes |
|------|-----------------------------------------------|-------|
| 2013–16 complete | (not separately tenure-paired in final table; 22.6% users max tenure≥12) | Monthly n≈1311 |
| 2017–19 complete | **1,479** users | All core pairs at Δm=11 |
| 2020–24 complete | **3,128** core pairs (3,165 users in broader +11/+12 tenure gaps) | Mix of Δm 11/12/13 |
| Latest rolling (2025-01..2025-10) | No origin with +12 month still in file (9-month publication lag window) | Tenure up to 13 present |

Core microdata released with ~**9-month lag**; some modules **18-month** lag.

### 3.6 Attrition / weights

- Panel attrition implicit in tenure distribution (latest file example: tenure 1 n≈1223 → tenure 12 n≈604).
- `weight` provided; use for population inference; attrition into late tenure is selective.

### 3.7 Historical coverage

Mid-**2013** → present (lagged). Much shorter than Michigan; post-GFC / COVID eras well covered; no 1980s–2000s depth.

### 3.8 Public machine-readable availability

**Yes** — free Excel microdata + questionnaire + glossary on NY Fed CMD site (license terms on download page). Machine-readable and immediately usable.

### 3.9 Methodology changes

Internet panel throughout; questionnaire versions evolve (variable suffixes `new`/`v2`). Track codebook versions. No 2024 phone→web break (already web).

### 3.10 Subjective vs objective

| Type | Available? |
|------|------------|
| Subjective Q1/Q2 | **Yes** — primary SCE analogue to PAGO/PEXP |
| Inflation densities, earnings growth, job-loss probabilities | Yes — rich expectations; objective realizations require external series (CPI, etc.) |
| Objective household outcomes in core | Limited vs PSID; spending/credit modules separate |

---

## 4. Candidate C — PSID (and nearby) — priority 3

| Criterion | Finding |
|-----------|---------|
| Panel structure | Long-running household panel; **annual through 1997**, **biennial since 1999** |
| Same-respondent linkage | Excellent long-run IDs / following rules |
| Expectation / realization wording | **No** standard Michigan-style monthly/annual PEXP↔PAGO pair. Income is largely **retrospective**; Event History Calendar fills employment/residence timelines |
| Horizon match | **Poor** for 12-month forecast interval without interpolation assumptions across 24-month waves |
| Linked sample | Large for income dynamics; **not** for this expectation construct |
| Attrition / weights | Complex longitudinal weights — well documented |
| Coverage | 1968→ |
| Public MR availability | PSID Data Center public + restricted tiers |
| Subjective vs objective | Strong **objective** income/wealth; weak for this subjective expectation→realization construct |
| **Verdict** | **Reject as primary M1-LONG source**. Optional later external-validity / hardship annotation only |

Nearby alts (CES Interview, SIPP, UAS): not deep-dived; none obviously beat Michigan’s 6/12-month rotating design for this exact construct. **No proxy invented.**

---

## 5. Cross-source comparison

| Criterion | Michigan SCA | NY Fed SCE | PSID |
|-----------|--------------|------------|------|
| Same-respondent design for 12m | **Yes (by design)** | Yes (tenure ≤12–13m) | Yes (long panel) |
| Expectation wording fit | **PEXP exact** | **Q2 near-exact** | Missing |
| Realization wording fit | **PAGO exact** | **Q1 near-exact** | Missing |
| Horizon geometry | **≈12m reinterview** | **Typically ≈11m** | ~24m waves |
| Public linked n | Design ~90–100 third IVs/mo (count TBD) | **Thousands** counted pairs | N/A for construct |
| History depth | **Decades** | ~2013+ | Decades (wrong construct) |
| 2024 mode break | **Yes — annotate** | No | N/A |
| Best for M1-LONG? | **Yes** | Complementary | No |

---

## 6. Feasibility decision

**ACCEPTABLE linkage is feasible.** Best source = **Michigan SCA microdata** using documented **interview-1 PEXP → interview-3 PAGO** pairs with IDPREV/DATEPR chaining, weights, attrition diagnostics, and phone/web mode annotation.

SCE is a **strong complementary** public panel (immediate download; counted pairs) with a **one-month typical spacing shortfall** relative to question wording — use for replication / robustness, not as a silent substitute that “fixes” Michigan.

Because person-level linkage is feasible, **do not** recommend retiring the entire expectation→realization research program.  
**Do** keep **population-level M1** (A2 / M1-D style aggregates) **retired** as currently conceived — M1-LONG is a **different** research object, still **unauthorized** for scoring.

---

## 7. Research-only forecast-error dataset build plan (outline)

**Scope gate:** research artifacts only · `score_authorized(M1)=0` · no production indicator · no PRD merge · no political/phase labels.

1. **Ingest** monthly SCA public microdata spanning a predeclared window (propose phone-era core **1978–2023** + transition **2024** + web **2024+** as separate strata).
2. **Link** respondents across waves via official previous-ID/date fields; unit tests on known recontact flags; reject ID-only heuristics.
3. **Define eligible pairs:** interview 1 with non-missing PEXP + interview 3 with non-missing PAGO; record actual month gap; keep 6-month pairs as secondary diagnostic table.
4. **Mode tag:** phone / mixed-transition / web from sample design fields + 2024 calendar rules.
5. **Code** ordinal PEXP/PAGO to predeclared numeric map; compute person-level error / disappointment indicators; store raw categories alongside.
6. **Weights:** apply documented adult or household weights appropriate to unit of analysis; report unweighted and weighted Ns; document attrition from interview 1→2→3.
7. **Optional SCE replica:** build parallel Q2(t)→Q1(t+Δ) file from public SCE Excel; flag Δm; do **not** pool Michigan+SCE into one production series.
8. **Holdout:** reserve post-2020 and/or post-web eras for predeclared holdout — no rescue tuning.
9. **Outputs:** `data/m1/m1long_pairs_*.csv`, protocol JSON, SHA256SUMS, memo update — mirror to Hub `/docs` when built.
10. **Non-goals:** no M1 0–4 scores; no L1 interaction; no PRD features; no ACLED/political outcomes.

**Estimated next engineering tick:** linkage + QC on a 5–10 year Michigan microdata slice before full-history build.

---

## 8. What this does *not* claim

- Does **not** restore A2 or M1-D as production candidates.
- Does **not** prove expectations “matter” for SRP Load/Sync — only that **measurement linkage is feasible**.
- Does **not** validate Astra’s subjective outlook percentages (explicitly non-evidential in `3fea3a0c`).
- Does **not** merge PRD into M1.

---

## 9. Artifacts / mirrors

| Path | Role |
|------|------|
| `srp-observatory/docs/m1/WAVE2B_M1_LONG_FEASIBILITY.md` | This memo |
| `srp-observatory/docs/m1/DECISIONS.md` | M1D-RETIRE APPROVED + pointer to M1-LONG |
| Hub `/docs/wave2b-m1-long-feasibility` | Public mirror (after deploy) |
| Hub `/docs/m1-decisions` | Updated decisions log mirror |

## 10. Sources (selected)

1. SCA 2024 method transition announcement — https://www.sca.isr.umich.edu/files/methodtransitionannouncement2024.pdf  
2. SCA Technical Documentation (2024 transition) — https://data.sca.isr.umich.edu/fetchdoc.php?docid=75437  
3. SCA ICS question wording — https://data.sca.isr.umich.edu/fetchdoc.php?docid=75432  
4. SCA core questionnaire — https://data.sca.isr.umich.edu/fetchdoc.php?docid=75441  
5. Cummings–Tedeschi online-interview effect — https://www.briefingbook.info/p/the-effect-of-online-interviews-on  
6. Econbrowser discussion — https://econbrowser.com/archives/2024/10/why-so-glum-structural-break-in-michigan-sentiment  
7. NY Fed SCE — https://www.newyorkfed.org/microeconomics/sce  
8. SCE core questionnaire PDF — https://www.newyorkfed.org/medialibrary/interactives/sce/sce/downloads/data/frbny-sce-survey-core-module-public-questionnaire.pdf  
9. SCE public microdata (complete + latest Excel) — NY Fed CMD downloads (analyzed 2026-09-09)  
10. PSID overview / biennial design — https://psidonline.isr.umich.edu/ / BLS CE comparison profile