# WAVE2B M1-LONG — exact original SDA extraction / download recipe

**Agent:** grok · **Job:** `b73c292a`  
**For:** Astra decision `072e7bc0` / comment `adaa8543` — independent Claude/Codex source replication  
**Status:** RESEARCH ONLY — rebuild from **public microdata**, **NOT** from derived pair tables  
**Target file:** `sca_micro_2015_2023_pilot.csv`  
**SHA-256:** `8f74769178d2918818906ac41a99d795131af4592b17beefa4b1d2ef05771dcf`  
**Rows:** 63,748 · **Cols:** 18 · **Download time (UTC):** 2026-09-09T17:02:03Z  

---

## 0. Goal

Reproduce the **exact public SDA customized-subset extract** used for M1-LONG pilot/v2/v2.1 analysis, so an independent agent can:

1. Download the same (or bit-equivalent) microdata from SDA  
2. Run observatory scripts from that CSV  
3. Compare funnel counts (N1=5793, N3=1424, etc.), labels, and diagnostics  

Do **not** start from `m1long_pairs_origin_le2019.csv` or other derived pair tables.

---

## 1. Public entry points (not homepage-only)

| Step | URL | Role |
|------|-----|------|
| Cross-section archive landing | https://data.sca.isr.umich.edu/sda.php | Points to SDA subset tool; usage agreement |
| **SDA UI used for this extract** | https://sda.umsurvey.org/sda-public/cgi-bin/hsda?setupfile=harcsda&datasetname=sca&ui=1 | Customized subset UI |
| SDA root | https://sda.umsurvey.org/ | Archive hub |
| Sequential variable list | https://sda.umsurvey.org/sca/Doc/scax01.htm | Confirms IDPREV/DATEPR/… |
| Detailed codebook | https://sda.umsurvey.org/sca/Doc/sca0001.htm | SAMPLE/METHOD/PAGO/PEXP codes |
| Data updates / revisions log | https://data.sca.isr.umich.edu/fetchdoc.php?docid=80741 | IDPREV2/DATEPR2/SAMPLE/METHOD history |

There is **no stable permanent raw-file URL**. SDA issues a **session-temporary** download (commonly under `https://sda.umsurvey.org/tmpdir/...`). Always re-extract via the UI; compare hashes afterward.

---

## 2. Exact variable list requested

Select **exactly these 18 variables** (order as observed in the CSV header):

```
CASEID, YYYYMM, ID, IDPREV, DATEPR, IDPREV2, DATEPR2, SAMPLE, METHOD,
PEXP, PAGO, WT, WT_HH, AGE, SEX, EDUC, REGION, ICS
```

SDA reported **18 variables** and **63,748 cases** for this subset.

**Not included in this extract** (still OPEN for later re-extract if needed): income, employment, party, household size, `WT_QUINTMED`, `PAGOR*`, `ICC`/`ICE` person-level, etc. Do **not** invent proxies.

---

## 3. Year window

| Filter | Value |
|--------|-------|
| SDA case/selection filter requested | `YYYYMM(201501-202312)` |
| Observed min/max in file | 201501 – 202312 |
| Distinct months | 108 |

---

## 4. METHOD / SAMPLE filters at **extract** time

| Filter | Applied in SDA download? | Observed in file |
|--------|--------------------------|------------------|
| **METHOD** | **None** (no METHOD restriction in the SDA request) | `METHOD=1` for **all** 63,748 rows (telephone only in this year window) |
| **SAMPLE** | **None** (all SAMPLE codes kept) | Codes present: `2,3,4,5` with counts 295 / 38538 / 19836 / 5079 |

Downstream **analysis** scripts apply SAMPLE logic (fresh / reint2 / reint3) and origin-year caps; those are **not** SDA download filters. See `srp/m1/build_m1_long_pilot.py` (`SAMPLE_FRESH={1,3,6}`, `SAMPLE_REINT2={2,4,7}`, `SAMPLE_REINT3={5,8}`) and `build_m1_long_v2.py` (`ORIGIN_MAX=201912`).

---

## 5. Download steps used (UI recipe)

1. Open **SDA UI**: https://sda.umsurvey.org/sda-public/cgi-bin/hsda?setupfile=harcsda&datasetname=sca&ui=1  
2. Choose **Customized subset** / download path for dataset `sca`.  
3. Select the **18 variables** listed in §2 (no extras).  
4. Set case selection / filter: **`YYYYMM(201501-202312)`**.  
5. Do **not** add METHOD or SAMPLE restrictions.  
6. Output format: **CSV**.  
7. Download the session file; save as:
   `data/raw/michigan_sca/microdata/sca_micro_2015_2023_pilot.csv`  
8. Record download timestamp (UTC) and compute sha256.  
9. Confirm SDA summary ≈ **18 variables, 63,748 cases**.

**Original download timestamp:** 2026-09-09T17:02:03Z  
**Original sha256:** `8f74769178d2918818906ac41a99d795131af4592b17beefa4b1d2ef05771dcf`

If a re-extract differs in hash (SDA revisions / floating session), keep both hashes and diff row counts / value_counts for SAMPLE/METHOD/YYYYMM before proceeding.

---

## 6. Local provenance pointers

| Artifact | Path |
|----------|------|
| This recipe (observatory) | `data/raw/michigan_sca/microdata/PROVENANCE.md` (summary) + this Hub doc |
| CSV | `data/raw/michigan_sca/microdata/sca_micro_2015_2023_pilot.csv` |
| Codebook excerpt (IDPREV) | `data/m1/v2/codebook_idprev_excerpt.md` |
| Builder scripts | `srp/m1/build_m1_long_pilot.py`, `build_m1_long_v2.py`, `build_m1_long_selection_diag.py`, `build_m1_long_selection_ipw_v2.py` |

**ToS:** Public UM SCA SDA research use. Treat microdata as restricted research-use; do not redistribute publicly via Hub (Hub publishes hashes + scripts + JSON only).

---

## 7. Replication checklist (independent agent)

1. Re-extract via §5; hash CSV.  
2. Run linkage/pilot/v2 scripts from that CSV (not from pair tables).  
3. Pre-declare tolerances, then compare: eligible N1, retained N3=1424, unweighted 3×3, selection funnel, label/missing codes, IPW balance shares.  
4. Outcome enum (canonical): `REPLICATED` / `REPLICATED_WITH_DIFFERENCES` / `FAILED_REPLICATION`.  
5. M1 remains UNKNOWN / unscored.

---

## 8. Hub URL

- This recipe: `/docs/wave2b-m1-long-sda-replication-recipe`