Law & Economics

Methods Toolkit

A working environment for turning legal questions into measurable claims, defensible comparisons, reproducible data rules, and bounded conclusions.

01

Select

Match the claim and data structure to a candidate design.

02

Record

Turn the research logic into a versioned design memo.

03

Govern

Control provenance, changes, access, responsibility, and release.

01

Orientation

From legal problem to auditable design

Method choice follows the claim, legal population, mechanism, measurement, and source of comparison. Complete each stage in order; unresolved matters remain TODO rather than being filled by inference.

  1. 01

    Question and scope

    Classify the claim, then define jurisdiction, legal population, unit of analysis, and time window.

    Output
    A research question and scope statement.

  2. 02

    Mechanism and measurement

    Connect the rule to behavior, then define outcomes, exposure, sources, coding rules, and missing values.

    Output
    A mechanism chain and variable dictionary.

  3. 03

    Comparison and threats

    State the estimand and counterfactual, then audit selection, timing, spillovers, measurement, and dependence.

    Output
    An identification and threat-to-inference statement.

  4. 04

    Checks and reporting

    Link diagnostics to named threats and state the empirical, external-validity, and normative boundaries.

    Output
    A robustness plan and bounded conclusion.

02

Design selection

Choose a design by claim, data, and assignment

The result is a design starting point, not automatic methodological approval. The recommendation changes when the evidentiary claim, data structure, or source of treatment variation changes.

1. What claim must the evidence support?
03

Design record

Build a methods memo before analysis

Work through five stages, preserve unresolved decisions as TODO, and export a structured Markdown record into the versioned project documentation.

Local working draft

Preparing local draft

0%

0 of 18 decisions recorded

Stage 1

Question and mechanism

Define the claim, population, legal setting, mechanism, and the observation that could challenge it.

0 of 4 recorded

Blank fields remain TODO in the exported memo.

Browser storage is a convenience copy, not a project archive. Download or copy the memo into the versioned project documentation before analysis or team review.

04

Project controls

Make the research reproducible and governable

Reproducibility concerns whether the evidence can be reconstructed. Governance concerns whether materials, discretion, responsibility, and release are handled under explicit authority and review.

Reproducibility controls

Make the path from source material to every reported table, figure, and claim inspectable and rerunnable.

  1. 01

    Source and provenance register

    Origin, collection date, access terms, coverage, versions, checksums where useful, and every transformation from raw to analytic data.

  2. 02

    Scripted analysis sequence

    Separate scripts for cleaning, merging, variable construction, descriptives, estimation, diagnostics, and output generation.

  3. 03

    Computational environment

    Software, package versions, seeds where relevant, locale or encoding dependencies, and the command or workflow that rebuilds outputs.

  4. 04

    Decision and deviation log

    Dated changes to sample restrictions, variables, fixed effects, clustering, windows, estimators, and hypotheses, with reasons and owners.

  5. 05

    Independent reproduction check

    A clean rerun by another team member or environment, with unresolved differences recorded before release.

Research governance controls

Make legal authority, sensitive-data handling, responsibility, conflicts, and release decisions explicit.

  1. 01

    Authority and permitted use

    Legal or contractual basis for collection and analysis, platform restrictions, consent where applicable, and downstream-use limits.

  2. 02

    Privacy and security

    Sensitive fields, minimization, pseudonymization, access groups, storage locations, retention periods, incident response, and deletion rules.

  3. 03

    Roles and accountability

    Named responsibility for data custody, coding, analysis, verification, approvals, authorship decisions, and conflict resolution.

  4. 04

    Conflicts and researcher discretion

    Funding, institutional interests, access-provider influence, analyst degrees of freedom, and separation between exploratory and confirmatory work.

  5. 05

    Disclosure and release review

    Privacy, confidentiality, legal-risk, disclosure-control, licensing, documentation, and archive checks completed before sharing.

Minimum project structure

Keep data, code, output, documentation, and paper separate

data/raw
Read-only source materials; never overwritten by cleaning or recoding.
data/processed
Script-generated intermediate and analytic datasets with documented lineage.
code
Ordered scripts for cleaning, merging, construction, analysis, diagnostics, and outputs.
output
Script-generated tables, figures, model objects, logs, and validation reports.
docs
Design memo, data dictionary, coding protocol, sample flow, decision log, and governance record.
paper
Manuscript sources that import generated outputs rather than manually copied estimates.

Review gates

Do not advance without a reviewable record

  1. Gate 1

    Before collection or access

    May the team obtain and use the material for the stated purpose?

    Exit evidence
    Scope note, authority or access basis, data-management plan, source register, role owners, and unresolved items marked TODO.

  2. Gate 2

    Before estimation

    Can another researcher reconstruct the sample, measures, comparison, and planned inference?

    Exit evidence
    Frozen design memo, variable dictionary, sample-flow rules, coding audit, threat register, analysis sequence, and dated deviations.

  3. Gate 3

    Before release

    Do the claims, files, and disclosures survive an independent verification and governance review?

    Exit evidence
    Clean rerun, output reconciliation, bounded conclusions, privacy and legal review, disclosure controls, documentation, and archive package.

05

Method reference

Minimum conditions for defensible use

Thirteen specifications separate data construction, observational adjustment, causal identification, and institutional interpretation. Open a card to review its minimum evidence and reporting boundary.

Data and measurement

Build an auditable source universe and convert legal materials or field evidence into defensible measures.

Legal data audit and structured coding

Description of doctrine, enforcement, case populations, institutional practices, and text features.

Requires
A defined source universe, unit of coding, inclusion rules, codebook, and an audit trail from source to variable.
Assumptions
Observed materials represent the stated population; coding categories preserve the legal concept; missingness is understood.
Common failure
Convenience sampling, silent duplicate removal, unstable legal categories, or reporting percentages without denominators.
Report explicitly
Source coverage, search or collection rule, sample flow, coder training, agreement measure, conflict resolution, and missingness.

Legal text analysis and NLP

Scalable measurement, description, classification, prediction, and pattern discovery in a defined legal corpus; not a causal effect by itself.

Requires
A documented corpus, unit of text, preprocessing pipeline, construct-valid labels or features, a validation set, and reproducible code and model settings.
Assumptions
Corpus coverage matches the stated population; labels and features represent the legal concept; train and test information do not leak across cases or time.
Common failure
Using a convenience corpus, treating opaque embeddings as self-validating measures, reporting accuracy alone, ignoring class imbalance, or leaking later information into training.
Report explicitly
Corpus construction, exclusions, preprocessing, labeling protocol and reliability, model and version, split strategy, class distribution, validation metrics, error analysis, and temporal or jurisdictional transport limits.

Surveys, interviews, and mixed methods

Evidence on attitudes, implementation, professional practice, and mechanisms; mixed methods can test convergence and explain disagreement across evidence types.

Requires
A target population, sampling or recruitment strategy, instrument or interview protocol, consent and privacy plan, coding process, and an explicit integration design.
Assumptions
Participants can report the relevant experience; nonresponse, selection, recall, and social-desirability risks are bounded; qualitative interpretation is auditable.
Common failure
Generalizing from convenience recruitment, using leading questions, cherry-picking quotations, claiming saturation without a transparent process, or placing qualitative and quantitative results side by side without integration.
Report explicitly
Sampling frame, recruitment, response or participation rate, instrument and pretest, ethics and access, interviewer role, coding and reliability, saturation if claimed, integration logic, and limits.

Observational analysis

Describe conditional relationships while keeping selection, overlap, dependence, and causal limits visible.

OLS, Logit, and cross-sectional association

Conditional means or probabilities associated with observed characteristics; not causation without additional design.

Requires
An outcome-appropriate functional form, legally meaningful measures, and enough variation in the focal variable.
Assumptions
Model specification is adequate for the stated association; observation dependence and influential cases are addressed.
Common failure
Calling a controlled association an effect, controlling for post-treatment variables, or interpreting odds ratios as probability changes.
Report explicitly
Outcome scale, reference groups, functional form, uncertainty, missing-data treatment, dependence structure, and substantive magnitude.

Matching, weighting, and propensity scores

Adjusted comparison of observed groups when the estimand, covariate set, and region of overlap are defined; not control of unobserved confounding.

Requires
A treatment or exposure definition, pre-treatment covariates, adequate overlap, a target estimand, and balance diagnostics.
Assumptions
Conditional exchangeability given measured covariates, positivity, consistency, and sufficiently reliable measurement of treatment, outcome, and confounders.
Common failure
Treating a propensity score as proof of causation, matching on post-treatment variables, retaining units without common support, or reporting only model fit.
Report explicitly
Target estimand, propensity model, matching or weighting algorithm, trimming rules, balance before and after adjustment, effective sample size, inference, and sensitivity to hidden bias.

Panel models and fixed effects

Within-unit associations after removing time-invariant unit differences and common time shocks.

Requires
Repeated observations, meaningful within-unit variation, correct time alignment, and a reason to use fixed effects.
Assumptions
Relevant time-varying confounding is addressed for the claimed interpretation; errors are handled at the dependence level.
Common failure
Using fixed effects as automatic causal identification, including variables with no within-unit variation, or clustering below assignment.
Report explicitly
Panel balance, fixed effects, within variation, sample restrictions, treatment timing, clustering level, and dropped observations.

Causal identification

Tie an effect claim to a legally grounded counterfactual, explicit assumptions, and design-specific diagnostics.

Difference-in-differences and policy event studies

Effects of a legal change for an exposed population relative to a credible comparison under a parallel-trends argument.

Requires
Correct legal dates, repeated outcomes, treated and comparison units, pre-treatment periods, and transparent cohort timing.
Assumptions
Parallel counterfactual trends, no material anticipation, limited spillovers, stable measurement, and interpretable treatment timing.
Common failure
A before-after comparison, an arbitrary control group, two-way fixed effects with unexamined staggered timing, or clustered errors at the wrong level.
Report explicitly
Estimand, estimator, cohort structure, event-time specification, pre-trends, anticipation window, spillovers, and assignment-level clustering.

Synthetic control

A post-intervention effect for one or a few treated jurisdictions or institutions relative to a weighted untreated donor pool.

Requires
A discrete intervention, comparable donor units, repeated outcomes, a substantial pre-treatment period, and limited spillovers or concurrent treatment.
Assumptions
A weighted donor combination can approximate the treated unit's untreated path; pre-treatment fit is informative; donors remain uncontaminated.
Common failure
Using a poor-fitting donor pool, hiding donor weights, selecting predictors after inspecting the result, or treating one placebo comparison as conventional significance.
Report explicitly
Intervention date, donor inclusion rules, predictors, pre-period, unit weights, pre-treatment fit, post-treatment gap, in-space and in-time placebos, and leave-one-out checks.

Interrupted time series

A change in the level or trend of a repeatedly measured outcome after a clearly dated legal intervention.

Requires
A stable outcome definition, many pre- and post-intervention observations, a defensible intervention date, and information on seasonality and concurrent shocks.
Assumptions
The pre-intervention trend is a credible counterfactual after accounting for time structure; no coincident event fully explains the break.
Common failure
Comparing only one pre and one post mean, ignoring autocorrelation or seasonality, choosing the break date after looking at the series, or omitting concurrent reforms.
Report explicitly
Outcome frequency, intervention and lag structure, trend specification, seasonality, autocorrelation correction, control series where available, alternative dates, and functional-form sensitivity.

Short-window market event study

The market valuation response to new, identifiable information around a legal or regulatory event.

Requires
A precise event timestamp, liquid price data, an expected-return model, and a defensible short event window.
Assumptions
The event conveys unanticipated information and no material confounding news explains the same return window.
Common failure
Long windows, overlapping events, data-snooped endpoints, or treating abnormal returns as direct welfare evidence.
Report explicitly
Event chronology, estimation window, event window, benchmark model, aggregation, inference method, and confounding-news screen.

Regression discontinuity

A local causal effect for units near a legally generated eligibility or assignment cutoff.

Requires
A running variable, known cutoff, discontinuous treatment probability, dense observations near the threshold, and no precise manipulation.
Assumptions
Potential outcomes and relevant covariates are continuous at the cutoff absent treatment; sorting is limited.
Common failure
Selecting a cutoff after inspecting outcomes, high-order global polynomials, sparse support, or generalizing far from the threshold.
Report explicitly
Assignment rule, bandwidth, local polynomial order, kernel, manipulation and continuity checks, plots, and local target population.

Instrumental variables

A treatment effect for units whose exposure changes because of a valid external instrument, under strong assumptions.

Requires
A legally and institutionally defensible source of exposure variation, a first stage, and a defined affected population.
Assumptions
Relevance, exclusion, independence, and the monotonicity or effect-homogeneity conditions needed for interpretation.
Common failure
Choosing an instrument because it predicts treatment, defending exclusion statistically, or ignoring direct institutional channels.
Report explicitly
Legal assignment mechanism, first stage, reduced form, exclusion argument, weak-instrument diagnostics, and the represented population.

Institutional interpretation

Explain doctrine and governance through source hierarchy, functional comparison, and bounded transfer.

Comparative institutional analysis

Structured comparison of how courts, regulators, firms, and private ordering perform similar governance functions.

Requires
Functional comparability, source hierarchy, enforcement context, ownership or market structure, and a portability framework.
Assumptions
Legal labels do not substitute for institutional function; differences in implementation and complements are made explicit.
Common failure
Comparing statutory text alone, treating one jurisdiction as a universal benchmark, or inferring effects from a few selected cases.
Report explicitly
Selection rationale, dimensions of comparison, legal sources, enforcement practice, institutional complements, and transfer limits.
06

Release checklist

What another researcher must be able to reconstruct

Use the final checklist to reconcile the legal construct, data lineage, model specification, inference, robustness record, and claim boundary before release.

Legal construct

  • Cite the rule, decision, regulatory action, or institutional source that defines exposure.
  • Record enactment, publication, effective, compliance, and enforcement dates separately.
  • Explain exemptions, thresholds, transitional rules, and changes in legal meaning over time.

Data and sample

  • Identify the source universe, collection date, unit of observation, linkage keys, and access restrictions.
  • Report inclusion and exclusion rules, duplicate handling, missingness, attrition, and the final sample flow.
  • Preserve raw data and separate cleaning, merging, variable construction, analysis, and output scripts.

Model and inference

  • State the estimand before the estimator and identify the comparison that supports it.
  • Record fixed effects, controls, transformations, windows, weights, and every sample restriction.
  • Cluster at the level implied by assignment or dependence; explain deviations and few-cluster limitations.

Robustness and interpretation

  • Link each robustness check to a named threat: measurement, timing, comparison, specification, or inference.
  • Distinguish statistical significance, substantive magnitude, legal relevance, and external validity.
  • Do not convert an empirical estimate directly into a welfare, rights, legality, or policy conclusion.

Reproducibility record

  • Provide an ordered, scripted path from unchanged raw sources to processed data, estimates, tables, and figures.
  • Record software and package versions, random seeds where relevant, source versions, and the command or workflow used for a clean rerun.
  • Keep a dated log of every change to samples, variables, windows, fixed effects, clustering, estimators, and planned analyses.

Governance and disclosure

  • State the authority and permitted purpose for data access, together with privacy, retention, security, and downstream-sharing limits.
  • Identify responsibility for data custody, coding, analysis, verification, release approval, and resolution of discrepancies.
  • Disclose funding, institutional interests, access-provider influence, conflicts, and any restriction that another researcher would need to assess.