Part 2: Methodological Commentary on Population Coverage and External Validity in U.S. Shelter Dog Census Research
Download the full version:
12. National Benchmarking Requires Measurement Beyond Shelter Admissions
A national framework intended to characterize companion animal homelessness cannot be limited exclusively to animals that successfully enter formal shelter custody. Shelter admissions are an important and measurable component of the animal welfare system, but they represent only one pathway through which animals requiring intervention are managed.
The broader system includes government-operated shelters, government-contracted shelters, nonprofit shelters, rescue organizations, foster networks, community-cat programs, finder-retention arrangements, diversion programs, and other community-based interventions. It also includes animals for whom shelter admission is requested but declined, deferred, redirected, or otherwise not completed.
Accordingly, the relevant population can be conceptualized as extending across multiple pathways:
Community population requiring intervention → requests for assistance → shelter admission, rescue intervention, finder retention, foster placement, diversion, community-based services, or continued presence in the community.
A shelter-based dataset observes some of these pathways in considerable detail once an animal enters the reporting organization. It may provide little or no information, however, about animals whose trajectories occur entirely outside participating organizations.
This distinction has implications for national benchmarking. A benchmark derived from shelter admissions may appropriately characterize recorded shelter operations, provided organizational coverage and measurement comparability are adequately established. It should not automatically be interpreted as a benchmark for the broader prevalence of companion animal homelessness or community-level need.
A more comprehensive national measurement framework would therefore distinguish at least three related but analytically separate dimensions: the population entering formal shelter custody; the population receiving intervention outside shelter custody; and the population requiring assistance for which no formal intervention is documented.
The third population is inherently the most difficult to measure. Nevertheless, difficulty of measurement does not eliminate its relevance. Where admission policies, rescue capacity, diversion practices, or community programs change over time, the proportion of animals distributed among these pathways may also change. A declining shelter-admission population could therefore coexist with stable or increasing demand elsewhere in the animal welfare system.
For this reason, national benchmarking should increasingly incorporate measures of access to shelter admission and community-based animal welfare activity, rather than treating shelter admission as the complete observational boundary of companion animal homelessness.
13. Implications for Scientific Research and Public Policy
The distinction between recorded shelter activity and broader companion animal need has implications beyond methodological terminology. Shelter data are increasingly used to inform scientific conclusions, operational planning, grantmaking, resource allocation, prevention programs, and public policy.
For scientific research, the principal concern is the scope of inference. Findings derived from participating shelter organizations may provide valid information about relationships occurring within those organizations while remaining insufficient to establish corresponding trends in the broader population of homeless or at-risk companion animals.
For example, a reduction in recorded intake may reflect a reduction in animals requiring assistance, but it may also reflect restricted admission, delayed surrender, finder retention, diversion, direct rescue intervention, reduced shelter capacity, or combinations of these mechanisms. Without variables measuring these alternative pathways, their respective contributions cannot be determined from intake and outcome statistics alone.
This distinction also matters when shelter data are used to inform allocation of resources. If recorded intake is treated as a direct measure of community need, jurisdictions with declining admissions could appear to require fewer resources even where substantial demand has shifted to residents, nonprofit rescues, foster networks, veterinary providers, or community-based programs. Conversely, jurisdictions maintaining comparatively open admission policies may appear to have greater animal population problems partly because a larger proportion of community need becomes visible in their shelter data.
Similar considerations apply to evaluation of prevention programs. Changes in spay/neuter access, veterinary capacity, breeder activity, abandonment, housing instability, and other upstream conditions may affect community animal populations without being immediately or completely observable through shelter admissions. Evaluation of such programs therefore benefits from multiple indicators rather than reliance on shelter intake alone.
For legislative and administrative interpretation, measurement boundaries should likewise be explicit. Policymakers using shelter statistics to evaluate animal welfare conditions should be able to determine what population is represented, what population is excluded, how participation has changed, and whether operational policies influence entry into the measured population.
The objective is not to diminish the value of shelter data. Rather, it is to define their evidentiary scope accurately. Shelter datasets are essential sources of operational information. Their usefulness for broader public-policy decisions increases when they are combined with transparent coverage measures, standardized methodologies, historical continuity, and information about animals receiving or seeking assistance outside formal shelter custody.
14. Recommendations for Future Research
Future national companion animal research could substantially improve population inference, reproducibility, and comparability through several methodological developments.
First, establish a defined organizational sampling frame. A national or state-level registry of government-operated shelters, government-contracted shelters, nonprofit shelters, and rescue organizations would provide a denominator against which participation can be evaluated. In the absence of such a registry, researchers should document how the target population was constructed and the limitations of that estimate.
Second, report both organizational and intake-weighted coverage. The proportion of organizations participating does not necessarily correspond to the proportion of animals represented. A relatively small number of high-volume organizations may account for substantial intake, while numerous smaller organizations may account for comparatively little—or the reverse. Reporting both measures would provide a more informative assessment of dataset coverage.
Third, characterize participation geographically and organizationally. Coverage should be evaluated by state, organization type, governmental status, intake volume, and other relevant characteristics to identify systematic differences between participating and non-participating organizations.
Fourth, standardize core definitions and calculations. Intake, admission, declined admission, deferred admission, euthanasia, transfer, return-to-field, finder retention, diversion, and composite measures such as LRR should use transparent definitions and denominator rules. Where alternative methodologies remain necessary, the methodology associated with each observation should be identifiable.
Fifth, incorporate shelter-access variables. National datasets should include standardized measures of managed-intake status, appointment requirements, periods of restricted or suspended admission, admission requests, declined and deferred admissions, waiting-list volume and duration, finder-retention activity, and diversion practices. These variables would permit researchers to distinguish changes in community demand from changes in access to shelter custody.
Sixth, measure intervention occurring outside shelters. Rescue organizations, foster networks, community-cat programs, direct-placement programs, and other community interventions represent meaningful components of the animal welfare system. Developing standardized methods for measuring these pathways would provide a more complete picture of animals receiving assistance without formal shelter admission.
Seventh, preserve permanent historical datasets. Static, versioned archives should be maintained alongside dynamic dashboards. Historical source data should remain accessible for replication, longitudinal analysis, and methodological reconciliation.
Eighth, document methodological changes over time. Changes in definitions, categories, denominators, exclusions, reporting systems, and organizational participation should be versioned and dated. Researchers should be able to determine whether observations from different periods are methodologically comparable.
Finally, future research should explicitly distinguish among shelter operational performance, shelter-system utilization, and companion animal population conditions. These constructs overlap, but they are not interchangeable. Measuring them separately would permit stronger analysis of the relationships among community need, access to services, shelter operations, and animal outcomes.
15. Discussion and Conclusion
The central methodological concern identified in this Commentary is not necessarily the statistical analysis performed within the available analytical dataset. Rather, it is the extent to which that dataset permits inference about the broader U.S. shelter system and the larger population of companion animals experiencing homelessness or requiring intervention.
The study provides a useful analysis of shelter census dynamics within a large subset of organizations participating in Shelter Animals Count (SAC). Within that analytical population, relationships among recorded intake, census, adoptions, transfers, and other outcomes can be examined quantitatively.
What the analysis does not establish, however, is that the participating subset constitutes a representative model of the U.S. shelter system or of the broader national population of companion animals experiencing homelessness.
That distinction is fundamental.
A large number of observations does not independently establish population representativeness. External validity depends upon knowing the population from which a sample is drawn, determining the proportion and characteristics of that population represented in the sample, and evaluating whether participating and non-participating organizations differ in ways relevant to the outcomes being studied. As discussed throughout this Commentary, no comprehensive national sampling frame identifying all potentially relevant government shelters, government-contracted shelters, nonprofit shelters, and rescue organizations is available against which SAC participation can be fully evaluated.
The study itself acknowledges limitations associated with self-reporting and participation bias. Our independent examination of California illustrates how consequential those limitations may be in practice. California lacks a comprehensive current statewide registry of its sheltering organizations, and comparison with available SAC participation information identified substantial gaps in organizational coverage.
California also demonstrates a second and separate limitation: the boundary of shelter data is not necessarily the boundary of companion animal homelessness.
Animals requiring intervention may be denied or deferred admission, remain with finders, be redirected to nonprofit rescues, enter foster or community-based systems, or otherwise remain outside formal shelter custody. The San Jose case study demonstrates that this is not merely theoretical: within the examined population, a substantial number of animals presented for assistance were not documented as entering shelter custody, and the ultimate disposition of many could not be established from the available shelter datasets.
At the same time, historical records from several major California shelter systems demonstrate substantial contractions in formal intake compared with earlier periods. Those declines do not independently establish whether the underlying community needs declined proportionally. They demonstrate why admission access, managed-intake practices, declined and deferred admissions, finder retention, diversion, rescue intervention, and other pre-admission mechanisms must be considered when shelter intake is interpreted as an indicator of the broader homeless-animal population.
The analytical boundary is therefore critical:
Shelter datasets generally begin when an animal enters the measured system. Companion animal homelessness begins before that boundary.
An animal can be homeless, abandoned, found, surrendered, or otherwise require intervention regardless of whether a participating organization ultimately admits and records that animal. Consequently, trends in recorded shelter admissions and trends in companion animal homelessness are related empirical questions, but they are not interchangeable measures.
This does not diminish the scientific value of shelter data. It defines more precisely what those data can establish.
The study's findings can reasonably be characterized as relationships observed among the participating SAC organizations included in the analytical dataset. Extending those findings to a “national reference,” however, requires an additional evidentiary step: demonstrating that the observed organizational population and measured animal population adequately represent the national populations to which that reference is intended to apply.
The distinction is particularly important because the analytical period of 2022–2024 occurred during a period in which shelter admission practices, managed-intake systems, diversion programs, finder-retention approaches, and other operational conditions were changing in many jurisdictions. Without corresponding measures of shelter accessibility and animals remaining outside formal custody, the mechanisms producing changes in recorded intake cannot be fully distinguished using intake, census, and outcome variables alone.
Similarly, national comparison depends not only upon organizational participation but upon measurement comparability. Variation in definitions, denominators, exclusions, historical availability, and performance calculations can limit the extent to which apparently equivalent observations are actually measuring equivalent constructs. The California examples reviewed in this Commentary demonstrate why standardized definitions, methodological documentation, and preservation of underlying operational data are essential for reproducible longitudinal and cross-jurisdictional research.
Requested Clarification to the Publisher
Given these limitations, we respectfully recommend that the publisher and authors clearly identify the population to which the study's findings apply and add an explicit limitation or disclaimer to the published article wherever its findings are characterized as a “national reference” or otherwise presented as representative of U.S. animal shelters generally.
Such a clarification could state, in substance:
The analysis is based on organizations voluntarily reporting data to Shelter Animals Count and should not be interpreted as establishing a nationally representative sample of all U.S. animal shelters or of the broader population of homeless companion animals. The dataset measures animals recorded by participating organizations and may not capture animals remaining outside formal shelter custody because of admission restrictions, diversion, finder retention, rescue intervention, or other community-based pathways.
This clarification would not invalidate the statistical relationships identified within the participating organizations. Rather, it would distinguish internal findings within the observed dataset from broader claims of national representativeness and population-level inference.
That distinction is essential to the accurate interpretation of the research, particularly when published findings may subsequently be cited in scientific literature, policy discussions, funding decisions, operational planning, or public descriptions of national animal-welfare trends.
National shelter research can and should continue to develop. SAC and other large-scale reporting systems can provide important analytical resources. Their scientific value will increase as organizational coverage becomes more measurable, definitions become more standardized, historical datasets remain reproducible, and information about shelter accessibility and animals outside formal custody becomes incorporated into population analysis.
But the evidentiary boundary should remain clear:
A large voluntary dataset is not automatically a national dataset. Aggregating thousands of observations from a partially observed system may increase statistical precision within the observed sample, but it does not, by itself, resolve unknown coverage, self-selection, measurement heterogeneity, or populations systematically absent from the dataset.
And ultimately:
The strength of a national benchmark depends not only on how many observations it contains, but on whether the population being measured is known, sufficiently represented, consistently defined, and appropriate to the national phenomenon the benchmark is intended to describe.
Sources:
Full version:
Comments