Data Accuracy

Leapfrog’s Online Hospital Survey Tool requires hospitals to “check for data review warnings” before they are able to submit a Survey. When a hospital clicks this button on the Survey Dashboard, sections that have been completed and affirmed will be scanned for potential data entry errors and inconsistencies. For example, if a hospital reports licensed and staffed ICU beds in Section 1 Basic Hospital Information, and then reports not caring for patients in an ICU in Section 5 ICU Physician Staffing, a “data review warning” will be generated. The hospital will still be able to submit their Survey but will be contacted via email by the Help Desk to either (1) correct the error or (2) document that the original response was correct. The data review warning system gives hospitals an opportunity to correct potential errors immediately – while they are still in the Online Hospital Survey Tool.

If a hospital receives a "data review warning," they can either:

• Review their response and make corrections on the spot. Hospitals will need to re-affirm any updated sections before they attempt to submit them again. 
• Print their “data review warning” and investigate the issue. Hospitals are encouraged to contact the Help Desk if they have any questions.

Upon the receipt of a hospital’s submitted Leapfrog Hospital Survey, The Leapfrog Group conducts an extensive Data Verification of each response submitted in the Survey. The Leapfrog Group conducts this Data Verification monthly beginning with June 30 submissions until the Survey is taken offline on January 31. For hospitals that submit the Survey by June 30, data verification messages are emailed to the primary Survey Contact, secondary Survey Contact (if applicable) and System Contact (if applicable) on or around July 12, when Survey Results are first published on the Hospital Details Page. After the June 30 deadline, data verification messages are emailed to Survey Contacts within the first 5 business days of the month AFTER Survey submission (i.e., messages will be sent within the first 5 business days of August for Surveys submitted by July 31).

To identify potential data entry or reporting errors, quantitative responses are assessed using empirically driven, normative data quality thresholds. The data quality thresholds are constructed with both high and low benchmarks of acceptability. Data quality thresholds for each question are determined from both external data sources (e.g., state quality reports, other national performance measurement entities) and historical Leapfrog Hospital Survey data. The Data Verification also includes assessing qualitative responses for potential data entry errors and inconsistencies. Applying this review methodology, The Leapfrog Group assigns each response into one of three categories:

• Category C: The response provided is plausible and within the data quality thresholds; Leapfrog will not contact the hospital. 
• Category B: The response provided is plausible, but it falls outside the data quality thresholds; Leapfrog will contact the hospital and action from the hospital is requested.
• Category A: The response provided is considered implausible, given that the response falls far outside of the data quality thresholds and appears to have been provided with the intent to mislead; Leapfrog will contact the hospital and action from the hospital is required.

Hospitals that receive a Category B Data Verification message at the beginning of the month for any measure will be asked to review the associated Survey questions and measure specifications to ensure the submitted responses are correct. Hospitals are asked to (1) contact the Help Desk to confirm the original responses are correct so we can discontinue the monthly Data Verification message or (2) to update and resubmit the Survey if a correction is needed. 

Hospitals that receive a Category A Data Verification message at the beginning of the month for any measure will have until the end of that same month to contact the Help Desk to either (1) document that the original response was correct or (2) correct the data entry or reporting error, or they will be publicly reported as “Pending Leapfrog Verification” for that measure. This term is used to indicate that the hospital has self-reported Survey responses that are under further review by Leapfrog.

If any Category A Data Verification messages are not resolved by January 31 (when the Online Hospital Survey Tool is taken offline), the entire Survey will be decertified and all measures will be publicly reported as “Declined to Respond.”

In the interim (that is, prior to Leapfrog's receipt of a hospital’s response to a Category B Data Verification message or prior to the end of the month for Category A Data Verification messages), the original Survey Results are publicly reported on Leapfrog’s website. 

When Leapfrog is contacted about a particular Survey response or Leapfrog representatives are made aware of a potential error in reporting, Leapfrog does investigate these concerns and the hospital’s Survey is subject to decertification.

In addition to the pre- and post-submission protocols described above, Leapfrog also randomly selects hospitals to submit documentation to verify the accuracy of various responses from each Section of the Leapfrog Hospital Survey on a monthly basis. Hospitals must respond to the documentation request within 30 days or risk having the section publicly reported as "Pending Leapfrog Verification." If any Monthly Verification Request is not responded to by January 31 (when the Online Hospital Survey Tool is taken offline), the entire Survey will be decertified and all measures will be publicly reported as “Declined to Respond.”

On-site Data Verification builds on Leapfrog’s robust protocols for reviewing Survey responses. In 2024, Leapfrog announced a new partner, MetaStar, will be administering On-site Data Verification in hospitals and ambulatory surgery centers. Hospitals that submitted a 2023 Leapfrog Hospital Survey will be eligible for On-Site Data Verification in 2024. Hospitals selected for On-site Data Verification will be contacted by MetaStar in June. On-Site Data Verification will take place in September and October.

MetaStar is an independent, nonprofit quality improvement organization with more than 50 years of experience, dedicated to effecting positive change in health and health care to achieve optimal health for all. Engaging with government agencies, health plans, and various stakeholders, MetaStar delivers a comprehensive range of services tailored to the unique needs of communities, payers, and healthcare organizations. MetaStar achieves impact by combining our long-standing reputation as a trusted advisor with evidence-based practices to drive improvement and accountability for quality healthcare.