“We all know that data based on insurance claims (administrative data) can be inaccurate and incomplete. NSQIP gives us what everybody wants – risk-adjusted, clinically relevant, outcomes data.”LaMar McGinnis, MD, FACS Emory University, Atlanta, GA
Data Collection, Analysis and Reporting
A trained Surgical Clinical Reviewer (SCR) collects American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP®) data. ACS provides SCR training for participating hospitals, ongoing education opportunities and auditing to ensure data reliability. Preoperative through 30-day postoperative data is collected on randomly assigned patients and entered online in a HIPAA-compliant, secure, web-based platform that can be accessed 24 hours a day. Built-in software checks provide guidance, and the ACS technical and clinical support staff provides ongoing assistance for hospitals.
ACS NSQIP relies on timely and accurate data. SCRs receive extensive initial training prior to starting data collection and ongoing training via continuing education modules, regular conference calls and an annual conference. Many checks and security features are built into the software, and SCRs are provided with support tools.
ACS NSQIP monitors accrual rates and data sampling methodologies and conducts Inter-Rater Reliability Audits on a random basis. Hospitals that are flagged by internal diagnostics are also audited. The ACS NSQIP training, data collection and auditing process has been shown to be highly reliable. A study in the January 2010 Journal of the American College of Surgeons found the training and audit procedures ACS uses have been reliable since ACS NSQIP’s inception and that reliability has improved each year.
Hospitals enrolled in ACS NSQIP receive their data in various useful formats:
A comprehensive report is prepared twice a year for administrators and surgical services staff to compare their risk-adjusted surgical outcomes to other participating sites. Risk-adjusted 30-day morbidity and mortality outcomes are computed for each participating hospital using hierarchical modeling and are reported as odds ratios (OR). This number is a ratio that represents the estimated odds of a complication or event happening in a specific hospital compared to the estimated odds of that event happening in all hospitals in ACS NSQIP. An OR of 1.0 means the hospital is doing as expected. A number greater than 1.0 means the hospital is doing worse than expected and a number less than 1.0 means the hospital is doing better than expected.
To supplement the Semiannual Reports, participating hospitals receive an Individual Site Summary Report and PowerPoint presentation, each with pre-populated, site-specific data.
Authorized users can view daily site-specific reports as well as those comparing their metrics to national averages. In this way they can monitor continuous improvement as desired between the more formal report cycles. These continuously updated reports include:
- Preoperative Risk Factors. Summary analysis and detailed preoperative patient demographics, risk factors, and laboratory data.
- Preoperative Occurrence Reports. Inpatient, outpatient, and 30-day morbidity analysis and details in three different reports.
- Mortality Report. Deaths occurring up to and including 30 days after surgery summary and detail reports.
- Surgical Site Infection. Surgical site infection outcomes stratified by wound classification.
- Patient Variable Statistics. Data-point analysis and annual trends.
- Database Statistics. Site-specific data such as case volume and types of procedures in three different reports.
- Occurrences. Comparison of patient populations with and without specific occurrences.
- Physician-Specific. Blinded site-specific physician outcomes reports.
The reports help hospitals identify areas where they may be underperforming. Hospital quality committees use the report findings as the basis for quality improvement action plans to re-engineer workflows, foster and improve internal education, and develop clinical performance improvement initiatives.
ACS NSQIP helps hospitals prevent the preventable. A study in the Annals of Surgery involving 118 ACS NSQIP hospitals concluded that the program helped each hospital prevent between 250 to 500 complications per year1. In addition, 82 percent of those hospitals saw improvement in morbidity levels and 66 percent saw improvement in mortality levels. Hospitals that have significantly improved their performance or sustained excellent performance over time are asked to share their experience with ACS NSQIP. This feedback is combined with the data collected during structured site visits to produce a continually updated set of “best practices” that is disseminated to all participating ACS NSQIP sites across the country in the published annual report.