Best Children’s Hospitals: Changes Coming in New Rankings

Best Children’s Hospitals: Changes Coming in New Rankings

Sharp-eyed users of our Best Children's Hospitals rankings will note the presence of new data points when the 2012-13 rankings are published June 5. The addition of these data points, along with a change in how we factor in each hospital's reputation as determined by a survey of pediatric specialists, reflect the latest refinements to the methodology behind our rankings.

We take a conservative approach to changing the Best Children’s Hospitals methodology. Any modification affects the rankings, and consequently might alter a parent’s choice of hospital for a sick child. Data points are most often added when previously unavailable clinical data emerge that can, and therefore should, help fill in the portrait of hospitals’ quality and safety.

In the forthcoming rankings, for example, one newly added measure evaluates bloodstream infections caused by urinary catheters. Children are at higher risk for such infections than adults are. The pathway into the body is shorter and more direct, giving a child’s immune system, which is less developed than an adult’s, less time to identify and kill invading microbes.

The reputational change lowered the tendency of hospitals that got the highest number of “votes” from surveyed doctors to float to the top of the rankings. We believe a hospital’s reputation among physicians, as determined by a well-conceived, well-executed survey, says something worthwhile about quality, but we also recognize its subjectivity and limitations. Blunting the impact of high reputational scores levels the playing field by giving hospitals with solid clinical data more opportunity to achieve high rankings.

The weight of reputation in a hospital’s overall score will not change. It will still be worth 25 percent, or 25 points in a hospital’s 0 to 100 score. But in past years, the hospital with the highest reputational score in cancer or other specialty got credit for all 25 points, even if it was nominated by considerably fewer than 100 percent of the specialists who responded to the survey. (This is a standard statistical practice, called normalization.) Points awarded to hospitals with lesser reputations depended on their scores compared with the top scorer. If Hospital A had the highest reputational score (say 75 percent) and Hospital B’s score was 45 percent, Hospital A got 25 points and Hospital B got 45/75ths as many, or 15 pointsâ€"a difference of 10 points.

In the 2012-13 rankings, however, all hospitals get the number of points indicated by their reputational score. Hospital A from the example above would receive .75 x 25, or 18.75 points, towards its overall score. Hospital B would get .45 x 25, or 11.25 points. The difference between them has narrowed to 7.5 points. Nor is any hospital ever likely to receive the full 25 points. It would require every physician responding to the survey to name the hospital. That has never happened.

With continued weighting of 25 percent, reputation still has significant power, of course. Across the 10 specialties for which we publish rankings, the mix of hospitals in the top dozen or so, most of which have sizable reputational scores, remains fairly stable compared with the current rankings.

Reputation will be treated similarly in the 2012-13 Best Hospitals rankings, which will be published in July.