Home » 1998PolicyStudies » Why The Quality of U.S. Health Care Must Be Improved

Why The Quality of U.S. Health Care Must Be Improved

NCHC Writer Update at May 21, 2023

Prepared for
The National Coalition on Health Care

By
Mark A. Schuster, M.D., Ph.D.
Elizabeth A. McGlynn, Ph.D.
Robert H. Brook, M.D., Sc.D.
RAND
Santa Monica, California

October 1997

TABLE OF CONTENTSI. PURPOSE

II. WHY IS IT IMPORTANT TO MEASURE QUALITY OF CARE?

III. HOW DO WE MEASURE QUALITY?

Structural Quality
Process Quality
1. Types of process quality
  - Technical process
    - Appropriateness
    - Adherence to professional standards of care
  - Interpersonal quality
2. Attributes of Good Process Measurement

Outcomes

Types of outcome measure
Attributes of Good Outcomes Measurement

IV. WHAT IS THE QUALITY OF THE CARE PROVIDED IN THE UNITED STATES TODAY?

V. HOW CAN AN ORGANIZATION USE INFORMATION ON QUALITY TO IMPROVE THE QUALITY OF CARE IT PROVIDES?

VI. WHAT SYSTEMS ARE USED FOR EXTERNAL QUALITY MONITORING IN THE UNITED STATES?
- Quality of Care Measurement Tools
  1. Systems that have been implemented
  2. Systems under development
Measurement Tied to Reimbursement
1. Blue Cross and Blue Shield of Minnesota’s Illness Outcome Group Reimbursement Program
2. Pacific Business Group on Health
Accreditation
Summary

VII. WHAT FACTORS ARE IMPORTANT FOR AN EFFECTIVE QUALITY MONITORING AND ASSESSMENT SYSTEM?
VIII. CONCLUSIONS

IX. TABLES
X. ACKNOWLEDGMENTS

XI. REFERENCES
I. PURPOSE In September 1996, President Clinton established the Advisory Commission on Consumer Protection and Quality in the Health Care Industry. The Commission is responsible for reviewing efforts to define, measure, and promote quality of health care and for helping build consensus on how to assure and promote quality of care. This report, funded by the bipartisan National Coalition on Health Care, is intended to provide the Commissioners, policymakers, health care providers, and the public with information that will facilitate discussion about quality of care.

We begin with a definition of quality of care. The Institute of Medicine has defined quality as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”

Good quality means providing patients with appropriate services in a technically competent manner, with good communication, shared decision making, and cultural sensitivity. In practical terms, poor quality can mean too much care (e.g., unnecessary tests, medications, and procedures, with associated risks and side effects), too little care (e.g., not receiving a lifesaving surgical procedure), or the wrong care (e.g., medicines that should not be given together, poor surgical technique).

In recent years, the health care system has been undergoing a dramatic transformation. New organizational structures and reimbursement strategies have created incentives that may lead to a reduction in quality of care. Concerns about such consequences have prompted a movement to assure that efforts to control costs will not sacrifice quality.

Some might doubt that the quality of health care delivered in the United States is anything but outstanding. There is good reason to be proud of our health care system. It is responsible for many important technological health care advances, and state-of-the-art health care is available in both large and small communities throughout the country. But just because outstanding care is available does not mean that it is always provided. Study after study has shown that many people are getting more care than they need and many are getting less than they need. This is true for different types of care and in different settings.

This paper will review why it is important to measure quality of care, how it is measured, what is known about the quality of care in the United States, how information on quality is being used, and which characteristics are important to ensure an effective national quality monitoring and assessment system.

II. WHY IS IT IMPORTANT TO MEASURE QUALITY OF CARE?

This section identifies five main reasons for measuring quality of care.

A. To help consumers and purchasers make informed choices about health care

Information on quality can help consumers make informed choices. Most consumers know little about the technical proficiency of the physicians and other health care providers they choose. Some ask friends for referrals. Some choose providers based on limited information in a health maintenance organization’s brochure, such as age, gender, and medical school. Some choose providers based on convenience of location. When selecting a health plan, consumers may compare price and covered services, but they can find it difficult to learn how well plans provide care in general or for particular conditions. Quality monitoring can provide such information to help consumers decide where and from whom to obtain care.

Information on quality can also help public and private group purchasers of care. Companies that provide health insurance for their employees must decide which health plans to make available. Similarly, state and federal governments must decide which health professionals can provide care to beneficiaries of government-funded health insurance. As concerns have increased about rising health care costs, many large purchasers have considered price of care as the primary factor in determining which health plans to offer. Yet, while it is tempting to believe that more efficient plans trim costs in health care while keeping essential and necessary services, research has not shown this to be true. Instead, studies reveal cost containment to be a blunt instrument that, by itself, results in the elimination of both necessary and unnecessary care. Quality assessment and monitoring provide the tools we need to balance cost and quality.

Nearly half of Americans who receive health insurance at work do not get to choose their health plan. Their employers choose their health plan for them. Therefore, if quality is to enter into the selection process, it must be at the employer level. Large purchasers have recently been demanding quality monitoring by the plans they select, but we do not know how and to what extent they use this information in selecting health plans, negotiating rates, or setting premiums.

B. To help physicians and patients make informed treatment and referral decisions

Information on quality is also useful for physicians and patients when making specific treatment and referral decisions. For example, studies have shown that carotid endarterectomy surgery for patients with symptomatic carotid stenosis of at least 60% (cleaning out clogged blood vessels to the brain that are narrowed by at least 60% and are causing symptoms) can improve survival. However, in many hospitals, the success rates are not as good as in the studies, which often include only the best clinicians and facilities and enroll patients who have no other diseases and who are likely to follow treatment instructions carefully. Because care provided in the “real world” may not have the same results as care provided under research conditions, information on local surgeons’ individual results will help primary care providers and patients determine which available surgeons at which hospitals have the best success rates and, given those rates, whether the potential benefits outweigh the risks. If the surgical success rates at available hospitals are low enough, a nonsurgical alternative, such as treatment with medications alone, may turn out to be a better option.

C. To help clinicians and health plans improve their care

Clinicians and health plans can use information concerning their own quality of care to improve the care they provide. Monitoring quality provides the opportunity not only to address inferior quality, but also to identify and learn from examples of superior quality. Health plans may find that they need to help individual clinicians improve their care, or they may identify organizational problems that hinder provision of high quality care. Clinicians and plans may welcome the opportunity to improve, and those that might not otherwise address quality problems may be motivated by public reporting to address them.

D. To determine the impact of new policies and systems

The health system is rapidly changing because of private market forces and public policies. However, we know relatively little about the impact of these changes on the health of the population. Sometimes we cannot even predict whether a change will increase or decrease quality because the change creates multiple, competing incentives for the various participants in the system.

With quality measurement systems, we can evaluate the impact of health system changes. For example, in 1983, the Health Care Financing Administration (HCFA) made a major change in Medicare financing of hospital care for the elderly. It had previously reimbursed hospitals on a cost basis. Starting in 1983, it established a prospective payment system (PPS) that paid hospitals an amount based largely on a flat rate per admission calculated for about 470 diagnosis related groups (DRGs). There was concern that quality would decline because the PPS created incentives to substitute lower-cost services for more expensive ones and to reduce the length of hospital stays.

HCFA commissioned a study to evaluate the effect of the PPS on quality. The study included a nationally representative sample of Medicare enrollees hospitalized for heart attack, stroke, congestive heart failure, hip fracture, or pneumonia. After implementation of the PPS, length of stay fell by 24%, but it fell for people who needed to be in the hospital (e.g., because they were still unstable) as well as for those who did not. The percentage of patients discharged with at least one medical instability rose by 43%. In addition to increasing the risk of mortality, the instabilities (e.g., incontinence, confusion) increased the amount of support needed at home. Despite the increased risk of death due to earlier discharge, there were no significant changes in mortality rates 180 days after admission, when adjusting for level of sickness at admission. This lack of impact on mortality rates appeared to be the result of counterbalancing forces: an increase in mortality due to higher instability at discharge was offset by a decline in mortality due to better quality of inpatient care (reflecting a time trend independent of the PPS). The study also examined the quality of care for Medicare enrollees hospitalized for depression before and after PPS. After PPS, they were more likely to have been hospitalized for appropriate reasons, to have received a more complete initial psychological assessment, to have received better management of psychotropic medications, and to have had fewer complications. In addition, there was no significant change in the percentage of patients having acceptable overall clinical status at discharge or in one-year mortality rates.

Currently, Medicaid beneficiaries in many states are being enrolled in managed care organizations. To determine what impact this shift has on people’s health, measurement tools could be used to compare the quality of care these patients receive in managed care settings and the quality of care they receive in traditional fee-for-service settings.

The health care system is now changing so rapidly that most studies of how well it provides care are already out-of-date. A systematic approach for collecting and measuring information on quality would facilitate the production of information that is more timely and representative of current delivery systems.

E. To provide clinical input to financial decisionmaking processes

Cost-cutting efforts create incentives for health plans to reduce the amount of care provided, which may actually improve the health of the population when useless or harmful care is eliminated. However, when necessary care is cut, health may decline. Quality of care criteria can help guide decisions about which type of care should be maintained and which type might be safely eliminated. Better decisions should result in the delivery of a more effective package of services.

III. HOW DO WE MEASURE QUALITY?

There are many approaches to measuring quality of care. Different approaches have different strengths and weaknesses, and some approaches work better in one setting than another. Quality can be evaluated based on three dimensions: structure, process, and outcomes. “Structural quality” refers to health system characteristics, “process quality” refers to what the clinician does, and “outcomes” refers to patients’ health. While producing good outcomes is the ultimate goal of the health care system, using outcome measures to assess quality is usually not the most effective approach for a variety of technical reasons (which we discuss below). Therefore, we often use structural and process measures instead. In this section, we review each type of measure and how it is used.

A. Structural Quality

Structural quality refers to characteristics of the health system that affect its ability to meet the health care needs of individual patients or communities. These characteristics include clinician characteristics (e.g., percentage who have board certification, average years of experience, distribution of specialties) and organizational characteristics (e.g., staffing patterns, per capita hospital beds). Structural characteristics are often necessary to provide good care, but they are usually insufficient to ensure excellent quality. The best structural measures are those that can be shown to have a positive influence on the provision of care (process quality) and on the patients’ health (outcomes), though this relationship has not been found for most measures. One structural measure that has been found to relate to outcomes is volume, which refers to the number of times each year that a clinician (or hospital) performs a particular procedure or takes care of patients with a particular disease. Several studies have found that the higher the volume, the better one’s results. This finding argues for using centralized centers, sometimes called centers of excellence, to which patients are referred for particular types of care (e.g., burn centers, cancer centers, centers for specialized heart surgery, AIDS centers).

Structural quality measures have long been the key component in accreditation procedures. Various independent organizations accredit hospitals or health plans based on a set of criteria that generally focus on structural measures such as appropriate capacity for the covered patient population. In recent years, accreditation organizations have also been incorporating process and outcome measures into their accreditation procedures.

B. Process Quality

Process quality refers to what clinicians do for patients and how well they do it, both technically and interpersonally.

1. Types of process quality

a. Technical process refers to whether the right choices are made in diagnosing and treating the patient and whether care is provided skillfully. The former is generally measured by assessing appropriateness or adherence to professional standards, which we describe. The latter is more difficult to assess directly (e.g., it may require direct observation of the clinical interaction) and may be inferred from outcomes (e.g., complications following surgery).

i. Appropriateness. An intervention or service (e.g., a lab test, procedure, medication) is considered appropriate if, for individuals with particular clinical and personal characteristics, its expected health benefits (e.g., increased life expectancy, pain relief, decreased anxiety, improved functional capacity) exceed its expected health risks (e.g., mortality, morbidity, anxiety anticipating the intervention, pain caused by the intervention, inaccurate diagnoses) by a wide enough margin to make the intervention or service worth doing. A subset of appropriate care is necessary or crucial care. Care is considered necessary if there is a reasonable chance of a nontrivial benefit to the patient and if it would be improper not to provide the care. In other words, this is care that is important enough that it might be considered ethically unacceptable not to provide it. Appropriateness criteria and necessary criteria can be used, respectively, to measure overuse of care, which is a problem because of treatment complications and wasted resources, and underuse of care, which means that people are not getting care that would improve their health.

ii. Adherence to professional standards of care. Another way to measure process quality is to determine whether care meets professional standards. This assessment can be done by creating a list of quality indicators that describe a process of care that should occur for a particular type of patient or clinical circumstance. Quality indicators are based on standards of care, which are found in the research literature and in statements of professional medical organizations or are determined by an expert panel. The performance of physicians and health plans is assessed by calculating their rates of adherence to the indicators. Current performance can be compared against a physician’s or plan’s own prior performance or against the performance of other physicians and plans. Indicators can cover a specific condition (e.g., children with sickle cell disease should be prescribed daily penicillin prophylaxis from at least six months of age until at least five years of age) or they can cover general aspects of care regardless of condition (e.g., patients prescribed a medication should be asked about medication allergies).

b. Interpersonal quality refers to whether the clinician provided care in a humane manner consistent with the patient’s preferences. It includes such topics as whether the clinician supplied sufficient information for the patient to make informed choices and involved the patient in decisionmaking. It is generally assessed using patient survey data.

2. Attributes of Good Process Measurement

Good process measures are supported by research studies or professional consensus. They are also flexible with respect to patient preferences. Some patients may not want what most people would consider proper care. Indicators can be constructed so that they are scored favorably if the care was offered but declined. Alternatively, there needs to be some recognition that a perfect score on indicators is not necessarily a feasible or even a desirable goal. For example, although higher childhood immunization rates are an important public health goal, some children are not eligible for certain vaccines because of medical conditions; some also do not get immunized because their parents decline them for religious or other reasons. Therefore, 100% adherence may not be a reasonable target for immunizations. Expectations of perfect performance scores in some areas may create incentives to ignore patient preferences.

The best process measures are those for which there is evidence from research that better process leads to better outcomes. For example, controlling blood pressure reduces mortality from stroke and heart disease; performing routine mammography identifies breast cancer at an earlier stage so that a cure is more likely; prescribing inhaled corticosteroids reduces the likelihood and severity of asthma flare-ups. Unfortunately, research has not covered all aspects of standard medical practice, and so in those cases we rely on expert consensus to decide which processes are important measures of quality.

Several studies of appropriateness and adherence to professional standards have tied process measures to outcomes. In a study of six hospitals in Los Angeles County, mortality rates were examined for patients who had coronary angiography and for whom a revascularization procedure was deemed necessary by explicit criteria. Those who received necessary revascularization within one year had a lower mortality of 9%, compared to those who did not (16%). Those receiving necessary revascularization also had less chest pain at follow-up.

Other research also demonstrates the link between process and outcome. In a study of Medicare enrollees hospitalized with congestive heart failure, heart attack, pneumonia, and stroke in 1981-82 and 1985-86, better process quality of care was significantly associated with lower mortality rates 30 days after hospitalization. Patients who went to hospitals in the lowest 25th percentile on a set of process of care measures had a 39% increased likelihood of dying within 30 days after hospital admission compared to patients who went to hospitals in the highest 25th percentile, after adjusting for patient sickness at admission.

C. Outcomes

Measurement of outcomes is probably the most intuitively appealing approach to quality monitoring. “Outcomes” refers to the results of a health care delivery process. The three main types of outcomes are clinical status, functional status, and consumer satisfaction.

1. Types of outcome measure

a. Clinical status. Clinical status consists of the biological outcomes of disease, such as how organ systems are functioning. Physicians have traditionally used clinical status to determine treatment success or failure. Cancer research, for example, has long used the outcome of five year survival rates. Another cancer outcome is change in tumor size. For diabetics, clinical status outcomes could include complications such as development of diabetic retinopathy (which can lead to blindness) or foot ulcers. For a person with asthma, examples of clinical outcomes include whether the patient is wheezing and how well the patient performs on a peak flow test.

b. Functional status. Measures of functional status assess how disease affects an individual’s ability to participate in physical, cognitive, and social activities. It also covers the ability to meet the regular responsibilities of one’s roles in society (e.g., parent, bank teller, volunteer). When functional status measurement is combined with the patient’s general sense of well-being, it is sometimes called health-related quality of life.

Functional status assessment is based on the premise that many aspects of health are important to patients and will influence them in making treatment decisions. Such assessment could help someone choose between a treatment that would give many more years of life with major incapacitation and a treatment that would give fewer years of life with full function. For example, treatment success for prostate cancer has historically been assessed by the clinical outcome of whether the patient lived. However, functional status measures would incorporate other treatment outcomes, such as the patient’s urinary, sexual, and bowel function. Functional status assessment often includes the degree to which disease limits one’s ability to perform the regular activities of daily living. Such activities include dressing oneself and carrying groceries and is particularly useful in evaluating elderly patients. Treatment for congestive heart failure can have an enormous effect on one’s ability to perform usual activities, and tracking improvement in them can be a useful measure of how the patient is responding to treatment.

c. Consumer satisfaction. Consumer satisfaction refers to how patients feel about the care they receive. One limitation of satisfaction ratings is that consumers cannot always tell if the care was appropriate or technically good; research has not shown a consistent relationship between consumer satisfaction and technical quality of care. A kind and caring physician may provide care that is technically poor. Also, consumer satisfaction may vary with expectations. For example, patients who have a history of poor access to health care may be so appreciative when they actually see a physician that they may report high satisfaction regardless of how well care was delivered. Although consumer satisfaction is important (e.g., patients who are satisfied are more likely to take their antibiotics properly and more likely to return for follow-up visits), it should not be used as a sole measure of quality. Satisfaction is generally measured by surveys of patients. These surveys can also be used to collect information about what services were provided as part of the clinical encounter (e.g., instructions for how and when to take medications); this information can be used in measuring process quality.

2. Attributes of Good Outcomes Measurement

Outcomes measurement has become increasingly popular in the past few years, perhaps because outcomes are the most direct measure of the health of a population. For example, outcomes can be used to assess the quality of care a health system provides its diabetic patients: it can measure whether the system’s diabetic patients have foot ulcers, how well its diabetic patients can maintain their regular activities such as walking to the store, and whether they are satisfied with their foot care as well as overall care.

The best outcome measures have certain key features. First, they should be risk-adjusted, in other words, adjusted for factors (such as age, gender, and severity of the illness) that influence outcomes but are beyond the health care system’s control. Without such adjustment, it is impossible to determine how much of the improvement or worsening of outcomes is due to the care delivered (or not delivered) by the health care system. A cardiac surgeon who receives referrals of patients with severe disease is likely to have a higher surgical mortality rate than a surgeon who takes only patients with mild disease, even though the former may be a better surgeon. To make comparisons in such a system, we need to adjust for how ill the patients are. Risk-adjustment is complex, and the selection of factors to use in risk-adjustment can be challenging. For example, if outcomes are risk-adjusted for race/ethnicity, the adjustment might mask differences in quality due to discrimination. Therefore, sound reasons (e.g., differences in biologic response to disease) are necessary for selecting each characteristic for inclusion in a risk-adjustment method.

Outcomes are most useful when we know the specific processes of care that relate to them. Then, if the outcomes are not as good as they should be, we know what aspects of care to focus on to try to improve them. In other words, if we do not know how an outcome relates to processes of care, we will not know what to do to improve the outcome when we find that it is poor at a particular hospital.

It also helps to measure outcomes from different perspectives. For example, hypertension medications control blood pressure but can also cause sexual dysfunction, so the patient’s clinical status might improve while functional status declines.

It is also important to use outcomes that can be reasonably related to the health care system one is assessing and to the particular part of the system one is assessing. Sometimes a single outcome may be influenced by many different clinicians over many years. For example, diabetic retinopathy often reflects the quality of care provided over approximately two decades. Given how frequently most patients change clinicians or health plans, it would be difficult to relate the quality of any one clinician or plan to this outcome. Similarly, if one is trying to use outcomes to assess the quality of surgeons in a community, it is important to distinguish whether the outcomes are related to the skill of the surgeon, the competence of the surgical team, or organizational characteristics of the hospital.

In addition, outcomes should be measured on samples that are large enough to detect differences in quality. Adverse outcomes are often uncommon events. For example, low birthweight occurs in about seven percent of all births. Therefore, large samples are needed to detect clinically meaningful differences between health plans. To detect a two percentage point difference in incidence of low birthweight between two plans (e.g., five percent for one and seven percent for the other), each plan would need about 1900 births.

In assessing outcomes, it is important to consider that many people will recover from a condition even if they receive poor or mediocre care. Patients with strep throat should receive penicillin to prevent rheumatic fever (which can cause serious heart disease). If all a person knew about two clinicians was that one prescribed penicillin and the other did not (a process assessment), most people would choose to go to the former clinician. However, only about 3% of people with strep throat who do not take antibiotics develop rheumatic fever. Therefore, if a person used an outcome measure to judge the clinicians (i.e., whether the patient developed rheumatic fever), most clinicians would appear to give good quality care, whether they prescribed penicillin or not. Thus, for situations in which care of poor quality rarely leads to poor outcomes, outcome measures are not the most effective way to assess quality.

In summary, there are many challenges inherent in using outcomes to measure quality of care. If we do not address them, we may find that we measure different asthma outcomes among the patients of three physicians but that we cannot determine whether those different outcomes are attributable to the physicians. Process measures have their own challenges (e.g., one needs to make sure that the processes really are important for improving outcomes), but such measures can be quite effective in showing whether providers are doing what they should to have the best chance of achieving good outcomes. We have more experience using process measures than outcome measures to assess quality, and many quality assessment systems depend primarily or exclusively on process measures. However, interest is growing in improving outcomes measurement, so that outcomes can be used along with process measures to provide more useful assessments of health care quality.

IV. WHAT IS THE QUALITY OF THE CARE PROVIDED IN THE UNITED STATES TODAY?

We have reviewed some of what is known about the quality of care, both good and bad, in the United States. We did not aim to be exhaustive but rather to find examples that cover a broad range of conditions and settings. We have concentrated on data from large or diverse populations, such as the nation, an entire state, an entire city, or several hospitals; we include data from a single hospital only when they provide a unique perspective on quality. Our review is based on (a) a search for “quality of care” articles from the National Library of Medicine’s Melvyl Medline Plus system (1993-present) conducted in June 1997, (b) studies identified from the bibliographies of these articles, and (c) additional articles known to the authors.

What is striking about such a review is that we have surprisingly little systematic knowledge about the quality of health care delivered in the United States. We have snapshots of information about particular conditions, types of surgery, and locations of care. Even though health care is a huge industry that affects the lives of most Americans, we have no national system to assess it.

We have divided our review of quality in the United States into three categories based on type of care: preventive, acute, and chronic, and we present our findings in Tables 1, 2, and 3. The tables describe the health care service for which quality is reported, the sample on which the report is based, the data source for the sample, the findings, and the reference for the findings. The tables include data from 43 articles covering about one-half million people.

The dominant finding of our review is that for most care that has been studied, there are large gaps between the care that people should receive and the care they do receive. This is true for all three types of care. It is true whether one looks at overuse or underuse. It is true in different types of health care facilities and for different types of health insurance. It is true for all age groups, from children to the elderly. And it is true whether one is looking at the whole country or only one city.

It is difficult to provide a numerical summary of the findings in these tables. As a rough summary, we offer some simple averages. However, we must emphasize that these studies cover different populations, time periods, and methodologies, and so the averages only give a ballpark idea of the quality of care people are receiving. When there is a range of values for a single group, we generally use the midpoint in our calculation. A simple average of the findings of the preventive care studies shows that about 50% of people received the recommended preventive care. For some acute care studies, data are reported as the percentage of people who received appropriate care, and for other studies, the data are reported as the percentage who received inappropriate care. For the former set of studies, averaging shows that 70% received appropriate acute care, and for the latter, 30% received inappropriate acute care. The comparable averages for chronic care show that 70% received appropriate care, and 30% received inappropriate care. We do not intend these numbers to indicate exact levels of quality in the United States but rather to provide a more quantitative sense of how much could be done in virtually all areas to identify and eliminate overuse and underuse of care. To provide more precise percentages, we would need to conduct a more systematic review of the literature. Furthermore, the analysis would have to take into account the relative prevalences of conditions in the U.S., how well the studies were conducted, how many people were in the studies, and how much impact each type of care has in promoting health.

We present a few examples of quality of care here and many more in the tables.

An annual influenza vaccine is recommended as a preventive measure for all adults 65 years or older, a group at especially high risk for complications and death from influenza. However, in 1993, 52% of people in this age group in the United States received the vaccine; among people who had been to the doctor at least once that year, the percentage was slightly higher at 56%.

A major issue in acute care is the overuse of antibiotics, which is leading to the development of strains of bacteria that are resistant to available antibiotics. Antibiotics are almost never an appropriate treatment for people with a common cold because almost all colds are caused by a virus, for which antibiotics are not effective. However, in a study of Medicaid beneficiaries diagnosed with a cold in Kentucky in 1993-1994, 60% filled a prescription for an antibiotic. Other types of medications are also not always used in the most appropriate manner. Among hospitalized elderly patients with depression who were discharged on antidepressant medication, 33% were on a dose below the recommended level. In a study of 634 patients with depression or depressive symptoms in Boston, Los Angeles, and New York, 19% were treated with minor tranquilizers and an additional 11% used them in combination with antidepressants, despite the lack of evidence that tranquilizers work for depression and the risk that they will cause side effects or addiction.

Patients with chronic conditions for which certain routine exams and tests are crucial for preventing complications do not all get the care they need. As we have discussed above, diabetes mellitus causes several complications that are less likely to occur with good care. One of these complications is an eye condition called diabetic retinopathy, which is the leading cause of new blindness among persons age 20-74 years in the U.S. It is recommended that patients with insulin-dependent diabetes mellitus have an annual dilated eye examination (the clinician uses drops to enlarge the pupil to see behind it more easily) starting five years after diagnosis and that patients with non-insulin-dependent diabetes mellitus have the exam annually starting at the time of diagnosis. In a national study in 1989, 49% of adults with either type of diabetes had had a dilated eye examination in the past year, and 61% had had any type of eye exam in the past year. Twenty percent of diabetics had had no eye exam in the past two years. Among diabetics who were at particularly high risk for vision loss because they already had retinopathy or because they had had diabetes for a long time, 61% and 57%, respectively, had had a dilated examination in the past year.

Quality is also not always high when it comes to decisions to perform surgery. Sometimes surgery is performed on people who do not need it. In a study of seven managed care organizations, about 16% of hysterectomies performed during a one-year period in 1989-1990 were performed for inappropriate reasons. An additional 25% were performed for reasons of uncertain clinical benefit. There are also examples of patients who need surgery but do not receive it. In a study of eight hospitals, 43% of patients with a positive exercise stress test demonstrating the need for coronary angiography had received it within 3 months; 56% had received it within 12 months.

Not all studies have found such poor quality. In a study of patients who had had cataract surgery, 2% had had the surgery for inappropriate reasons. In a study of patients who underwent coronary artery bypass graft surgery (CABG) surgery, 1.6% had surgery for inappropriate reasons. Nonetheless, the majority of studies described in the tables below show much room for improvement of quality.

There has been a fair amount of concern over whether managed care is responsible for much of the poor quality that is found in the American health care system. The short answer is that studies have not provided a clear answer to this question. Some studies find that managed care organizations provide better care than fee-for-service, some find that fee-for-service provides better care, and some find that the care is about the same. Results vary depending on the setting, the type of care assessed, and the methodology. This topic is complicated by the research approach, which has generally been to lump together managed care organizations without distinguishing them either by type (e.g., group/staff-model health maintenance organizations, independent practice associations, preferred provider organizations, point-of-service plans) or by features (e.g., comprehensiveness of the benefits package, non-profit vs. for-profit status). It would be more useful to look at the impact of specific characteristics of managed care organizations. For example, whether or not the benefits package covers immunizations may have a larger impact on immunization rates than whether one gets care from a managed care organization or fee-for-service provider. Finally, managed care is changing so rapidly that most studies currently available are already out-of-date. We do not have a large enough ongoing quality measurement system to enable timely assessment of the rapid changes occurring in the health care marketplace. Even the most widely used systems are far from universal and do not cover both managed care and fee-for-service.

Finally, lest we give the impression that the United States alone suffers from quality gaps, we have included data on quality from other countries. Much less information is available from other countries, but the few studies we found indicate that quality problems are not limited to the U.S. (See Table 4.)

V. HOW CAN AN ORGANIZATION USE INFORMATION ON QUALITY TO IMPROVE THE QUALITY OF CARE IT PROVIDES?

The principal purpose of internal quality assessment is to improve quality within an organization. Ideally, information on quality shows the plan or clinician where to focus efforts to improve quality. Even when the information is collected so that another organization can conduct external quality assessments of multiple plans or providers (see next section), it can generally be adapted for use in internal quality improvement.

Many techniques are available for using internal quality assessments to improve quality within an organization. Traditional quality assurance programs focus on improving or removing clinicians who stand out as being particularly inferior. By contrast, continuous quality improvement (CQI) programs (also known as total quality management or TQM programs) focus on improving the quality of care delivered by all clinicians, with the goal of raising the average level of quality in an organization. CQI assumes that most examples of poor quality are due to correctable systematic problems rather than to individual incompetence or irresponsibility. It often incorporates routine collection and monitoring of information to assess quality so that the organization can identify and respond to problems in a timely fashion. CQI generally uses interdisciplinary teams both to identify inefficiencies that increase errors and to institute checks that make errors easier to prevent and catch. For example, a group of all cardiothoracic surgeons practicing in three states used CQI and other techniques to improve their practices and found a 24% reduction in their combined mortality rates.

A hospital that finds many medication errors might review the many steps involved in the process by which physicians, nurses, pharmacists, and orderlies provide medications to patients. Solutions might include having nurses and pharmacists double-check doses, using standardized doses for patient weight ranges to reduce calculation errors, and having a standard location for documentation of all drug allergies.

One approach a hospital might take if it finds wide variation in how a clinical condition is being treated is to provide practice guidelines, which are a set of systematically-developed recommendations about some or all aspects of decisionmaking for a particular condition or clinical situation. Although guidelines can be presented as a simple list, they are often structured as a decision algorithm with branch points and if-then statements, thereby allowing clinicians to understand the rationale behind each step involved in diagnosing and treating the condition. Guidelines are not intended to dictate a rigid approach to care; rather, they give options that a clinician should be aware of, even if he or she chooses a different strategy for a particular patient. Practice guidelines have been developed by government agencies (e.g., the National Institutes of Health, the Agency for Health Care Policy and Research), specialty organizations (e.g., American College of Physicians, American Academy of Pediatrics), third party payers (e.g., Blue Cross/Blue Shield, Kaiser Permanente), and academic research organizations.

Internal quality improvement must take place primarily at the local level. While models can be provided at a national level, actual activities and work must be adapted to fit a local context.

VI. WHAT SYSTEMS ARE USED FOR EXTERNAL QUALITY MONITORING IN THE UNITED STATES?

External quality assessment makes quality information routinely and systematically available to consumers and large purchasers of care to inform their choices of physicians, hospitals, and health plans. Without such quality comparisons, variations in the costs of plans may not be meaningful.

In this section, we discuss some of the major U.S. systems being used and developed to measure quality of care. We have chosen a handful of systems as examples, and we present them in three categories: quality of care measurement tools, quality measurement used to determine reimbursement rates, and accreditation services.

A. Quality of Care Measurement Tools

Report cards are often used to present the results of external quality assessment. They use a standardized format for making the results of quality measures available to consumers (e.g., when choosing among health plans), purchasers of care, physicians, and health plans. They generally show comparative scores for different plans or clinicians on single quality measures or groups of measures. Report cards are most useful when standardization of information allows reasonable and fair comparisons among plans and when they present results in a way that does not mislead users into placing undue importance on differences that are not statistically significant or clinically relevant. We describe examples of systems that have already been implemented and that are currently being developed.

1. Systems that have been implemented

a. National Committee for Quality Assurance (NCQA) NCQA produces a widely-used report card monitoring system called the Health Plan Employer Data and Information Set (HEDIS®). HEDIS® is a performance measurement tool designed to assist purchasers and consumers in evaluating managed care plans and holding plans accountable for the quality of their services. Because HEDIS® has standard measures and uniform data reporting requirements, comparisons can be made across various health plans and their organizational structures (e.g., staff model health maintenance organizations, point of service plans).

The most recent iteration, HEDIS® 3.0, assesses plans with 71 measures in eight domains: effectiveness of care, accessibility and availability of care, satisfaction with care, cost of care, stability of the health plan, informed health care choices, use of services, and plan descriptive information. HEDIS® uses primarily process measures but also includes some outcome and structural measures. It covers preventive care, acute care, and chronic care. HEDIS® 3.0 requires plans to provide separate reports for enrollees covered by commercial insurance, Medicaid, and Medicare. In addition, it includes a set of measures that are not used in current scoring but are being tested for use in future iterations of HEDIS®. At present, HEDIS® results are reported by insurance type, but they are not adjusted for individual patient characteristics.

HEDIS® is a voluntary system, although managed care plans are finding it increasingly necessary to participate to compete for patients. Most large employers request HEDIS® data from the managed care plans with which they contract, and HCFA requires all Medicare risk providers to report select HEDIS 3.0 measures. In 1996, more than 330 plans (out of more than 600 managed care plans in the U.S.) were reporting HEDIS® statistics to NCQA.

b. New York State’s Cardiac Surgery Reporting System The New York State Department of Health reports statistics for individual surgeons and hospitals on risk-adjusted mortality after CABG surgery. Hospitals voluntarily report data on all open heart surgeries to the state. Patients’ preoperative risk factors for mortality following CABG are used to estimate and predict hospital- and physician-level risk-adjusted mortality rates. Performance is evaluated by comparing the predicted rates to the actual mortality rates, and the results are made public.

After the program began, it was determined that surgeons performing fewer than 50 CABG surgeries per year had higher risk-adjusted mortality rates. Many of these surgeons stopped performing surgery, in some cases because their hospitals would not allow them to. Some hospitals have assigned higher risk patients to surgeons with superior performance. Hospitals have also responded to the program by reviewing specific processes, identifying problems, and addressing the problems, with subsequent improvement in mortality rates.

The impact of this system has been the subject of debate. Initial reports indicated that the system was having a dramatic effect on quality. During its first four years (starting in 1989), there was a 41% decline in risk-adjusted mortality after CABG (from 4.17% to 2.45%). However, some argue that this improvement is deceptive, because over the same period there was similar improvement in Massachusetts and nationwide for Medicare CABG surgery. National improvement may have been related to publication by HCFA of CABG mortality data for hospitals around the country (see below), which might have sparked efforts to improve care in communities and hospitals outside of New York. Also, technological advances at the time may have universally lowered mortality rates. In addition, changes in data collection mechanisms and reporting of risk factors in New York over time may have made the program appear more successful than it actually was. After it began, hospitals had an incentive to more routinely document risk factors that are used in the adjustment process. Whether or not cause and effect can be determined, the system produces useful information for consumers, purchasers, physicians, and hospitals on outcomes of an important surgical procedure.

There have also been concerns about the potential negative impact of the program. One such concern is that hospitals and physicians might attempt to lower their risk-adjusted mortality by avoiding high-risk but appropriate candidates for CABG. However, this problem does not appear to have occurred in New York.

c. Pennsylvania Health Care Cost Containment Council In 1992, the Pennsylvania Health Care Cost Containment Council began publishing an annual “Consumer Guide to Coronary Artery Bypass Graft Surgery,” which compares risk-adjusted actual and expected mortality rates for all Pennsylvania hospitals and surgeons providing CABG surgery. In a 1995 survey, most cardiologists reported that the guide had minimal or no influence on their referral patterns. They also reported that it had become more difficult to find surgeons willing to take severely ill patients. The majority of cardiac surgeons reported that they were less willing to operate on such patients. According to both cardiologists and cardiac surgeons, the most important limitations of this system were the absence of quality indicators other than mortality, inadequate risk assessment, and unreliability of data from hospitals and surgeons. However, no evidence indicates whether the limitations they perceive actually exist.

d. Cleveland Health Quality Choice Program In 1989, businesses, hospitals, and physicians in the area surrounding Cleveland, Ohio, formed the Greater Cleveland Health Quality Choice Coalition, which established the Quality Information Management Corporation in 1991 to oversee the Cleveland Health Quality Choice (CHQC) Program. This program developed a system (a) to measure outcomes (e.g., observed deaths, length of stay, complications) for surgery, general medicine, intensive care, and obstetrics and gynecology, (b) to compare the outcomes for each hospital to both the predicted outcomes for the hospital and the mean for all Cleveland hospitals, and (c) to measure patient satisfaction. The information is distributed both to hospitals so they can improve quality and to businesses so they can consider quality as well as cost when choosing plans.

e. Health Care Financing Administration (HCFA)

Hospital Mortality Reports
In 1986, HCFA began reviewing mortality rates as a marker of hospitals’ quality of care. This system used billing data and information routinely collected from the Social Security Administration to compute adjusted annual 30-day (and other) mortality rates for Medicare patients at all acute care hospitals. The results were released annually to the public from 1987 to 1993.

This system’s case mix adjustment was repeatedly criticized. In 1993, HCFA discontinued the routine release of these data with the explanation that they were inappropriate for ranking hospitals or for use as a direct measure of quality of care.

Peer-Review Organizations
In 1982, Congress established utilization and quality control peer review organizations, now called simply peer review organizations (PROs). They were set up under the jurisdiction of HCFA, and their purpose was to monitor hospital use and quality of care for Medicare patients. (Some also set up contracts with Medicaid and other third-party payers.) They were established in each state and territory. PROs replaced professional standards review organizations (PSROs), which had aimed to promote efficiency and to eliminate unnecessary hospital utilization.

Until 1992 (see below), PROs were primarily responsible for reducing unnecessary hospital admissions and operations, reviewing readmissions, and validating assignments to diagnosis-related groups (DRGs). Nurse reviewers used criteria established by HCFA to evaluate a random selection of medical records of hospitalized patients. The criteria included adequacy of discharge planning, medical stability at discharge, unexpected deaths, nosocomial infections (infections contracted in the hospital), unscheduled returns to surgery, and trauma suffered in the hospital. If the record failed any criterion, a physician adviser usually had to determine whether the record demonstrated a quality problem. When PROs confirmed quality problems, they had several options. They could formally notify the institution or practitioner; they could require continuing medical education, preadmission review, detailed retrospective reviews, or referral to medical staff committees; or they could inform licensing and accrediting bodies. In addition, the Inspector General of the Department of Health and Human Services could apply sanctions, such as the loss of Medicare billing privileges.

Health Care Quality Improvement Program (HCQIP)
In 1992, HCFA established Medicare’s Health Care Quality Improvement Program (originally called the Health Care Quality Improvement Initiative), which promotes partnerships between PROs and hospitals, health plans, and physicians. These partnerships profile patterns of medical care, identify areas in which treatment could be improved, assist in the development of quality improvement efforts, and measure improvement. HCQIP emphasizes continuous quality improvement (CQI) methods. It monitors patterns of care and outcomes to help identify persistent deviations from desired quality rather than individual instances of deviation. Consistent with the CQI approach, hospitals are expected to improve the quality of medical care in general rather than to concentrate on weeding out particularly poor clinicians. HCQIP initially covered only fee-for-service Medicare, but it is now being used for PRO review of managed care organizations providing care to Medicare beneficiaries as well.

HCFA has also been working with NCQA to develop and modify HEDIS measures to meet the particular needs of Medicaid and Medicare beneficiaries. For example, many children who receive Medicaid switch providers often. Therefore, an immunization measure that requires continuous enrollment in the same health plan from birth to two years would exclude many Medicaid beneficiaries. Changing eligibility criteria to a single year of enrollment may produce information that applies to a larger portion of the population.

Medicaid Quality Assurance Reform Initiative (QARI)
HCFA established the Quality Assurance Reform Initiative’s (QARI) Health Care Quality Improvement System in 1991 to improve oversight of Medicaid managed care. QARI consists of a uniform set of guidelines for a comprehensive, state-based system for ensuring and improving quality of care. The QARI system requires health plans to have internal quality assurance programs, and it requires states to establish and then monitor compliance with standards for an acceptable program. QARI requires states to provide for an annual, external review of quality of care in their health plans. This process consists primarily of detailed investigations of clinical areas or health services delivery topics that the state and plans select jointly. QARI emphasizes quality measurement, continuous quality improvement, and involvement of health plans and Medicaid beneficiaries in quality oversight.

This system has been evaluated through a three-state demonstration conducted in Minnesota, Ohio, and Washington, which found that QARI is generally effective for monitoring and improving quality of care. However, the evaluators recommended modifications to make QARI easier to implement and more effective.

HCFA is now developing a common set of plan requirements for Medicare and Medicaid called the Quality Improvement System for Managed Care (QISMC).

f. Pacific Business Group on Health (PBGH) PBGH is a nonprofit coalition of large health care purchasers in California and Arizona created in 1989 to improve the quality of health care and address rising costs. As of October 1996, it represented 33 private and public sector organizations, almost all of which have at least 2000 benefits-eligible employees. PBGH estimates that it represents about 2.5 million insured individuals. PBGH members have established a Quality Improvement Fund that supports projects to measure and improve quality provided by health plans, hospitals, and physician groups.

One of its projects is the California Collaborative Healthcare Reporting Initiative (CCHRI), which represents the collective interests of health plans, purchasers, and provider organizations. CCHRI collects, analyzes, and reports performance data; promotes the use of accurate and comparable quality measures; and designs and engages in collaborative quality improvement projects. It provides purchasers with this information to gauge health plan performance, and it makes this information available to the public in a report card format. It currently reports on six HEDIS preventive care measures but will add others over time.

PBGH has several outcomes projects. The Cesarean Section and Perinatal Mortality Outcomes Project uses hospitals’ risk-adjusted cesarean section and perinatal mortality rates to encourage hospitals with particularly high rates to analyze their internal physician-specific cesarean section rates and to adopt guidelines to discourage the use of medically unnecessary cesarean sections. In 1998, the Coronary Artery Bypass Graft Mortality Reporting Program will begin making hospitals’ risk-adjusted, operative mortality for CABG available to plans, purchasers, and consumers. The system will be similar to CABG systems in New York and Pennsylvania (described above), though it will only report on hospitals and not individual surgeons. PBGH also has similar outcomes programs for transplants, asthma, and breast cancer.

From 1989-1996, PBGH conducted an annual health plan member satisfaction survey called the Health Plan Value Check survey. Starting in 1997, PBGH implemented the NCQA Member Satisfaction Survey. Results are provided to consumers, health plans, and employers. In addition, PBGH conducts a survey of patients in 56 physician groups that assesses satisfaction with health plans, processes of care for two chronic conditions, receipt of preventive care services, and two-year changes in health and functional status. Detailed results are provided to medical groups, employers, and consumers through a publicly released report card.

The Consumer Information Hub increases consumer access to, awareness of, and use of health care quality information. Through the Internet and a toll-free number, consumers can request mailed or faxed comparative performance data on plans, physician groups, and hospitals. Through the Health Care Information Initiative, PBGH is collaborating with plans and providers to change the data infrastructure system, so that it will include an electronic medical record, an electronic data interchange, and a real-time feedback mechanism to facilitate continuous process improvement at the clinician level.

g. New England HEDIS Coalition The New England HEDIS Coalition was created in 1993 as a non-profit coalition of employers, health plans, and state agencies to develop a strategy to implement HEDIS. It aims to help plans and purchasers develop, test, produce, and report measures of health plan performance and to improve quality and cost efficiency in plans. It began when Digital Equipment Corporation and other employers joined with health plans to implement HEDIS.

The Coalition reviews HEDIS indicators and recommends how data should be collected and reported. In October 1994, the Coalition released the 1993 Baseline Performance Profile, which provided comparisons of performance measures for 15 New England health plans. These plans provided care to over three million people and represented $4.5 billion in health care premiums. In 1995, the plans submitted their 1994 data directly to NCQA. The Coalition selected a subset of the HEDIS measures for its report. These reports are used by consumers, purchasers, and health plans. Because several years of reports now exist, one can both compare plans and look for improvement within a plan over time. The Coalition is also working with plans on CQI initiatives.

h. Minnesota Health Data Institute In 1993, the Minnesota State Legislature created the Minnesota Health Data Institute as a non-profit, public-private partnership to improve the quality of health care services available in the state. The Institute will collect and distribute performance information on the state’s health care delivery system on an ongoing basis. In 1995, it fielded its first satisfaction survey of over 17,000 members from 46 health plans in the state. The Institute published the results in over 60 newspapers and distributed individual copies throughout the state. The goals of the report were to help consumers choose health plans; to help large purchasers incorporate quality into their decisions and hold plans and clinicians accountable for quality; to provide health plans with detailed information on their own quality; and to help policymakers by providing comparisons of types of plans (e.g., public vs. private). The publicly available version of the report grouped plans into types: private health insurance, Medicare, and state health programs (including Medicaid). Questions included overall satisfaction with the plan, its doctors, its administrative functions, and the waiting time for appointments. Results were reported in tabular format; for each type of satisfaction, the tables listed whether each health plan was above average, average, or below average. The Institute also collects and reports performance measurement information on health plans in the state.

For overall consumer satisfaction, 81% of HMO members were “very” or “extremely” satisfied, compared with 71% of commercial indemnity plan members.

2. Systems under development

a. Foundation for Accountability (FACCT) FACCT is a non-profit organization created in 1995 to bring the consumer perspective to quality assessment. Its board includes representatives from large consumer groups and from public and private purchasers of care that supply health insurance to over 80 million Americans.

FACCT’s principal mission is to develop and endorse quality measures, encourage their use, and work to ensure that information on quality is effectively communicated to and used by consumers. It emphasizes outcome measures but also endorses process measures. It designs measurement sets to assess quality at the community level as well as at the plan and medical group levels.

FACCT incorporates the patient perspective into its endorsement process and only approves measures that are relevant to consumers. Measures are also assessed in terms of scientific merit and feasibility. Measurement sets for individual topics are designed to cover three categories: (a) results of care (outcomes), (b) patient satisfaction and experience of care (consumer satisfaction and reports about process), and (c) steps to good care (process quality).

FACCT has endorsed condition-specific measurement sets for diabetes, asthma, breast cancer, and major depression. It has also endorsed measurement sets designed for general use among a population of patients: health risk behaviors, health status of people 65 years and older, and consumer satisfaction. It is currently working on developing measures for coronary artery disease, health status for people under 65 years old, alcohol misuse, pediatrics, geriatrics, and end-of-life care. Market-wide testing is currently being conducted for several of the sets that have been endorsed.

b. RAND Global Quality Assessment Tool RAND is developing a system to measure quality of care in managed care organizations. Its key features are that (a) it covers a broad array of conditions, thereby diminishing the chances of clinicians focusing on improving quality only for those aspects of care covered by a small indicator set; (b) it assesses quality for all enrollees in a health plan, even if they have not obtained any care, so that underuse as well as inappropriate use can be identified; (c) it aggregates scoring across indicators and conditions, which allows inclusion of indicators and conditions that do not apply to large percentages of the enrolled population and so would otherwise be missed in most reviews of quality; (d) it covers all ages, including groups that are often omitted from measurement systems, such as children and adolescents; (e) its indicators are grounded in scientific literature or expert consensus; and (f) it is primarily based on medical record abstractions, but will make some use of enrollment and claims files. Development of this system is being funded by cooperative agreements with HCFA and the Agency for Health Care Policy and Research (AHCPR); the full system will be available by the year 2000.

c. Consumer Assessment of Health Plans Study (CAHPS) CAHPS is developing a kit that provides information to help consumers and group purchasers choose among health plans. The kit contains (a) surveys that ask consumers about their experiences with their health plans, (b) sample formats for reporting results to consumers, and (c) a handbook to help implement the survey and produce the reports. The kit addresses information such as access to specialists, quality of communication of physicians and other health care providers, and helpfulness of customer service. It includes questions targeted to Medicaid and Medicare beneficiaries, persons with chronic conditions or disabilities, and parents/guardians of children. The surveys are designed to assess experiences for different populations and care delivery systems. The kit is being developed by RAND, Harvard Medical School, and the Research Triangle Institute (RTI) under a cooperative agreement from AHCPR.

B. Measurement Tied to Reimbursement

1. Blue Cross and Blue Shield of Minnesota’s (BCBSM) Illness Outcome Group Reimbursement Program

BCBSM has developed a reimbursement system based on whether hospitals’ rate of adverse outcomes is higher or lower than expected. The Illness Outcome Group uses a classification system that is based on diagnosis-related groups and includes severity of illness. In 1995, BCBSM reported that the system had improved morbidity and mortality and lowered length of stay while the hospitals’ risk load increased by 4%. In 1996, it reported fewer hospitalizations for lower risk patients and a further decrease in the length of stay. BCBSM plans to compare its network hospitals against standards.

2. Pacific Business Group on Health (PBGH)

In 1994, employers that belong to PBGH (described above) formed a Negotiating Alliance to negotiate collectively with California MCOs to establish a standardized benefits package emphasizing both price and quality. PBGH negotiates an individual set of performance targets with each participating MCO. If the MCO does not meet the targets, then 2% of the premium paid by PBGH employers to the MCO is turned over to PBGH. Therefore, the MCOs have a financial incentive to meet their targets.

C. Accreditation

Accreditation is a key component of quality assessment because it provides an in-depth evaluation of the capacity of health care organizations to deliver acceptable quality of care. Accreditation status signals that the structural elements needed to deliver good care exist; however, by itself, accreditation does not guarantee the level of quality provided.

1. National Committee for Quality Assurance (NCQA)

NCQA (discussed above) has been accrediting managed care organizations since 1991. It evaluates how well a health plan manages its delivery system in providing and continually improving care and service for its members. The program is voluntary.

A team of reviewers (including physicians) makes a several-day site visit and evaluates a plan in six areas: quality management, physicians’ credentials, members’ rights and responsibilities, preventive health services, utilization management, and medical records. A national oversight committee makes the final accreditation decisions.

NCQA’s accreditation process has three levels: (a) full accreditation, which is a three-year approval, (b) a one-year approval for plans that meet most NCQA standards but must address a specific list of recommendations to move up to full accreditation, and (c) provisional accreditation for plans that have adequate quality improvement programs and that meet some NCQA standards. Some plans are denied accreditation but can apply again in the future. Of 230 plans reviewed as of September 1996, 90 (39%) had received full three-year accreditation, 88 had received one-year accreditation, and 23 had received provisional accreditation; 26 were denied accreditation, and the remainder were still under review. Few plans (18%) receive the full three-year accreditation during their first review.

2. Joint Commission on Accreditation of Healthcare Organizations (JCAHO)

JCAHO is a non-profit organization that evaluates and accredits more than 16,000 health care organizations in the United States. It accredits not only hospitals and health plans, but also home health care organizations, nursing homes, other long term care facilities, behavioral health care organizations (e.g., chemical dependency services), ambulatory care providers (e.g., infusion centers), and clinical laboratories.

For accreditation, JCAHO conducts an on-site quality assessment every three years. It uses structural, process, and outcome measures for its assessment, and covers such topics as patient rights, patient care, patient education, continuity of care, ongoing efforts to improve quality, safety plans, information management, and infection control.

JCAHO provides different levels of accreditation. In 1995, 12% of hospitals received accreditation with commendation, 10% received accreditation, 74% received accreditation with recommendations for improvement (which means that the improvements must be made within a specified period of time to maintain accreditation), 3% received provisional accreditation (a category relevant to hospitals that are newly seeking accreditation), less than 1% received conditional accreditation (indicating serious problems that need to be addressed), and less than 1% received no accreditation. A new category of preliminary nonaccreditation was added in 1996 to cover the initial period of accreditation denial.

Participation in the accreditation process is voluntary, although the incentive to participate is high because JCAHO accreditation qualifies an institution for Medicare reimbursement. About 80% of U.S. hospitals participate; they represent about 96% of all inpatient admissions.

3. Mammography Quality Standards Act of 1992 (MQSA)

The MQSA required that starting on October 1, 1994, all U.S. mammography facilities, except those managed by the Department of Veterans Affairs, must be certified by the Food and Drug Administration (FDA). To be certified, a facility must be accredited by an FDA-approved accrediting body (e.g., the American College of Radiology, the Bureau of Radiological Health of the Iowa Department of Public Health) and undergo annual inspection by federal or state personnel. Accreditation requires periodic review of clinical images and an annual survey by a medical physicist. The facility must also meet quality standards for equipment, personnel qualifications, quality-assurance programs, recordkeeping, and reporting. FDA certification is required as the basis for Medicare reimbursement. Some feared that MQSA would lead to the closure of many facilities, thereby reducing access to mammography. However, a study showed that access was not reduced in Minnesota, where many facilities received provisional accreditation and appear to be improving to receive full accreditation.

4. Utilization Review Accreditation Commission (URAC)

URAC is a non-profit organization that accredits utilization management organizations. It has also developed the National Network Accreditation Standards to serve as guidelines for preferred provider organizations, health maintenance organizations, physician-hospital associations, independent practice associations, and other managed care organizations.

D. Summary

In this section, we have briefly described some examples of organizations and government bodies that are doing work to measure and improve quality of care. There are other such efforts that we have not described. Most of this activity has begun over the past decade. The rapid development of the field is encouraging, but these organizations cover specific sections of the country or specific segments of the health care marketplace. Their work, as well as individual studies such as those listed in the tables, provide snapshots of what is going on throughout the country. However, there is no system in the United States to provide a comprehensive assessment of quality of care for the nation, how quality varies by population subgroups (e.g., gender, age, race/ethnicity, income, region of country, size of community), and how quality is changing over time. With all the developments in the health care system that have occurred over the past few years, we cannot even say if quality is improving or declining or how it may be changing for different groups. It is possible the efforts described in this section will eventually lead to a more comprehensive system, but it is doubtful that this will occur any time soon without an organized effort to create such a system.

VII. WHAT FACTORS ARE IMPORTANT FOR AN EFFECTIVE QUALITY MONITORING AND ASSESSMENT SYSTEM?

A. Population-based

The system should be population-based so that it takes into account everyone who could benefit from care whether or not they use services. For example, in assessing quality in a managed care organization, quality measurement should include all enrollees rather than just those who receive care so as to account for patients who have difficulty gaining access to the system (e.g., because they are unable to get an appointment in a timely fashion, because there are no translators for their language). Otherwise, the system could create an incentive to discourage enrollees with complex or expensive conditions from seeking care.

B. Broad coverage

The system should cover all aspects of a health care delivery system because quality of care may vary within an organization. These aspects include types of care (e.g., preventive, acute, chronic), appropriateness (e.g., overuse of services, underuse of services), and settings (e.g., outpatient, inpatient, nursing home and other institutional care, laboratory and other ancillary services, and home health care).

The system should also cover conditions that are of particular interest to certain communities or groups. Cataract surgery will be important to many elderly patients, while prenatal care will be important to prospective parents. It is also important to include complex medical conditions in quality measurement systems. Care for such conditions can be more expensive and difficult to provide, and accounts for only a small proportion of care delivered in most plans. Therefore, if the quality of this care is not specifically measured, some plans may choose to skimp on such care while still obtaining good overall quality scores.

C. Risk-adjustment

The system should control for demographic and other differences in enrolled populations (i.e., risk-adjust) and for factors outside of the control of the health care delivery system that influence the processes and outcomes of care. If the system does not risk-adjust, it will create incentives for plans to discourage enrollment and encourage disenrollment of people who are less likely to have good outcomes.

D. Adaptability

The system should be capable of being modified as we learn more about health care. As we learn more about existing drugs and treatments and as new drugs and treatment strategies are developed, we need to be able to adapt our assessment systems accordingly. Otherwise, they will become obsolete.

E. Diverse measures

Different types of quality measures can provide different perspectives on care. Process measures will be most effective for distinguishing differences in the quality of health care delivery. There is relatively less experience with measuring outcomes. It can be more difficult to detect quality variations with outcome measures because large populations in a health plan are often needed to study outcomes, many outcomes take a long time to develop, and multiple factors outside the control of the health care system influence outcomes. However, when developed and used properly, outcomes can add valuable information to quality measurement, and there are ongoing efforts to improve outcome measurement techniques. Structural measures may also be used to supplement a measurement system.

F. Information systems

If quality measurement systems are going to assess the full spectrum of quality in health plans, we will need to develop more effective information and data collection systems. Information must be collected from multiple sources, such as medical records, laboratory reports, claims data, and patient surveys. In addition, abstraction of paper records to collect information is expensive and tedious. Computer systems that collect and merge key pieces of clinical information will facilitate quality measurement. Related issues of confidentiality and security of computer records will also need to be addressed.

G. Full participation

If the marketplace does not penalize plans that fail to participate in quality monitoring programs, plans will not have an incentive to participate, and external quality monitoring will be ineffective. All plans need to participate so that appropriate comparisons can be made.

H. Uniformity

Health plans are finding it difficult, and at times overwhelming, to collect different data for multiple purchasers using different quality measurement systems. Even purchasers that use HEDIS often add extra measures of their own for plans to report. Lack of uniformity also makes it difficult to compare quality that has been measured by different systems. Quality measures that appear to be the same may be quite different if the specifications for data collection are different. For example, two systems may have a measure of whether two-year-old children have received all appropriate vaccines; however, in counting vaccines given by a previous provider, one system may accept a parent’s report that the vaccines had been given while another system may require the health plan to obtain medical records from the previous provider. Similarly, different systems may include different vaccines in their immunization indicators, or they may use different age cut-offs, thereby making comparisons across systems difficult.

One solution is to develop a standard set of measures that covers the domains of interest to various constituencies, or different organizations that measure quality might work together to create greater uniformity across their systems.

I. Interpretability

The results of a quality measurement system must be presented in a format that can be understood easily by consumers, large purchasers, clinicians, health plans, and policymakers. The format does not need to be the same for each of these groups, but each group needs a format that it can use. Particular attention may need to be paid to making information useful for consumers. There is evidence that consumers may not always understand the health care system well enough to interpret quality of care information and that they may consider information unimportant if they do not understand it. Therefore, information needs to be presented in a manner that clarifies its relevance to the health care system and to consumers themselves.

Differences in quality should be presented in a way that takes into account both statistical significance and relevance to health. When large numbers of patients are included in a system, trivial differences in quality of care, such as one physician having a 15 second longer waiting time than another physician, can turn out to be statistically significant (i.e., the difference between the two groups is unlikely to be due to chance). A system that reports all statistically significant differences without consideration of clinical relevance or without providing clarifying explanations may lead users to make decisions based on unimportant distinctions. By contrast, a system comparing health plans using data on only a few patients with a particular disease may fail to show a statistically significant difference in quality of care for the disease, even when there actually is a clinically important difference among the plans. Therefore, reports on quality must describe any limitations on the conclusions that can be reasonably drawn from them.

VIII. CONCLUSIONS

The quality of health care provided in the United States varies among hospitals, cities, and states. The same is true in other countries. Whether the care is preventive, acute, or chronic, it frequently does not meet professional standards. We can do much better. The solution is not simply a matter of spending more money on health care. A large part of our quality problem is the amount of inappropriate care that is provided in this country. If such useless and potentially harmful care were eliminated, there would be a large savings in human and financial costs. However, there are also plenty of examples of people who receive too little care or care that is technically poor; fixing these problems will generally increase financial costs.

The good news is that techniques exist to measure quality of care. Clinicians and health plans can use information on quality to determine where the quality of the care they provide needs improvement. If this information is made available regularly and in an interpretable form, consumers and large purchasers can also use it to make informed decisions when choosing among clinicians and plans, which will, in turn, give providers an added incentive to improve quality.

Policymakers can also use information about quality of care to determine the impact of public and private changes in the health care marketplace. We are currently experiencing a dramatic shift in the organization and financing of health services delivery in the United States. The private sector has been the driving force behind this transformation, but the public sector has been an active participant as well. Medicaid managed care is only one of many examples of public sector reform. Changes are occurring faster than evaluations of these changes can be performed. Much of the information concerning the relationship between the organization of the health system and the quality of care is already outdated. At present, we have only a patchwork of systems that measure quality, with little uniformity, breadth, or ability to produce rapid results. Furthermore, these systems do not yet assess most providers of health care in the United States.

The U.S. cannot afford to let this situation continue. A systematic strategy for routine monitoring and reporting on quality, as well as the information systems needed to support such activities, will be essential if we are to preserve the best of the American health care system while striving to improve the efficiency with which high quality services are provided. This strategy could be organized by the federal government. It could be the product of the non-profit or for-profit sectors. It could involve coordination between all three. Regardless, the strategy will need to cover the aspects of quality that patients care about; collect data in a way that is manageable, reasonable, and affordable; and produce information in a format that is useful for those who are in a position to improve quality.

The U.S. is capable of setting up a quality measurement system that can provide the multiple participants in the health care system with the information they need to make sure that the health care system provides and continues to provide high quality care. In light of the changes that the health care system has been experiencing, a strategy to measure quality is needed now.

IX. TABLES

TABLE 1: Examples of Quality of Preventive Health Care in the United States

Health Care ServicesSample descriptionData SourceEvidence of QualityRef.ImmunizationsChildrenRoutine childhood vaccine (1995 schedule)3 Polio; 4 Diphtheria, Tetanus, Pertussis; 3 Haemophilus Influenzae type b; and 1 Measles, Mumps, Rubella by 18 months old.Nationally representative sample of US children ages 19-35 months; 31,997 parents completed phone interviewsNational Immunization Survey (NIS), 199576% received all recommended vaccines by 18 months of age.78AdultsInfluenza vaccineAnnual vaccination of all persons 65 years and older.People 65 years and older; ~8,000 respondentsNational Health Interview Survey (NHIS), 199352% received annual influenza vaccine. 37Pneumococcal vaccineOne-time vaccination for all persons 65 years and older.Same as aboveSame as above 28% received pneumococcal vaccine.37 Cancer ScreeningBreast CancerClinical breast examination (CBE) and mammographyRecommendations vary. In 1989, the USPSTF recommended an annual CBE for women 40 years and older, and mammography every 1-2 years for women 50-75 years old. In 1996, it recommended mammography every 1-2 years with or without annual CBE for women 50-69 years old.21,601 nationally representative women age 50 years and olderBehavior Risk Factor Surveillance System, 199257% had CBE in the prior year; 45% had mammography in the prior year; 39% had both examinations in the prior year.80Cervical CancerPapanicolaou (Pap) smearWomen with an intact uterus should have a Pap smear after initiation of sexual intercourse and every 1-3 years thereafter. Some organizations recommend starting Pap smears for all women who have reached 18 years old, regardless of sexual history.Women age 18 years and older included in a nationally representative sample of 128,412NHIS, 199267% of women had a Pap smear in the past 3 years.81 Colon CancerFecal occult blood testing (FOBT) and proctosigmoidoscopyVarious recommendations. In 1980, the American Cancer Society recommended annual FOBT starting at 50 years old. Some other organizations made similar recommendations. In 1989, the USPSTF did not make recommendations, but in 1996, it recommended annual FOBT, sigmoidoscopy (periodicity unspecified), or both starting at 50 years old.People age 40 years and older included in a nationally representative sample of 128,412 peopleSame as above 14% of men and 27% of women had FOBT in the past year. 11% of men and 7% of women had proctosigmoidoscopy in the past 3 years. 81FOBT and proctosigmoidoscopySame as above.388 women age 40-65 years selected from group practices because they had chronic uncomplicated hypertension or were receiving preventive care4 group practices in Massachusetts, 1985-198751%-59% of women had biannual fecal occult blood testing or flexible sigmoidoscopy every 5 years.82 Cardiac Risk FactorsSmoking cessationCounselingTobacco cessation counseling should be offered on a regular basis to all patients who smoke cigarettes, pipes, or cigars, and to those who use smokeless tobacco.43,732 smokers age 18 years and olderNHIS, 199137% were advised to quit smoking.83CholesterolBlood cholesterol testingIn 1988, the National Heart, Lung, and Blood Institute recommended routine cholesterol screening at least every 5 years starting at 20 years old. In 1989, the USPSTF recommended periodic screening for middle-aged men, and in 1996, it recommended periodic screening for men age 35-65 years and women age 45-65 years.1,604 adults in a nationally representative sampleTelephone survey by the National Heart, Lung, and Blood Institute, 199065% of adults surveyed in 1990 had a blood cholesterol test in the previous 5 years; 51% had the test in the prior year.84 Blood cholesterol testingSame as above.People age 20 years and older in 47 states and Washington, DC; sample sizes for individual states vary from 670 to 3190 peopleCDC’s Behavioral Risk Factor Surveillance System data, 199164% of adults in 1991 had cholesterol screening in the previous 5 years.85 General Preventive CareWell child careThe American Academy of Pediatrics recommends routine history, physical examination, screening tests, and anticipatory guidance throughout childhood.2,024 Medicaid outpatient records of 135 providers were abstractedMaryland Medicaid claims data, 198835%-60% of well child care was of poor technical quality.86Well adult care:Patients should have the following preventive servicesómedical history and physical examination every 1-3 years for people age 19-64 years old and every year for people age 65 years and older.Same as aboveSame as above45%-55% of well adult care was of poor technical quality.86 TABLE 2: Examples of Quality of Acute Health Care in the United StatesHealth Care ServicesSample descriptionData SourceEvidence of QualityRef.Antibiotic UseCommon coldsAlmost all colds are caused by a virus, for which antibiotics are not an effective treatment.1,439 patients seen for the common cold (acute nasopharyngitis) in 2,171 outpatient and ER visitsKentucky Medicaid claims data, July 1993-June 199460% of patients filled prescriptions for antibiotics for the common cold.40Upper respiratory tract infections Same as above.Nationally representative sample of 3,000 office-based physicians in the U.S. asked about use of antibioticsNational Ambulatory Medical Care Survey (NMACS), 199217,922 prescriptions (16% of all prescriptions) were written for upper respiratory tract infections.87Pharyngitis, nasal congestion, and other upper respiratory tract infectionsSame as above.Same as aboveSame as aboveOver 70% of patients received antibiotic prescriptions for pharyngitis not specified as caused by Streptococcus; over 50% received them for rhinitis, and over 30% received them for nonspecific upper respiratory tract infection, cough, or cold.88Respiratory IllnessPneumoniaNationally representative sample of 14,012 patients hospitalized with any of five conditions including pneumoniaMedicare enrollees, 1985-1986For pneumonia, appropriate components of care were documented for 52%-79% of patients.13 Pneumonia445 children younger than 18 years old admitted to hospital with pneumonia5 communities in the greater Boston area, 19869.4% of admissions were inappropriate.89Bronchitis/asthma1038 children younger than 18 years old admitted to hospital with bronchitis/asthmaSame as above4.4% of admissions were inappropriate.89Otitis Media (middle ear infection)Treatment2,204 Medicaid outpatient records of 135 providers were abstractedMaryland Medicaid claims data, 198810%-40% of care for otitis media was of poor quality.86Use of tympanostomy tubesThe major indications for tympanostomy tube placement include refractory middle ear infection, chronic mastoiditis, etc.6,611 children younger than 16 years of ageOtorhino-laryngology practices from 49 states and the District of Columbia, January 1990-July 199123% of tube insertions were inappropriate.90 Hip FracturesHip fractureNationally representative sample of 14,012 patients hospitalized with any of five conditions including hip fractureMedicare patients, 1985-1986For hip fracture, appropriate components of care were documented for 67%-94% of patients.13Prenatal CareMedical history9,924 women who had live births were surveyed about their prenatal careNational Maternal and Infant Health Survey (NMIS), 198880% were asked about health history.91Physical examinationSame as aboveSame as above98% had their weight and height measured, 96% received blood pressure measurement, and 86% received a physical examination during the first or second visit.91Laboratory testsSame as aboveSame as above79% received blood tests and 93% received urinalysis during the first or second visit91Medical history, physical examination, and laboratory testsSame as aboveSame as above56% received all of the evaluations listed above.91Counseling about nutrition and weight gainSame as aboveSame as above97% were counseled about vitamins, 93% were counseled about diet, and 72% were counseled about proper weight gain during pregnancy.91Counseling about alcohol, tobacco, and illicit drugsSame as aboveSame as above68% were counseled to stop drinking alcohol, 69% to stop smoking, 65% to stop use of illegal drugs.Counseling about breastfeedingSame as aboveSame as above53% were counseled about breastfeeding.91All types of counselingSame as aboveSame as above32% received all of the counseling listed above regarding vitamins, diet, weight gain, alcohol, smoking, illegal drugs, breastfeeding. Routine prenatal screening tests (e.g., hepatitis B surface antigen)24,170 women who gave birth among all women enrolled in 6 health maintenance organizations (HMOs)6 HMOs in 6 states, August 1989-July 199082% received routine prenatal screening tests.34Other prenatal care (e. g., blood pressure measurement at each visit)Same as aboveSame as above84% received other routine prenatal care.34 Care for complications of pregnancy (e.g., gestational diabetes)Same as aboveSame as above70% received proper care for complications.34Check for protein in urine to evaluate for preeclampsia, a serious complication of pregnancy2,336 Medicaid inpatient and 823 prenatal care records abstracted2 California counties,1985Testing provided at 75%-83% of appropriate visits.92Recording of gestational ageSame as aboveSame as aboveGestational age recorded at 76%-95% of appropriate visits.92Assessment of fetal heart tones after 18 weeks of gestationSame as aboveSame as aboveFetal heart tones assessed at 81%-93% of appropriate visits.92Follow-up for low hematocrit (a measure for anemia)Same as aboveSame as aboveFollow-up for low hematocrit performed at 32%-51% of appropriate visits.92Follow-up of high blood pressureSame as aboveSame as aboveFollow-up for high blood pressure performed at 31%-53% of appropriate visits.92Follow-up of proteinuria (protein in urine)Same as aboveSame as aboveFollow-up performed for proteinuria at 41%-65% of appropriate visits.92

TABLE 3: Examples of Quality of Chronic Health Care in the United States

Health Care ServicesSample DescriptionData SourceEvidence of QualityRef.DiabetesDilated eye examination for retinopathyAnnual dilated eye examination starting at time of diagnosis of noninsulin-dependent diabetes mellitus (NIDDM) and 5 years after diagnosis of insulin-dependent diabetes mellitus (IDDM).Nationally representative sample of 2,392 civilian, noninstitutionalized US population 18 years of age and older with IDDM (124 patients), NIDDM treated with insulin (922 patients), and NIDDM not treated with insulin (1,346 patients)NHIS, 198949% had a dilated eye examination in the past year; 66% had the examination in the past two years;61% and 57% of patients at high risk of vision loss because, respectively, they already had retinopathy or because they had longterm diabetes had an examination in the past year.44Any eye examination (including nondilated for retinopathy)Dilated eye examination is recommended, as described above, but any eye examination is also reported to determine whether any effort was made to assess for retinopathy.Same as aboveSame as above61% had an eye examination in the past year; 80% had an examination in the past 2 years.44Eye exam by ophthalmologistSame as dilated eye examination above.97,388 Medicare patients with diagnosis of diabetes mellitusMedicare claims data from 3 states (Alabama, Iowa, Massachusetts), July 1990-June 199154% did not have an ophthalmologic examination during the past year.93Hemoglobin A1CHemoglobin A1C (or glycosylated hemoglobin) is a blood test which reflects the metabolic control of diabetes. The test should be performed at least once a year.Same as aboveSame as above84% did not receive a hemoglobin A1C test during the past year.93Cholesterol screeningSame as aboveSame as above45% did not receive blood cholesterol screening during the past year. HypertensionTreatment for hypertension Hypertension (or high blood pressure) is a leading risk factor for coronary heart disease, congestive heart failure, stroke, ruptured aortic aneurysm, renal disease, and retinopathy, all of which contribute to high morbidity and mortality. Adequate hypertension control is defined as a mean blood pressure less than 150/90 in adults.246 patients with chronic uncomplicated hypertension with mean age of 56 years, in fee-for-service (FFS) and health maintenance organization (HMO) practices4 group practices in Massachusetts, 1985-198741% of FFS patients had their hypertension controlled, and 54% of HMO patients had their hypertension controlled.82 Treatment for hypertensionSame as above.Nationally representative sample of people with hypertension (sample number not available)National Health and Nutrition Examination Survey III (unpublished data provided by the Centers for Disease Control, National Center for Health Statistics), 1988-199145% of people with hypertension did not have blood pressure under control; 79% when using very strict criteria.94Treatment for hypertensionSame as above8,697 adults with hypertension from a nationally representative sample of adults age 18 years and olderNHIS, 199080% of adults with hypertension reported taking at least one action to control hypertension (e.g., taking medication, decreasing salt, losing weight, and exercising); 89% received advice from a physician about controlling hypertension95DepressionTreatment20,291 adults aged 18 years and older who were interviewedNational Institute of Mental Health Epidemiologic Catchment Area (5 sites: Connecticut, Massachusetts, Missouri, North, Carolina, and California), 1980-198545% of adults with depression received therapeutic care.96Treatment with minor tranquilizers and antidepressant medicationsThere is no evidence that minor tranquilizers are effective for depression.364 patients with current depressive disorder or depressive symptomsMedical Outcomes Study in Los Angeles, Boston, and Chicago, February 1986-October 198619% of patients were treated with minor tranquilizers, and 11% were treated with a combination of minor tranquilizers and antidepressant medications.42Treatment with antidepressant medicationsSame as aboveSame as above33% of patients discharged with antidepressants had doses below recommended level.41Admission for depressionAppropriate reasons for admission include depression, medical condition meriting acute care, comorbid major psychiatric disorder, or medical reasons, precluding outpatient care for depression.2,746 depressed elderly patients hospitalized in 297 general medical hospitals before or after implementation of Medicare’s Prospective Payment System (PPS)Prospective Payment System Quality of Care (PPSQOC) data from 5 states (California, Florida, Indiana, Pennsylvania, and Texas), 1981-1982 and 1986-19867% were admitted for inappropriate reasons.15 Admission assessmentAdmission assessments include documenting suicide status and cognitive status, evaluating for psychosis, taking psychiatric history, listing prior medications, examining heart sounds, and performing a complete neurologic examination.Same as aboveSame as above23% had inadequate psychological assessment, 26% did not have cognitive assessment, 50% did not have assessment of psychosis, 19% did not have documentation of psychiatric history, 46% did not have documentation history of suicide attempts or risks, 24% did not have documentation of prior or current medications.15HysterectomyHysterectomyHysterectomy is the surgical removal of the uterus. The major indications for hysterectomy include recurrent uterine bleeding, leiomyomas, endometrial hyperplasia, endometrial polyps, pelvic relaxation, endometriosis, chronic pelvic pain, and dysmenorrhea.642 women age 20 years and older who underwent nonemergency, nononcologic hysterectomies in seven managed care organizations7 HMOs in 6 states (California, Colorado, Florida, Minnesota, New York, and Washington), August 1989-July 199016% of hysterectomies were performed for inappropriate reasons.2Cardiac DiseaseCoronary Artery DiseaseCoronary angiographyCoronary angiography is a procedure involving selective injection of a radiographic contrast agent into the coronary arteries to evaluate their anatomy. The most frequent appropriate indications include valvular disease and unstable angina.Medicare patients who underwent coronary angiography (1,677 procedures)Medicare physician claims data in 8 states, 198117% of coronary angiographies were inappropriate. 97Coronary angiographySame as above.352 patients with positive exercise stress test who met explicitly defined criteria for coronary angiographyMedical records from 4 teaching hospitals in Los Angeles, January 1990-June 199143% of patients received coronary angiography within 3 months of determined need; 56% received coronary angiography within 12 months of determined need.24Coronary artery bypass graft (CABG)In CABG surgery, damaged blood vessels supplying the heart are replaced with transplanted vessels from elsewhere in the body. The most frequently used appropriate indications include chronic stable angina with left main coronary artery disease or 3-vessel coronary artery disease and unstable angina.386 patients who underwent CABG in 3 hospitalsNational Institute of Health Consensus Development Program data 1979, 1980, and 198214% of CABG surgeries were done for inappropriate reasons.98CABGSame as aboveRandom sample of 1,156 patients who had isolated CABG surgery12 academic centers in 10 states, 19901.6% of CABG surgeries were inappropriate.99 CABGSame as above1,338 patients who had isolated CABG surgery in New York15 hospitals providing CABG procedure in New York State, 19902.4% of CABG surgeries were performed for inappropriate reasons.4Percutaneous transluminal coronary angioplasty (PTCA)The major indication for PTCA include chronic stable angina, unstable angina, myocardial infarction, post-myocardial infarction, flash pulmonary edema, near sudden death, and ventricular arrhythmia.Random sample of 1306 patients undergoing PTCA in New York in 199015 randomly selected hospitals providing PTCA in New York state, 19904% of PTCA were inappropriate.3Heart AttackTreatment with aspirinAspirin is an effective, inexpensive, and safe treatment for a heart attack. Aspirin therapy reduces short-term mortality in patients with suspected heart attack by 23%.Medicare enrollees hospitalized with heart attack who were “ideal” candidates for treatment with aspirin in the first two days (10,018 patients)Cooperative Cardiovascular Project Pilot data in 4 states (Alabama, Connecticut, Iowa, and Wisconsin), June 1992-February 1993Only 61% of patients received aspirin within the first 2 days of hospitalization.100Treatment with aspirinSame as above.Medicare enrollees hospitalized with principal diagnosis of heart attack who were “ideal” candidates for treatment with aspirin(7,486 patients)Medicare’s National Claims History File data in 4 states (Alabama, Connecticut, Iowa, and Wisconsin), June 1992-February 199383% received aspirin during hospitalization; 77% received aspirin at discharge.65Treatment with thrombolytics Thrombolytics are medications which break down some of the blockage in the heart vessels that causes a heart attack. Thromolytic therapy can reduce in-hospital mortality from a heart attack by up to 50% when administered within the first hour of the onset of symptoms, with much of this benefit maintained for one or more years.Same as above for enrollees who were “ideal” candidates for treatment with thrombolytic agents (1,105 patients)Same as above70% received thrombolytics during hospitalization. 65HeparinHeparin should be given to patients with heart attack.Same as above for enrollees who were “ideal” candidates for treatment with heparin (9,857)Same as above69% received heparin during hospitalization.65Intravenous nitroglycerinIntravenous nitroglycerin should be given to patients with heart attack who have persistent pain, provided that there is no hypotension.Same as above for enrollees who were “ideal” candidates for treatment with intravenous nitroglycerin (1,754 candidates)Same as above74% received intravenous nitroglycerin during hospitalization.65 Avoidance of calcium channel blockers for patients with a contraindication (i.e., low left ventricular ejection fraction, shock, or severe congestive heart failure)Medicare enrollees hospitalized with principal diagnosis of heart attack who should avoid calcium channel blocker medication (785 patients)Same as above21% who were ineligible for calcium channel blockers received them anyway.65Beta blocker therapy3,737 Medicare enrollees with heart attacks who were eligible for treatment with beta blockersMedicare hospital admission data in New Jersey, 1987-199221% of patients received beta blocker within 90 days of a heart attack; mortality rate for patients with treatment was 43% less than that of patients without treatment.101Permanent cardiac pacemaker Among the major indications for pacemaker insertion include acquired complete heart block and sick sinus syndrome (i. e., the heart is beating too rapidly or too slowly).Medicare enrollees in 6 university hospitals, 11 university-affiliated hospitals, and 13 community hospitals who underwent a total of 382 pacemaker implantationsMedicare claims data in Philadelphia, January 1983-June 198320% of pacemaker implantations were not indicated.102Carotid ArteriesCarotid endarterectomyMedicare enrollees who underwent carotid endarterectomy (1,302 procedures) Medicare physician claims data in 8 states, 198132% of carotid endarterectomies were inappropriate.97Gastrointestinal DiseaseUpper gastrointestinal tract endoscopyAmong the major indications for upper gastrointestinal endoscopy include difficulty swallowing (dysphagia), unexplained weight loss, atypical chest pain, vomiting blood (hematemesis), etc.Medicare enrollees who underwent upper gastrointestinal tract endoscopy (1,585)Medicare physician claims data in 8 states, 198117% of upper gastrointestinal tract endoscopies were inappropriate.97CataractsCataract surgery1,020 patients in 10 academic medical centers who underwent a total of 1,139 cataract surgeries10 academic medical centers, 19902% of cataract surgeries were inappropriate.104HIV/AIDSRelationship between hospital experience and mortality300 patients with acquired immunodeficiency syndrome (AIDS)Massachusetts AIDS Surveillance Program data from 40 hospitals, 1987-1988Patients in low-experience hospitals were 2.16 times more likely to die compared to those in high-experience hospitals; adjusted relative risk increased to 2.92.105Relationship between physician experience and mortality125 primary care physicians’ clinical experiences in treating 403 adult male patients with AIDS was analyzedGroup Health HIV/AIDS Surveillance Database, 1984-1994Patients of physicians with least experience survived 14 months after AIDS diagnosis compared to 26 months for patients of physicians with most experience.20 Preventable DeathsEvaluation of preventable deaths182 patients who died in hospitals from stroke, pneumonia, or heart attackMedical records from 12 hospitals, 198514% of deaths were preventable due to inadequate diagnosis or treatment.106Multiple ConditionsCare for asthma, diabetes mellitus, and hypertension2,024 Medicaid adult and child outpatient records of 135 providersMaryland Medicaid claims data, 198830%-60% of care showed poor technical quality.86Adverse EventsAdverse events30,121 randomly selected medical records51 randomly selected acute care, nonpsychiatric hospitals in New York state, 19843.7% of hospitalizations had adverse events, and 27.6% of adverse events were due to negligence.107Medications4,031 adult admissions to a stratified random sample of 11 medical and surgical unitsMedical records of 2 tertiary care hospitals in Boston, February 1993-July 1993There were 1.8 preventable adverse drugs events (ADEs) per 100 admissions, of which 20% were life-threatening, 43% were serious, and 37% were significant. There were an additional 5.5 preventable potential ADEs per 100 admissions.108

TABLE 4: Examples of Quality of Health Care Outside the United States

Health Care ServicesSample DescriptionData SourceEvidence of QualityRef.CanadaCoronary angiography533 randomly selected patients age 59 years and older who underwent coronary angiographyMedical records of all hospitals performing coronary angiography in 2 Canadian provinces (Ontario and British Columbia), April 1989-March 19909% of coronary angiographies were inappropriate using Canadian expert panel’s criteria; 5% were inappropriate using US criteria.109Coronary artery bypass graft (CABG)556 randomly selected patients who underwent CABGSame as above4% of procedure were inappropriate using Canadian expert panel’s criteria; 3% were inappropriate using US criteria.109IsraelCholecystectomy702 patients who underwent cholecystectomyMedical records of four hospitals in Israel 198612% of cholecystectomies were inappropriate.110NetherlandsPercutaneous transluminal coronary angioplasty (PTCA) and CABG3,207 patients of whom 1,589 underwent PTCA and 1,618 underwent CABGDutch inventory of invasive coronary atherosclerosis treatment (DUCAT) data 199229% of PTCA were inappropriate; 4% of CABG surgeries were inappropriate.111SwitzerlandUpper gastrointestinal endoscopy8,135 clinic visits by 611 patients age 16 years and older with complaints of upper digestive symptoms, 63 of whom underwent endoscopyMedical records of 20 primary care practices and a university-based multispecialty outpatient clinic in Switzerland 1994-199512% of patients with indications for test did not receive test.103United KingdomCoronary angiography and CABG320 patients who underwent coronary angiography and 319 patients who underwent CABG; all patients age 25 years and olderMedical records of hospitals in Trent region in the United Kingdom 1987-198821% of coronary angiography and 16% of CABG were inappropriate.112

X. ACKNOWLEDGMENTS

We are indebted to Molla Donaldson, Ph.D., Rachel Spilka, Ph.D., and Joseph H. Triebwasser, M.D., for comments on drafts of this paper. We are also indebted to Lauren N. Nguyen, M.P.H., and Sinaroth Sor, M.D., for research assistance.

XI. REFERENCES
1. Lohr K, ed. Medicare: A Strategy for Quality Assurance. Washington D.C.: National Academy Press; 1990:441.
2. Bernstein S, McGlynn E, Siu A, et al. The appropriateness of hysterectomy, a comparison of care in seven health plans. Journal of the American Medical Association. 1993;269:2398-2402.
3. Hilborne L, Leape L, Bernstein S, et al. The appropriateness of use of percutaneous transluminal coronary angioplasty in New York state. Journal of the American Medical Association. 1993;269:761-765.
4. Leape L, Hilborne L, Park R, et al. The appropriateness of use of coronary artery bypass graft surgery in New York state. Journal of the American Medical Association. 1993;269:753-760.
5. Lohr K, Brook R, Kamberg C, et al. Use of medical care in the Rand health insurance experiment, diagnosis- and service-specific analyses in a randomized controlled trial. Medical Care. 1986;24:S1-S87.
6. Mark BA, Burleson DL. Measurement of patient outcomes: data availability and consistency across hospitals. JONA. 1995;25:52-59.
7. Etheredge L. The structure and accountability for Medicare health plans: government, the market, and professionalism. In: Jones S, Lewin M, eds. Improving the Medicare Market Adding Choices and Protections. Washington, D.C.: National Academy Press; 1996:137-158.
8. Winslow C, Solomon D, Chassin M, Kosecoff J, Merrick N, Brook R. The appropriateness of carotid endarterectomy. The New England Journal of Medicine. 1988;318:721-727.
9. Brook R. Doing more good than harm: the evaluation of health care interventions. In: Warren K, Mosteller F, eds. Using Scientific Information to Improve Quality of Health Care. New York: Annals of The New York Academy of Science; 1993:74-78.
10. Brook RH. Medicare quality and getting older: a personal essay. Health Affairs. 1995;14:73-81.
11. Kahn K, Keeler E, Sherwood M, et al. Comparing the outcomes of care before and after implementation of the DRG-based prospective payment system. Journal of the American Medical Association. 1990;264:1984-1988.
12. Kosecoff J, Kahn K, Rogers W, et al. Prospective payment system and impairment at discharge, the ‘Quicker-and-Sicker’ story revisited. Journal of the American Medical Association. 1990;264:1980-1983.
13. Kahn K, Rogers W, Rubenstein L, et al. Measuring quality of care with explicit process criteria before and after implementation of the DRG-based prospective payment system. Journal of the American Medical Association. 1990;264:1969-1973.
14. Rogers W, Draper D, Kahn K, et al. Quality of care before and after implementation of the DRG-based prospective payment system: a summary of effects. Journal of the American Medical Association. 1990;264:1989-1994.
15. Wells K, Rogers W, Davis L, et al. Quality of care for hospitalized depressed elderly patients before and after implementation of the Medicare prospective payment system. American Journal of Psychiatry. 1993;150:1799-1768.
16. Donabedian A. Explorations in Quality Assessment and Monitoring, Volume 1: The Definition of Quality and Approaches to Its Assessment. Ann Arbor, MI: Health Administration Press; 1980.
17. Brook R, Park R, Chassin M, Kosecoff J, Keesey J, Solomon D. Carotid endarterectomy for elderly patients: predicting complications. Annals of Internal Medicine. 1990;113:747-753.
18. Grumbach K, Anderson G, Luft H, Roos L, Brook R. Regionalization of cardiac surgery in the United States and Canada: outcomes, choice, and geographic access. Journal of the American Medical Association. 1995;274:1282-1288.
19. Hannan E, Racz M, Ryan T, et al. Coronary angioplasty volume-outcome relationships for hospitals and cardiologists. Journal of the American Medical Association. 1997;277:892-898.
20. Kitahata M, Koepsell T, Deyo R, Maxwell C, Dodge W, Wagner E. Physicians’ experience with the Acquired Immunodeficiency Syndrome as a factor in patients’ survival. The New England Journal of Medicine. 1996;334:701-706.
21. Luft H, Garnick D, Mark D, McPhee S. Hospital Volume, Physician Volume, and Patient Outcomes: Assessing the Evidence. Ann Arbor: Health Administration Press; 1990:406.
22. Brook R, Chassin M, Fink A. A method for detailed assessment of the appropriateness of medical technologies. International Journal Technology Assessment in Health Care. 1986;2:53-63.
23. Kahan K, Bernstein S, Leape L, et al. Measuring the necessity of medical procedures. Medical Care. 1994;32:357-365.
24. Laouri M, Kravitz R, Berstein S, et al. Underuse of coronary angiography: application of a clinical method. International Journal for Quality in Health Care. 1997;9:5-22.
25. Kravitz R, Laouri M, Kahan J, et al. Validity of criteria used for detecting underuse of coronary revascularization. Journal of the American Medical Association. 1995;274:632-638.
26. Aharony L, Strasser S. Patient satisfaction: what we know about and what we still need to explore. Medical Care Review. 1993;50:49-79.
27. Cleary P, McNeil B. Patient satisfaction as an indicator of quality care. Inquiry. 1988;25:25-36.
28. Davies A, Ware J. Involving consumers in quality of care assessment. Health Affairs. 1988;7:33-48.
29. Hayward R, Bernard A, Rosevear J, Anderson J, McMahon L. An evaluation of generic screens for poor quality of hospital care on a general medicine service. Medical Care. 1993;31:394-402.
30. Bartlett E, Grayson M, Barker R, Levine D, Golden A, Libber S. The effects of physician communications skills on patient satisfaction: recall, and adherence. Journal of Chronic Diseases. 1984;37:755-763.
31. Deyo R, Inui T. Dropouts and broken appointments, A literature review and agenda for future research. Medical Care. 1980;18:1146-1157.
32. Iezzoni L. An introduction to risk adjustment. American Journal of Medical Quality. 1996;11:S8-S11.
33. Ventura S, Martin J, Curtin S, Mathews T. Report of final natality statistics, 1995. Monthly Vital Statistics Report. 1997;45:1-81.
34. Murata P, McGlynn E, Siu A, et al. Quality measures for prenatal care, a comparison of care in six health care plans. Archives of Family Medicine. 1994;3:41-49.
35. Todd J. Rheumatic Fever. In: Behrman R, Kliegman R, Arvin A, eds. Nelson Textbook of Pediatrics, 15th Edition. Philadelphia, PA: W.B. Saunders Company; 1996:754-760.
36. U.S. Preventive Services Task Force. Guide to Clinical Preventive Services, 2nd edition. Alexandria: International Medical Publishing; 1996.
37. CDC. Influenza and pneumococcal vaccination coverage levels among persons aged > 65 years — United States, 1973-1993. Morbidity and Mortality Weekly Reports. 1995;44:506-515.
38. Begley S. The end of antibiotics. Newsweek. 1994:46-51.
39. CDC. Addressing emerging infectious disease threats: a prevention strategy for the United States. Morrbidity and Mortality Weekly Report. 1994;43:1-18.
40. Mainous A, Hueston W, Clark J. Antibiotics and upper respiratory infection, do some folks think there is a cure for the common cold? The Journal of Family Practice. 1996;42:357-361.
41. Wells K, Norquist G, Benjamin B, Rogers W, Kahn K, Brook R. Quality of antidepressant medications prescribed at discharge to depressed elderly patients in general medical hospitals before and after prospective payment system. General Hospital Psychiatry. 1994;16:4-15.
42. Wells K, Katon W, Rogers B, Camp P. Use of minor tranquilizers and antidepressant medications by depressed outpatients: results from the medical outcomes study. American Journal of Psychiatry. 1994;151:694-700.
43. Depression Guideline Panel. Depression in Primary Care: Volume 2. Treatment of Major Depression. Clinical Practice Guideline, Number 5. pub. no. 93-0551. Rockville, MD: U.S. Department of Health and Human Services, Public Health Service, AHCPR; 1993.
44. Brechner R, Cowie C, Howie L, Herman W, Will J, Harris M. Ophthalmic examination among adults with diagnosed diabetes mellitus. Journal of the American Medical Association. 1993;270:1714-1718.
45. Miller R, Luft H. Managed care plan performance since 1980; a literature analysis. Journal of the American Medical Association. 1994;271:1512-1519.
46. Miller R, Luft H. Managed care: past evidence and potential trends. Frontiers of Health Services Management. 1993;9:3-37.
47. Berwick D. Continuous improvement as an ideal in health care. The New England Journal of Medicine. 1989;320:53-56.
48. Kritchevsky S, Simmons B. Continuous quality improvement; concepts and applications for physician care. Journal of the American Medical Association. 1991;266:1817-1823.
49. O’Connor G, Plume S, Olmstead E, et al. A regional intervention to improve the hospital mortality associated with coronary artery bypass graft surgery. Journal of the American Medical Association. 1996;275:841-846.
50. Leape L, Bates D, Cullen D, et al. Systems analysis of adverse drug events. Journal of the American Medical Association. 1995;274:35-43.
51. Field M, Lohr K, eds. Clinical Practice Guidelines: Directions for a New Program. Washington, DC: National Academy Press; 1990.
52. National Committee for Quality Assurance. HEDIS 3.0, Volume 1: Narrative: What’s in It and Why It Matters. Washington, DC: National Committee for Quality Assurance; 1997.
53. Chassin M, Hannan E, DeBuono B. Benefits and hazards of reporting medical outcomes publicly. The New England Journal of Medicine. 1996;334:394-398.
54. Hannan E, Kilburn H, Racz M, Shields E, Chassin M. Improving the outcomes of coronary artery bypass surgery in New York state. Journal of the American Medical Association. 1994;271:761-766.
55. Green J, Wintfeld N. Report cards on cardiac surgeons: assessing New York state’s approach. The New England Journal of Medicine. 1995;332:1229-1232.
56. Jencks S. Can large-scale interventions improve care? Journal of the American Medical Association. 1997;277:419-420.
57. Ghali W, Ash A, Hall R, Moskowitz M. Statewide quality improvement initiatives and mortality after cardiac surgery. Journal of the American Medical Association. 1997;277:379-382.
58. Schneider E, Epstein A. Influence of cardiac-surgery performance reports on referral practices and access to care: a survey of cardiovascular specialists. The New England Journal Of Medicine. 1996;335:251-256.
59. Berwick D, Wald D. Hospital leaders’ opinions of the HCFA mortality data. Journal of the American Medical Association. 1990;263:247-249.
60. Kent C. HCFA’s mortality data: what do they say about quality? Medicine and Health Perspectives. 1994;48:1-4.
61. Dans P, Weiner J, Otter S. Peer review organizations: promises and pitfalls. New England Journal of Medicine. 1985;313:1131-1137.
62. Luce J, Bindman A, Lee P. A brief history of health care quality assessment and improvement in the United States. Western Journal of Medicine. 1994;160:263-268.
63. Hartz A, Gottlieb M, Kuhn E, Rimm AA. The relationship between adjusted hospital mortality and the results of peer review. Health Services Research. 1993;27:765-777.
64. Jencks S, Wilensky G. The health care quality improvement initiative. Journal of the American Medical Association. 1992;268:900-903.
65. Ellerbeck E, Jencks S, Radford M, et al. Quality of care for Medicare patients with acute myocardial infarction; a four-state pilot study from the Cooperative Cardiovascular Project. Journal of the American Medical Association. 1995;273:1509-1514.
66. Jencks S. Changing health care practices in Medicare’s health care quality improvement program. Journal on Quality Improvement. 1995;21:343-347.
67. Feldman S, Rundall T. PROs and the health care quality improvement initiative: insights from 50 cases of serious medical mistakes. Medical Care Review. 1993;50:123-152.
68. Armstead R, Elstein P, Gorman J. Toward a 21st century quality-measurement system for managed-care organizations. Health Care Financing Review. 1995;16:25-37.
69. Vladeck B. From the Health Care Financing Administration Managed Care and Quality. Journal of the American Medical Association. 1995;273:1483.
70. Felt-Link S, St. Peter R. Quality assurance for Medicaid managed care. Health Affairs. 1997;16:248-252.
71. Consumers…largest consumer health survey in history [news]. Hospitals and Health Networks; 1995:26.
72. Guadaguoli E, McNeil B. Outcomes research: hope for the future or the latest rage? Inquiry. 1994;31:14-24.
73. Iglehart JK. Managed care and mental health. The New England Journal of Medicine. 1996;334:131-135.
74. CDC. Quality standards compliance for mammography facilities. Morbidity and Mortality Weekly Report. 1994;43:596-597.
75. Korn J, Casey-Paul A, Lazovich D, Ball J, Slater J. Impact of the mammography quality standards act on access in Minnesota. Public Health Reports. 1997;112:142-145.
76. Donaldson M, Lohr K, eds. Health Data in the Information Age: Use, Disclosure, and Privacy. Washington, DC: National Academy Press; 1994.
77. Hibbard J, Jewett J. Will quality report cards help consumers? Health Services Research. 1997;16:218-228.
78. CDC. National, state, and urban area vaccination coverage levels among children aged 19-35 months — United States, January-December 1995. Morbidity and Mortality Weekly Reports. 1997;46:176-183.
79. U.S. Preventive Services Task Force. Guide to Clinical Preventive Services, 1st edition. Baltimore: William & Wilkins; 1989.
80. CDC. Mammography and clinical breast examinations among women aged 50 years and older–behavioral risk factor surveillance system, 1992. Morbidity and Mortality Weekly Reports. 1993;42:737-741.
81. CDC. Trends in cancer screening — United States, 1987 and 1992. Morbidity and Mortality Weekly Report. 1995;45:57-61.
82. Udvarhelyi S, Jennison K, Phillips R, Epstein A. Comparison of the quality of ambulatory care for fee-for-service and prepaid patients. Annals of Internal Medicine. 1991;115:394-400.
83. CDC. Physician and other health-care professional counseling of smokers to quit — United States, 1991. Morbidity and Mortality Weekly Report. 1993;42:854-857.
84. Schucker B, Wittes J, Santanello N, et al. Change in cholesterol awareness and action. Archives of Internal Medicine. 1991;151:666-673.
85. CDC. State-specific changes in cholesterol screening — behavioural risk factor surveillance system, 1988-1991. Morbidity and Mortality Weekly Reports. 1993;42:663-667.
86. Starfield B, Powe N, Weiner J, et al. Costs vs quality in different types of primary care settings. Journal of the American Medical Association. 1994;272:1903-1908.
87. McCaig L, Hughes J. Trends in antimicrobial drug prescribing among office-based physicians in the United States. Journal of the American Medical Association. 1995;273:214-219.
88. Dowell S, Schwartz B. Resistant Pneumococci: protecting patients through judicious use of antibiotics. American Family Physician. 1997;55:1647-1654.
89. Payne S, Donahue C, Rappo P, et al. Variations in pediatric pneumonia and bronchitis/ asthma admission rates, Is appropriateness a factor? Archives of Pediatrics and Adolescent Medicine. 1995;149:162-169.
90. Kleinman LC, MD, MPH, Kosecoff J, PhD, Dubois RW, MD, PhD, Brook RH, MD, ScD. The Medical Appropriateness of Tympanostomy Tubes Proposed for Children Younger Than 16 Years in the United States. Journal of the American Medical Association. 1994;271:1250-1255.
91. Kogan M, Alexander G, Kotelchuck M, Nagey D, Jack B. Comparing mothers’ reports on the content of prenatal care received with recommended national guidelines for care. Public Health Reports. 1994;109:637-646.
92. Carey T, Weis K, Homer C. Prepaid versus traditional Medicaid plans: lack of effect on pregnancy outcomes and prenatal care. Health Services Research. 1991;26:165-181.
93. Weiner J, Parente S, Garnick D, Fowles J, Lawthers A, Palmer H. Variation in office-based quality; a claims-based profile of care provided to Medicare patients with diabetes. Journal of the American Medical Association. 1995;273:1503-1508.
94. The Joint National Committee on Detection E, and Treatment of High Blood Pressure. The fifth report of the Joint National Committee on Detection, Evaluation, and Treatment of High Blood Pressure (JNC V). Archives of Internal Medicine. 1993;153:154-183.
95. CDC. Adults taking action to control their blood pressure — United States, 1990. Morbidity and Mortality Weekly Reports. 1994;43:509-517.
96. Regier D, Narrow W, Rae D, Maderscheid R, Locke B, Goodwin F. The de facto US mental and addictive disorders service system. Archives of General Psychiatry. 1993;50:85-94.
97. Chassin M, Kosecoff J, Park R, et al. Does inappropriate use explain geographic variations in the use of health care services? A study of three procedures. Journal of the American Medical Association. 1987;258:2533-2569.
98. Winslow C, Kosecoff J, Chassin M, Kanouse D, Brook R. The appropriateness of performing coronary artery bypass surgery. Journal of the American Medical Association. 1988;260:505-509.
99. Leape L, Hilborne L, Schwartz J, et al. The appropriateness of coronary artery bypass graft surgery in academic medical centers. Annals of Internal Medicine. 1996;125:8-18.
100. Krumholz H, Radford M, Ellerbeck E, et al. Aspirin in the treatment of acute myocardial infarction in elderly Medicare Beneficiaries; patterns of use and outcomes. Circulation. 1995;92:2841-2847.
101. Soumerai S, McLaughlin T, Hertzmark E, Thibault G, Goldman L. Adverse outcomes of underuse of beta-blockers in elderly survivors of acute myocardial infarction. Journal of the American Medical Association. 1997;277:115-121.
102. Greenspan A, Kay H, Berger B, Greenberg R, Greenspon A, Gaughan M. Incidence of unwarranted implantation of permanent cardiac pacemakers in a large medical population. The New England Journal of Medicine. 1988;318:158-163.
103. Froehlich F, Pache I, Burnand B, et al. Underutilization of upper gastroenterology. Gastroenterology. 1997;112:690-697.
104. Tobacman J, Lee P, Zimmerman B, Kolder H, Hilborne L, Brook R. Assessment of appropriateness of cataract surgery at ten academic medical centers in 1990. Ophthamology. 1996;103:207-215.
105. Stone V, Seage G, Hertz T, Epstein A. The Relation between hospital experience and mortality for patient with AIDS. Journal of the American Medical Association. 1992;268:2655-2661.
106. Dubois R, Brook R. Preventable deaths: who, how often, and why? Annals of Internal Medicine. 1988;109:582-589.
107. Brennan T, Leape L, Laird N, et al. Incidence of adverse events and negligence in hospitalized patients. The New England Journal of Medicine. 1991;324:370-376.
108. Bates D, Cullen D, Laird N, et al. Incidence of adverse drug events and potential adverse drug events. Journal of the American Medical Association. 1995;274:29-34.
109. McGlynn E, Naylor D, Anderson G, et al. Comparison of the appropriateness of coronary angiography and coronary artery bypass surgery between Canada and New York state. Journal of the American Medical Association. 1994;272:934-940.
110. Pilpel D, Frase G, Kosecoff J, Weitzman S, Brook R. Regional differences in appropriateness of cholecystectomy in a prepaid health insurance system. Public Health Review. 1993;20:61-74.
111. Meijler A, Rigter H, Berstein S, et al. The appropriateness of intention to treat decisions for invasive therapy in coronary artery disease in the Netherlands. Heart. 1997;77:219-224.
112. Gray D, Hampton J, Bernstein S, Kosecoff J, Brook R. Audit of coronary angiography and bypass surgery. The Lancet. 1990;335:1317-1320.

More from 1998PolicyStudies

Why The Quality of U.S. Health Care Must Be Improved

NCHC Writer

May 20, 2023

Prepared forThe National Coalition on Health Care ByMark A. Schuster, M.D., Ph.D.Elizabeth A. McGlynn, Ph.D.Robert H. Brook, M.D., Sc.D.RANDSanta Monica, California…
Read more
1998 Policy Studies

Admin

May 20, 2023

As Good as it Should Get: Making Health Care Better in the New Millennium – September 1998By Donald M. Berwick, M.D.,…
Read more
A REALITY CHECK:
THE PUBLIC’S CHANGING VIEWS OF OUR HEALTH CARE SYSTEM
BY
JOEL E. MILLER, DIRECTOR, POLICY
NATIONAL COALITION ON HEALTH CARE

Admin

May 20, 2023

Introduction In January 1997, a national poll of American households commissioned by the National Coalition on Health Care, the…
Read more
As Good As It Should Get:
Making Health Care Better In The New Millennium

Admin

May 20, 2023

By: Donald M. BerwickInstitute For Healthcare Improvement TABLE OF CONTENTS EXECUTIVE SUMMARYINTRODUCTIONREDUCING OVERUSE OF CESAREAN SECTIONS IN GREEN BAYREDUCING COSTS WHILE…
Read more

Why The Quality of U.S. Health Care Must Be Improved

Latest news

Emotional Support Animal 2023: Everything You Need To Know

How To Detox Your Body Naturally & Safely 2023: Tips To Note

How to Stop a Receding Hairline in 6 Effective Ways 2023

What To Eat When You Have Diarrhea: Foods to Eat & Avoid 2023

How To Get A Flat Stomach Fast 2023: Tips To Shed Excess Fat

Healthy Breakfast To Lose Weight & Start The Day Right

More from 1998PolicyStudies

Why The Quality of U.S. Health Care Must Be Improved

1998 Policy Studies

A REALITY CHECK:
THE PUBLIC’S CHANGING VIEWS OF OUR HEALTH CARE SYSTEM
BY
JOEL E. MILLER, DIRECTOR, POLICY
NATIONAL COALITION ON HEALTH CARE

As Good As It Should Get:
Making Health Care Better In The New Millennium