|
|
III.
Conference Conclusions
The following section summarizes
themes and conclusions derived from conference presentations and discussions.
This summary is intended to highlight key points from the sessions and
identify select program initiatives. It also attempts to capture the discussions
around the conference themes that bring to light additional strategies
and questions.
A.
Cultural competence must align its objectives with broader quality-of-care
initiatives to strengthen its position in the national health care agenda.
In his keynote address, Mark
Smith, from the California HealthCare Foundation, urged the cultural competency
movement to join together with the quality movement to bring about real
change in American medicine.
As defined by the Institute
of Medicine, quality health care must be safe, effective, patient-centered,
timely, efficient, and equitable. Quality measurement resides in two domains:
the clinical/technical and patient experience. Dr. Smith gave examples
of specific questions, whose answers he believes “are indisputably measures
of the quality of care. Some of them are reasonable proxies for cultural
competence of institutions and providers and should be used by this movement
for that purpose.”
Questions of Quality.
The Picker survey, for example, measures patient experience with in-patient
care by asking: Did you have confidence and trust in the doctors treating
you? Did doctors talk in front of you as if you weren't there? Did a doctor
or nurse explain the results of tests in a way you could understand? Such
questions can be quite revealing when stratified by ethnicity or language.
For example, if a hospital has an overall satisfaction rate of 80 percent,
but only 40 percent among a certain subgroup, that would point to a group
that deserves special attention.
Other powerful information
includes reports on mortality or preventive services—mammograms, pap smears,
diabetes screening, prostate antigen,—segmented by race or language. An
example of this type of information comes from Far Rockaway, New York,
whose predominantly poor, African American population was found to have
the highest rate of diabetic amputations in the country. As Dr. Smith
noted, “The endocrinologists know there should be no amputations and so
do the general internist and family doctors. An amputation is, until proven
otherwise, prima facie evidence of low-quality care.” He went on to say
that patients themselves should understand that they shouldn’t be getting
amputations. They should know the yearly tests they need to prevent that
outcome.
Patient Safety. Paul
M. Schyve, from the Joint Commission on Accreditation of Healthcare Organizations
(JCAHO), also highlighted the essential role of the patient in another
component of quality health care—assuring patient safety.
Special attention to safety
began in 1995, a year when several serious events occurred in hospitals
with good reputations: the wrong leg cut off, the wrong side of the brain
operated on, fatal overdoses of chemotherapy. Analysis of these and similar
events showed that communication failures were the first and second most
common causes of events that caused death or disability. These failures
might have been in information technology; or in lack of communication
or misunderstanding between caregivers or between caregiver and patient.
When patients and families
feel they’ve engaged in successful communication, they report that they
are more satisfied with the care they receive. More importantly, an atmosphere
that fosters communication actually affects health outcomes. If the caregivers
get the information they need from the patient, and if the patient understands
what is being advised, the outcome improves. When patients fully participate
in decision-making about their care, they increase their own safety.
The commission recently created
the “Speak Up Campaign,” which encourages patients to participate in decisions
about their care. Brochures in both Spanish and English advise patients
to choose an advocate if they feel they cannot speak up for themselves.
Patients are advised to ask questions, know what their medication is treating,
know who is providing the care, and educate themselves about their illness
and treatments.
Closely linked to patient
participation is the need for relevant and understandable public information
on patient safety and overall quality of care in health care institutions.
Dr. Smith acknowledged that not everyone will or can use quality reports
to inform his or her health care choices. However, he listed four reasons
public reporting is important:
- To assist those consumers
who do, can, and will use those reports in choosing and using health
care.
- To guide people who make
wholesale purchasing decisions, such as employers, health plans, and
government agencies.
- To equip health care providers
with benchmarks, performance targets, and a focus for quality improvement.
- To have a way of tracking
the social good that's produced by the vast amount of resources that
go into health care. The themes raised by these speakers were echoed
in many other presentations during the course of the conference. Managed
care executives observed that there is better acceptance of cultural
competence programs when they are framed in the larger context of organizational
quality-of-care goals. Researchers implementing health promotion interventions
noted a more subtle understanding of program strengths and shortcomings
when patients and providers participated collaboratively in quality
assessment and improvement exercises. Several presenters described steps
being taken to modify standard data collection instruments to allow
for more accurate reporting on the perceptions and needs of culturally
diverse populations. In closing, Dr. Smith reminded the audience that
the cultural competence movement must first focus on the quality issues;
second, frame them as quality issues, as doing so will provide more
leverage in the health care debates; and third, use existing metrics
whenever possible as levers to highlight problem areas.
B.
Reducing health disparities is a matter of civil rights.
The primary goals of providing
culturally competent care are to ensure high-quality health care for everyone
and to eliminate racial and ethnic health disparities. Bob Ross, of the
California Endowment, reminded conference participants that at its core,
cultural competence is simply the “right thing to do.” He emphasized,
“Let us not forget for a minute that this is about social justice.”
The Institute of Medicine
(IOM) report, “Unequal Treatment,” also highlights the civil rights aspect
of health disparities. Carolina Reyes, from the California Endowment and
a member of the IOM committee for the report, stated that the report title
was chosen because “it very poignantly speaks to the need for an action-oriented
agenda that the public could not ignore….It stands in contrast to equal
treatment under the law as defined in the Fourteenth Amendment of the
Constitution of the United States.” Similarly, a key recommendation stemming
from the IOM report is civil rights enforcement. Dr. Reyes highlighted
the common themes of “Unequal Treatment” and the “Quality Chasm” report,
which also made reference to health care disparities. As summarized by
Dr. Reyes, the “Quality Chasm” report concluded, “Racial and ethnic health
disparities should be eliminated because racial discrimination is intolerable
by law, most notably the Civil Rights Act of 1964. It’s contrary to our
moral creed and health care ethic.”
As the civil rights enforcement
agency for the US DHHS, the Office for Civil Rights (OCR) sent several
representatives who talked about national origin discrimination, Limited
English Proficiency (LEP) issues, immigrant access, and the role OCR plays
in “breaking down the barriers that individuals of color face in the health
care system.” Sylvia Philpot and Patricia Harper, both from OCR Region
V, discussed the relationship between health disparities and discrimination
and differentiated between those disparities that might be the product
of disparate treatment and those that appear to be nondiscriminatory in
nature. They reported on “promising practices that have proven to be effective
means of persuading providers to take voluntary actions in order to meet
the needs of the racially diverse populations seeking/in need of services.”
Robinsue Frohboese, also of OCR, reported on several projects related
to LEP, including a collaborative Web site, www.lep.gov, that “brings
together in one-stop shopping information about legal responsibilities,
guidance, data, and technical assistance.” A preconference skills-building
session addressed strategies for enforcing legal requirements, including
filing an administrative complaint with OCR, using legislative efforts
to pass bills promoting culturally and linguistically appropriate services,
and working with government agencies to seek reimbursement for language
assistance services.
C.
The National CLAS standards can guide organizational change and assist
with the implementation of cultural competence in a host of health care
settings.
Representatives of health
care organizations—clinics, hospitals, county health departments, and
for-profit and nonprofit managed care organizations—reported on several
years of experience implementing cultural competence activities. Gaining
support for organizational change can be very challenging, but many providers
are using the national standards for Culturally and Linguistically Appropriate
Services (CLAS) to guide the interventions they are implementing.
Various Strategies.
Health organizations are using many strategies to implement cultural competence.
Nilda Chong explained that Kaiser Permanente’s Institute of Culturally
Competent Care captures and disseminates cultural competence expertise
for the entire national network. The institute serves not only as a repository
of information, but also develops health strategies and resource tools
and has identified Centers of Excellence for key populations among its
regional plan partners. Also on the national level, the Washington Business
Group on Health has implemented “The Health Disparities Initiative,” which
seeks to educate companies about the impact of health disparities and
support their efforts to purchase and plan for appropriate health care
that will lead to disparity reductions. Among their approaches are a white
paper framing the business case, a toolkit guide for employers, and an
online resource center.
Many presenters reported that
the first task of any cultural competence initiative is raising basic
awareness about the need for and goals of cultural competence among both
administrative and front-line staff. Carola Green spoke about the Vista
(CA) Community Clinic’s Cultural Awareness Program, which focuses on educating
administrators and business managers within its own organization and among
other health care organizations in the San Diego area. The Los Angeles
County health department has found the work of its Cultural and Linguistic
Work Group in developing standards to be instrumental in defining for
its many providers exactly what cultural competence means and how it can
be implemented.
The list of cultural competence
interventions being implemented in clinical sites across the country grows
each year. Among the many activities are:
- Language services, including
bonuses for bilingual staff, interpreter services (including reimbursement
for providers in some locales), interpreter training, translation of
documents and signage, education of staff on the need for and use of
language services
- Cultural competence education
and training for clinical and administrative staff
- Culturally tailored health
education programs and health disparity-reduction strategies
- Strategic plans and organizational
capacity assessments, staff surveys, direct observation of clinical
operations, performance audits, and data collection
- Development of resource
and educational materials and Web sites
Facilitating acceptance.
The two greatest challenges to implementing organizational cultural competence
strategies remain persuading leadership and staff, and finding resources.
Integrating cultural competence into broader organizational goals and
programs is key, said Kaiser’s Gayle Tang, who led a roundtable discussion
on making the business case for cultural competence. Kelvin Quan, of the
Alameda (CA) Alliance for Health, described how his organization has placed
cultural competence into the framework of larger strategic objectives.
Compliance with state and federal laws and regulations is a strong motivator.
And culturally competent care delivery, he said, can increase patient
compliance and follow-up, decrease no-shows, reduce medication errors,
improve patient satisfaction and membership retention, and improve community
image—all objectives in the larger agendas of improved patient quality
and enhanced corporate value.
In a for-profit health plan,
however, there can be more resistance to new initiatives that do not demonstrate
an immediate return on investment. Gamini Gunawardane, of California’s
Care 1st Health Plan, noted that conflicting requirements for
cultural competence between different state and federal laws and regulations,
and the fact that there is little enforcement, reduce the power of legal
compliance as an effective motivator. However, when the plan was informed
by a purchaser that their contract might not be renewed because of inadequate
attention to cultural competence, Care 1st had to weigh the
price of improving its performance against the cost of losing business.
With the approval of a small start-up fund, the plan adopted the CLAS
standards, developed strategic plans and work plans with practical steps,
sought support from community-based organizations, worked to get plan
and affiliate managers on board, and surveyed the attitudes and beliefs
of providers about cultural competence.
One of the most difficult
challenges is the often limited degree of control a health plan has over
the activities of its contracted providers, even after trying to persuade
them to change their behavior and practice patterns. Dr. Gunawardane noted
that Care 1st’s provider survey revealed a significant lack
of knowledge about the benefits of culturally competent practice and frustration
with too many government or plan rules. Similarly, Mr. Quan reported that
even when interpretation services are paid for and easy to use, many providers
still don’t use them.
D.
Establishing standards and models for cultural competence training in
the health professions will improve the clinical encounter.
Patient-physician communication
serves many important functions that affect clinical outcomes. In addition
to obtaining clinical information, the purpose of the medical interview
is to build a relationship with the patient, to educate and counsel, and
to develop a partnership regarding treatment decisions. Increasingly,
researchers are trying to make the link between improved clinical encounters
and better clinical outcomes. As illustrated through studies presented
by Lisa Cooper, Debra Roter, and Susan Larson of Johns Hopkins University,
“providing communication skills and cultural competence training to physicians
could improve patient involvement in care, patient adherence, patient
satisfaction, and clinical outcomes.”
Numerous professional organizations,
including the American Academy of Family Physicians and the American College
of Obstetrics and Gynecology, have recognized the importance of cultural
competence by adopting policies or guidelines relating to the care of
culturally diverse populations. Recently, the Liaison Committee on Medical
Education, the national accrediting authority for medical education programs,
established cultural competence as a topic that all medical school programs
must cover in order to meet accreditation standards: “Faculty and students
must demonstrate an understanding of the manner in which people of diverse
cultures and belief systems perceive health and illness and respond to
various symptoms, diseases, and treatments.”
According to independent researcher
Jean Gilbert, “While there has been an increase in the number of residency
programs and medical and nursing school curricula that offer cultural
competence education, the objectives, content, duration, teaching methodologies,
and evaluation of such programs have varied significantly.” In order to
forge consensus across disciplines and different perspectives, the California
Endowment sponsored a project directed by Dr. Gilbert and Julia Puebla
Fortier, of Resources for Cross Cultural Health Care, that produced a
document on principles and recommended standards for cultural competence
education of health care professionals. This document is intended to provide
guidance in curriculum and program design.
Training Tools. Recognizing
the need to move cultural competency to the formal curriculum of medical
schools, the Association of American Medical Colleges (AAMC), supported
by a grant from the Commonwealth Fund, has convened experts to develop
a Tool for the Assessment of Cultural Competence Training (TACCT). Experts
in cultural competence and medical education identified key domains of
cultural competence education and for each of these components considered
the specific knowledge, skills, and attitudes required. They agreed that
cultural competence curricula must be integrated throughout the four years
of medical school. A primary concern for implementing curricula involves
“process” issues: building capacity, generating support from school leaders,
ensuring involvement by teachers and students, and providing faculty with
the necessary time to develop new programs and curricula. All of these
issues must be addressed first in order to implement an effective program.
However, cultural competence
curricula cannot use a “one-size fits all” approach, according to the
AAMC’s Deborah Danoff. Or put another way, she said, “If you’ve seen one
medical school, you’ve seen one medical school.” The aim of their project
is to offer suggestions and examples of what has worked in different settings,
and to generate awareness of those models so “they don’t have to reinvent
the wheel and create a whole new curriculum right from scratch.”
Model curricula and training
programs are also being developed at the continuing medical education
level. The Cultural Competency Curriculum Modules Project (CCCMs), undertaken
by the US DHHS Office of Minority Health and the American Institutes for
Research, seeks to develop a model that is anchored in uniform, nationally
recognized standards and can be directly applied to clinical practice.
These modules have specific learning objectives linked to the CLAS standards.
After completing each module, participants should be able to describe
or explain key components of culturally competent care, such as the triadic
interview process, relevant legislation and policies, and the benefits
of including minority community members in health partnerships. A pilot
test of the CCCMs revealed increased knowledge and understanding; development
of self-awareness about attitudes, beliefs, biases, and behaviors; and
enhancement of overall skills in culturally competent care.
Educators are experimenting
with different teaching tools and methods for more effectively presenting
cultural competence to medical students. In their intensive skills-building
workshop, Joe Betancourt, Tricia Tang, Melanie Tervalon, and Alex Green
(of the medical schools at Harvard, University of Michigan, University
of California San Francisco, and Cornell respectively) reviewed the use
of video trigger tapes, student- and faculty-generated sociocultural training
cases, interdisciplinary approaches (culture and bioethics), small group
case conferences, and OSCE stations. They challenged participants to brainstorm
tools and approaches that would work within the structure and curriculum
of their respective institutions. Multimedia teaching tools, such as video
vignettes, films, skits, computer modules, and Web sites have taken on
a significant role in cultural competence training and were featured throughout
the conference.
Attitude Adjusters.
Levels of awareness and interest in cultural competence vary greatly in
the physician community, which poses a challenge to developing effective
training programs. Medical educators Jeanette South-Paul from the University
of Pittsburgh and Robert Like from UMDNJ Robert Wood Johnson observed
that one of the ongoing challenges to providing training in cultural competence
is figuring out how to “generate interest, deal with resistance, and support
the desire to become more culturally competent in the practicing physician
community.” Likewise, it is critical to ensure that medical students understand
the importance of cultural competence and not simply view it as “fluff”
material while they are trying to master “core” requirements, such as
biochemistry.
Kaiser Permanente responded
to the “need to understand the cultural, ethnic, and racial issues that
affect our patients” by establishing its Institute for Culturally Competent
Care, mentioned earlier. Input from a cross section of physicians and
key stakeholders and organization-wide representation on the planning
committee were essential to creating the center. From a process standpoint,
Kaiser learned the importance of building trust, creating a safe learning
environment, and easing education into physicians’ practices. Kaiser found
it important to educate the physicians first and then focus on educating
other members of the provider team. To ensure staff members understand
the relevance of the program, modules are tailored to provide information
pertinent to the specific patient populations being served. An important
aspect of the Kaiser program is its underlying flexibility. Programs are
continually modified per participant evaluations and based upon current
issues and events occurring in the local community and the world. Kaiser
Permanente credits its Institute for Culturally Competent Care with improved
clinical outcomes, including decreased hospitalizations for sickle cell
and congestive heart failure patients. Kaiser has concluded that these
educational interventions have resulted in a “culture shift” in their
organization.
The Computerized Communication
Skills Program, from Johns Hopkins University School of Medicine, links
curriculum, self-assessment, feedback, and evaluation. This interactive
CD-ROM features an “expert interview” with an African-American simulated
patient to demonstrate patient-centered skills and culturally sensitive
communication approaches. The Roter Interaction Analysis System (RIAS)
allows the user to navigate the interview and review examples of specific
skills. The RIAS software can measure proficiency in various elements
of the communication process and provides a detailed, reliable window
into performance. It is currently being tested in a randomized controlled
trial aimed at improving patient adherence to recommended therapies for
hypertension.
E.
Health care organizations are using and improving upon a wide variety
of approaches to communicating with linguistically diverse populations.
Discussions on improving access
to health services for limited English speakers revealed a wide range
of experimentation and investigation into different methods of planning
for, delivering, and assessing the impact of language services. The field
of face-to-face interpretation focused its discussions on the challenges
of developing ethical codes of practice and certification programs. At
the same time, technological advances have extended the reach of interpretation
services to locales where face-to-face services are expensive or unavailable.
A consortium of organizations
in New York City has piloted a Remote Simultaneous Medical Interpreting
System, using interpreters who interpret through wireless headsets for
patients and doctors sitting face to face with each other. The Center
for Immigrant Health is conducting research on the effectiveness and efficiency
of different modes of interpreting, looking specifically at medical outcomes,
linguistic errors, and cost-effectiveness.
Strong Connections is using
videoconference technology to bring sign language interpreters based at
another locale “into the room” at health care settings where sign interpreters
aren’t available. They are now attempting a national launch of their services
after several years of trials and testing in New York State. A major challenge
lies in persuading hospitals to use such a service while maintaining the
infrastructure needed to operate such a program.
The roundtable discussion
“(Wo)man or Machine: Is there a best way to provide interpretation services?”
explored many of the issues that arise when deciding whether to use face-to-face
or remote interpretation services. Acknowledging that very little formal
research has been conducted in this area, some participants observed from
their experiences that on-site interpreters might be preferred in situations
where visual aids are being used, with groups of more than two or three
individuals, or with patients who have hearing difficulties or are disoriented.
Others noted that when high-quality equipment was available, telephonic
or video interpretation works well in urgent situations, for brief encounters,
or encounters where privacy is preferred. Some felt that mental health
encounters should always have on-site interpreters, while others disagreed.
Regardless of their opinion about ideal circumstances, most participants
agreed that in everyday practice, health care staff tend to gravitate
to the most convenient or most familiar option, and that having a variety
of choices is best.
Many providers are trying
to make better use of the skills of bilingual employees, while struggling
to compensate for the fact that language skills without interpreter training
can put patients at risk for miscommunication errors. Glenn Flores, of
the Medical College of Wisconsin, reported on a study to determine the
rate and consequences of errors in medical interpretation in pediatric
outpatient encounters, where both professional and ad hoc interpreters
were used. Sixty-three percent of all interpreting errors had clinical
consequences, and errors committed by ad hoc interpreters were more likely
to have such consequences than those committed by hospital interpreters.
Elizabeth Colón, of Cross-Cultural
Interpreting Services of Chicago, reported that even bilingual employees
hired to communicate with limited-English proficient patients are not
always adequately prepared to interpret. The results of a language assessment
exam administered to 114 bilingual employees found that very few were
actually competent enough to render a message from one language to another.
Findings included poor command of basic medical terminology, poor sentence
structure, violation of patient confidentiality, and lack of compliance
with ethical codes.
In conducting assessments
of bilingual staff among its provider networks, the L.A. Care Health Plan
found the need for an outside vendor to conduct language assessments to
ensure unbiased results, as many individuals contested the results of
some parts of the assessment. Like many other health care organizations,
L.A. Care also instituted an interpreter training course for bilingual
staff, which raised many interesting challenges. On several occasions,
supervisors of registered participants needed to be convinced of the importance
of the training program, and some individuals were pulled back from participating
initially due to job constraints. Filling in for staff on training was
an issue for not only small clinics but also large ones that rely on bilingual
staff to interpret.
Nevertheless, many large health
care providers are trying to make the best use of bilingual staff in complicated
delivery arrangements where interpreter use is still very low. Hao Duong,
of the California managed care program CalOptima, reported on an assessment
of the language capability of its network providers. The results of a
survey to record self-assessed language proficiency was compared with
the language needs of enrolled members, accounting for geographic factors.
The plan was able to assess whether language capacity of its affiliated
networks matched need. For the largest language groups, Spanish and Vietnamese,
members could access a provider speaking their languages within less than
a 10-mile radius. The greatest difficulties were found for small language
groups such as Hmong, Samoan, and Cambodian.
F.
Researchers and providers are experimenting with different ways to measure
cultural competence and its impact and can offer insight into successful
approaches and difficult challenges.
A major focus of the conference
was on assessment, measurement, and data collection, with nearly thirty
presenters touching on this area. Participants noted that funders and
consumers want more detailed information about the quality and impact
of cultural competence programs, yet the task remains difficult due to
the scarcity of appropriate tools and resources as well as reluctance
on the part of some providers and health care organizations to participate
in evaluation and data collection activities. The progress being made
in this area, however, was demonstrated by the creative and persistent
efforts described by researchers and managers from across the country.
In response to goals articulated
by the CLAS standards, researchers, government agencies, and health care
organizations are developing assessment processes to determine baseline
capacities as well as to audit performance. Presenters reported many challenges
to conducting cultural competence assessments. According to Karen Linkins
of the Lewin Group, who developed an organizational assessment profile
for the USDHHS Health Resources and Services Administration (HRSA), they
include:
- Finding the balance between
the fluid and dynamic nature of culture and cultural competence and
the concrete demands of measurement
- Managing the complexity
that stems from multiple levels of analysis
- Balancing short-term versus
longitudinal measurement
- Compensating for the frequent
lack of baseline data, and minimizing the burden of subsequent data
collection
- Accurately weighing the
impact of cultural competence interventions against other factors
- Impressing on organizations
the value of measurement, and securing the tools, resources and expertise
to conduct it
The HRSA profile is based
on seven domains of cultural competence that are broken down into specific
focus areas, with a standard set of indicators for each (structure, process,
output, and outcome).
The HRSA tool joins many other
organizational assessment tools designed by researchers for a variety
of health care organizations. Most of them are intended to provide a comprehensive
snapshot of organizational strengths and shortcomings. But health care
organizations themselves are constructing self-assessment processes and
experimenting with different methods of data collection in the context
of limited resources and other organizational priorities.
Commitment, Time, Resources.
Elisa Friedman, of the Cambridge Health Alliance, noted several factors
critical to the successful implementation of a cultural competence assessment.
Leaders must commit to the process and provide adequate time and resources
to see it through. Managerial and staff perception are pivotal—is the
assessment process seen as supportive to their work or potentially punitive?
Presenters described several
different methods of data collection used when conducting assessments.
Fred Kobylarz, of UMDNJ Robert Wood Johnson Medical School, outlined the
Multi-Method Assessment Process (MMAP). It is a short assessment of a
clinical practice that looks at quantitative and qualitative data on system
values, functions, practices, and day-to-day operations. Methods include
observation, in-depth interviews, chart audits, reviews of existing documents,
and the examination of relationships among staff and between staff and
patients. The Cambridge Alliance is also experimenting with using a “patient
walk-through” as a means of collecting data. The patient (or unidentified
staff person) walks through the entire patient process at a site, from
making an appointment to post-appointment interactions. The process and
perceptions of it are documented.
Several presenters noted that
a tension often arises between the goals of program evaluation and the
desire to produce outcomes data. Programs implementing cultural competence
interventions are often under pressure to demonstrate the impact of interventions
on different health measures when they are still struggling to understand
how best to run their programs and collect basic data on outputs.
Mariann O’Keefe, of Data Matters,
and Jeanne Nelson, of the Mayo Clinic, described some of the challenges
they had in evaluating their Pathfinder program, which helps immigrants
and refugees understand and navigate health and social services. Initially
they had big goals—did participation in the program affect ER use, compliance
with appointments, or satisfaction with physicians? However, barriers
to collecting needed data (such as the inability to get institutional
review board approval from three different organizations) made answering
these questions impossible. They found it difficult to overcome the conflict
between addressing research objectives that require long-term research
and collecting basic output information that would allow them to revise
and improve the program on an ongoing basis.
Both the Pathfinder program
and the Outreach and Interpretation Project of Illinois discovered the
need to be very explicit about program and evaluation goals. Wendy Siegel,
of Millenia Consulting, and Louise Cainkar, of the University of Illinois,
described using the SMART model goals (specific, measurable, attainable,
realistic, and time-limited) to guide their work with the Illinois project.
They also used a community participation model to assist with redefining
the evaluation goals.
Both programs noted the value
of both quantitative and anecdotal/story information in assessing the
impact of programs. They found it difficult to do outcome measures in
a short-term project where so much time is required simply to set up and
work out the bugs in an intervention. And they found real problems attempting
to collect and compare data across different types of organizations involved
in the same program, in the context of different missions and approaches.
Accurate race, ethnicity, and language data is essential in the pursuit
of improved health care quality. According to Arlene Bierman of the US
DHHS Agency for Healthcare Research and Quality, it can be used by providers
to inform program development, planning, and priority-setting; to target
quality improvement efforts; to understand performance differences within
an organization; to appropriately target CLAS services; and to evaluate
and monitor the effectiveness of interventions.
Data Collection Challenges.
Collecting data, from the clinical to the national level, remains one
of the most frequently mentioned challenges in the quest to provide quality
health care for diverse populations. At the organizational level, staff
may already feel overburdened by paperwork and may not have the time for
or see the benefit of participating in additional surveys, interviews,
or assessment processes. These issues are further complicated when researchers
are attempting to compare data or performance between different organizations,
each of which has its own priorities, systems, and procedures. Longitudinal
assessments are complicated by the need to translate and culturally adapt
instrumentation for multiple ethnic groups, and it is difficult as well
to maintain an adequate sample size when frequent relocation or switching
of health providers is common.
Producing usable data often
boils down to what the question is and how it is asked. California advocates
working with two state agencies to collect cultural competence services
data among health plans found that responses were often vague or unrevealing.
Plans participating in the survey complained that the questions were poorly
worded and developed without consulting them. The L.A. Care Health Plan
found that response rates to its multilingual survey of Medicaid enrollees
were improved by conducting the interviews in person or over the phone
with bilingual interviews from each of the target groups.
Even collecting the most basic
data about race, ethnicity, and language (r/e/l) is fraught with complications
related to perceptions, standardization, and comparability. On a broad
scale, Merry Tantaros, California Medical Review, Inc., described a combination
of data profiling techniques that would enable Medicare+Choice plans to
access data and generalize enough to plan with reasonable accuracy. However,
she noted that CMS Medicare enrollment data frequently underestimate some
populations, that the race and ethnic identification data are not ideal
due to forced choice options, and that “unknown” and “other” data categories
are not useful. Researchers Romina Kee of Cook County Hospital and Elizabeth
Jacobs of Rush Medical College described the development and use of a
culturally appropriate instrument for collecting data on race and ethnicity.
They were looking for feedback on how the terms race and ethnicity were
perceived, and what attitudes people had about questions related to collecting
race and ethnicity data. Through semi-structured interviews in focus groups,
they found that those in favor saw data collection as a tool for improving
care, and those opposed saw it is a tool for discrimination, and their
willingness to answer such questions depends on how they are asked.
Health care organizations,
researchers and quality agencies are still stymied by the different rules
and approaches to collecting data on race, ethnicity, and language. Paul
Schyve noted that the JCAHO has not included these demographic variables
as part of its performance measures because of concerns about some states
prohibiting the collection of this data. California voters will face a
ballot measure in 2004 that would prohibit collecting, inquiring, sorting,
or organizing data by race, ethnicity, color, or national origin. Ruth
Perot, of Summit Health Institute for Research and Education, and Mara
Youdelman, of the National Health Law Program, have made several recommendations
to the US DHHS about r/e/l data collection, including: improving the accuracy
of Medicare data, enforcing the collection of r/e/l data in the Medicaid
and SCHIP programs, and revising HIPAA rules.
G.
Outcomes research on cultural competence is challenging to design and
conduct, but researchers are slowly establishing a base that makes a link
between cultural competence and improved outcomes.
Those involved in culturally
competent health care often speak of good outcomes research as the holy
grail that will lead to broad acceptance by mainstream health care. However,
as conference attendees were reminded by Elizabeth McGlynn, of RAND Health,
“One of the really challenging things about measurement is you can provide
excellent, perfect care and get a bad outcome, and you can provide the
worst care in the world and get a good outcome.” There are few instances,
she said, of a one-to-one relationship between the intervention being
measured and the result.
Those caveats notwithstanding,
conference presenters discussed in both broad and specific terms the challenges
and successes associated with evaluating the impact of culturally competent
services on outcomes. Julia Puebla Fortier reported on the development
of a federally funded research agenda on CLAS interventions, especially
as they relate to health and health service delivery outcomes. A review
of the literature showed some glimmerings of links between CLAS and outcomes,
especially in the areas of racial and ethnic concordance between patients
and clinicians and in addressing language differences in clinical encounters.
However, there are significant challenges to conducting good research
in this area. Along with researcher Elizabeth Jacobs, Fortier noted that
researchers must grapple with these facts:
- quality data is scarce
and costly to collect
- a lack of standard definitions
complicates comparability between studies
- research instruments often
need to be reworked to address linguistic and cultural issues
- the impact of the intervention
is often difficult to isolate from other factors
- the ‘gold-standards’ of
research design are often inappropriate (randomized control trials,
for example)
- funding for such research
is scarce and difficult to obtain
- the costs and cost-benefits
of interventions are often difficult to measure
- the lack of a solid base
of existing research makes new studies difficult to evaluate and get
published
- journal editors and reviewers
are often unfamiliar with the field and the difficulties of designing
studies
Both Ms. Fortier and Dr. Jacobs
noted a number of strategies to address each of these challenges.
Other conference presenters
focused on specific projects and studies to assess the impact of cultural
competence interventions. The Agency for Healthcare Research and Quality
is funding EXCEED projects to examine why disparities occur and what strategies
might reduce them (while building capacity of institutions to conduct
health services research that relates to these populations).
Several presenters reported
on projects to evaluate the impact of cultural competence training for
health care providers. The Asian Health Coalition evaluated its program
through a combination of pre- and post-training tests of knowledge and
attitudes. Telephone surveys were conducted one to two months after training
to investigate for self-reported behavior changes. David Thom, of the
University of California at San Francisco, and Miguel Tirado, of California
State University, reported on their efforts to develop an outcomes assessment
of a tailored cultural competence training curriculum for practicing physicians.
While their study is ongoing and final results have not yet been achieved,
their success in the set-up phase may be encouraging to other researchers.
They report that it is possible to enroll a relatively large number of
LEP patients and their physicians in a longitudinal study and follow them
over time, and their measure of cultural competence appears to have good
psychometric properties.
Sharon Lee, of Portland State
University, reported that an analysis of data from a large managed care
organization revealed that the introduction of interpreter services could
be associated with the use of such services. The number of LEP patients,
especially those who had previously below average health visits, increased
after the introduction of interpreter services. There was a reduction
in the number of “other” complaint codes for LEP patients, suggesting
that when interpreters are available, more specific complaints are being
communicated to providers. LEP patients also increased their rate of advance
cancellations of appointments, although there are no significant reductions
in “no-shows,” and use of emergency and urgent care visits increased.
next
>
|
