CDC Logo Tuberculosis Information CD-ROM   Image of people
     
jump over main navigation bar to content area
Home
TB Guidelines
Surveillance Reports
Slide Sets
TB-Related MMWRs and Reports
Education/Training Materials
Newsletters
Ordering Information
Help

 

U.S. Department of Health and Human Services

  

This is an archived document. The links are no longer being updated.

TB Notes 1, 2002

Updates from the Research And Evaluation Branch

Evaluating Contact Investigation Programs

The contact investigation (CI) is a key strategy in the elimination of TB, and TB programs have established a variety of structures and practices to use in conducting these investigations. Recent studies conducted by DTBE staff and others have shown that TB control programs vary widely in their success in achieving outcomes related to identifying, screening, and appropriately treating contacts. These findings demonstrate the need for some programs to revise their contact investigation procedures. But what changes should be made? What aspects of the CI program work well as is and what areas need enhancement?

To answer these questions, local TB programs need a way to evaluate their current program processes and practices to appropriately plan and implement improvements and recommendations. CDC has developed a framework for public health program evaluation.1 This well-researched framework provides an adaptable six-step process for public health programs to use in evaluating their programs and operations. However, for those unfamiliar with evaluation concepts, understanding how to apply the framework and identifying appropriate resources can be difficult and time-consuming.

To help local TB programs build capacity for the evaluation of contact investigation processes, the Prevention Effectiveness Section (PES) of DTBE's Research and Evaluation Branch (REB) initiated a project that involves partnering with two health departments in the use of the CDC evaluation framework and applying the evaluation concepts to their programs. The ultimate goal of the project is to develop a translated version of the framework specific for TB-program managers and staff. Such a translated guide of the evaluation framework will enable even those new to program evaluation to easily identify and apply the six-step process to a self-evaluation of their contact investigation practices.

Over the past year, the two grantees, the Massachusetts and New Jersey health departments, have been designing and conducting their evaluations. In Massachusetts, a partnership between state and local TB staff to facilitate CIs has been in place for 5 years. Case managers at local health departments coordinate with state-level case managers to ensure that the CI protocol is implemented. The protocol includes use of the concentric circle approach to identify contacts, time lines for accomplishing tasks, and systems for ensuring proper exchange of information among state, local, and private providers. The state aggregates and oversees CI data to ensure nothing falls through gaps. State staff led the self-evaluation effort in five local TB programs.

In Newark, the New Jersey health department selected to self-evaluate the implementation and effectiveness of its specially developed protocol for conducting expanded CIs in congregate facilities. The protocol aims to ensure that all potentially exposed persons are included in the investigation, while not depleting resources or unnecessarily testing the unexposed. This protocol involves collaboration between the investigator and supervisor to determine if such a CI is necessary, a systematic environment assessment to determine likelihood of transmission, and a communication / education plan to allay fears and promote effective testing and treatment. The evaluation process will also document details of the protocol to facilitate translation to other programs.

While each program may report findings from their evaluations at a later time, the evaluators learned much about the evaluation process and how it can work for contact investigation program activities. The lessons they have learned may help guide other TB programs in understanding how evaluation can be applied to their contact investigation activities to provide them with useful, practical information for program enhancement. Lessons learned from each of the six steps in the framework are presented below.

Step 1: When using the CDC framework, the first step in the process is to identify "stakeholders." These are the people who are interested in contact investigations and will ultimately be affected by the evaluation. By identifying the stakeholders, and engaging key stakeholders throughout the process, the evaluators ensure that the evaluation is relevant and provides useful information. There are three main categories of stakeholders for contact investigations: 1) the state and local health department staff who conduct contact investigations and their managers and supervisors; 2) the persons receiving services, i.e., TB patients or suspects, contacts of cases, TB care providers and other health service providers, managers or employees at schools, jails, congregate facilities, or private businesses, and the general public; and 3) those who will directly use evaluation findings to enhance the programs and improve services.

Step 2: Once the key players have been identified, the next step in the process is to clearly and thoroughly describe the contact investigation system and its relationship to the general TB control program. This description ensures that all participants share the same understanding about what contact investigations are, how they are conducted, why they are needed, what program resources are used in conducting these investigations, and what effects are anticipated for each activity. One key feature of CDC's framework is developing a logic model that graphically describes how the different aspects of contact investigation link together to achieve the expected results. The graphic serves as a communication tool as well as explicitly showing how activities logically result in outcomes.

Step 3: From the description and logic model, the evaluators should then be able to focus evaluation questions and develop their data collection strategies. The evaluation question selected in Massachusetts was, "How well does our new case or suspect assessment tool work in five participating local TB control programs?" The evaluators decided to conduct a process evaluation to assess how well the new tool is being implemented by the five programs and an outcome evaluation comparing patient-level outcomes collected before the new tool was implemented and after its adoption. In New Jersey, the evaluation question was, "How useful and effective is our protocol for expanding contact investigations in congregate settings?" The evaluation methods centered on comparing case studies of investigations in which the protocol was fully implemented with case studies of investigations in which the protocol was not implemented. Outcomes, including those related to resource use, were also compared.

Step 4: Identifying indicators and selecting sources for data should be a simple process, given the work done in the early steps. The evaluators selected similar indicators, including process indicators: program organization and infrastructure, staff training, adherence to protocol steps, use of forms or records, staff perceptions, intensity of resource use, and outcome indicators: number of contacts identified, percentage of contacts fully tested, percentage of contacts testing positive for disease or LTBI, percentage of contacts started on treatment, and percentage of contacts completing treatment. The data sources that were selected matched each indicator, and included policies, practice instructions, training agendas and records, case files or records of contact investigations, completed forms or other written documentation, interviews with investigators and supervisors, and aggregate reports or forms. All data sources were assessed to ensure that they were reliable and considered valid by the stakeholders before data collection began.

Step 5: Analyzing data and determining findings also flow from the previous steps. Content analysis, domain analysis, and other methods were used to analyze qualitative data and statistical methods were applied to the quantitative data. To ensure that the evaluation findings could be easily understood by all stakeholders, simple data analysis methods were selected. With data in hand, the evaluators once again went back to the logic model to compare the data collected on implementation and results against original intentions and levels of success defined previously. These facts were then discussed and interpreted by the stakeholders to provide meaning and context. Findings were developed that identified what objectives and goals were met or unmet and why.

Step 6: The most important step is an on-going process to ensure that the findings from the evaluation are used. Currently, the findings are being used to

  • assess contact investigation process and practice as it is implemented,
  • target areas for enhancement or improvement,
  • develop standardized tools for data collection during interviews and field visits,
  • develop strategies to make necessary changes to operations,
  • prioritize program activities and resources,
  • identify effective policies, procedures, or practices for replication,
  • organize key information for training staff and informing those outside the program,
  • garner political support by demonstrating effectiveness of contact investigations,
  • understand the implications of policy and guidelines on the investigation process, and
  • identify areas for future research and evaluation.

The last use completes the circular nature of the framework. As information is obtained, more information is needed and more questions emerge. The process is repeated and will become routine for the TB programs. A learning cycle is established as programs strive for continuous quality improvement in their contact investigations and in other aspects of their operations. For more information on the Framework for Program Evaluation in Public Health, please see http://www.cdc.gov/eval/framework.htm or contact Maureen Wilce at 404-639-8123.

Submitted by Maureen Wilce, MS
Division of TB Elimination

Reference
1. CDC. Framework for program evaluation in public health. MMWR 1999;48(RR-11).

 


Released October 2008
Centers for Disease Control and Prevention
National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention
Division of Tuberculosis Elimination - http://www.cdc.gov/tb

Please send comments/suggestions/requests to: hsttbwebteam@cdc.gov, or to
CDC/Division of Tuberculosis Elimination
Communications, Education, and Behavioral Studies Branch
1600 Clifton Rd., NE - Mailstop E-10, Atlanta, GA 30333