|
Newsletters
> TB Notes Newsletter > TB
Notes 2, 2004 > International Updates
This is an archived document. The links are no longer being updated.
TB Notes 2, 2004
INTERNATIONAL UPDATES
Evaluation of the Brazilian National TB Surveillance
System
Most epidemiologists, public health policy makers, TB program managers,
and health experts would agree that TB surveillance is a crucial
tool in their decision making. Where disagreement may arise among
the various stakeholders is in their assessment of the system’s
reliability, completeness of coverage, and utility, among other
attributes. For example, in Brazil, the national TB surveillance
system (SINAN-TB: Sistema de Informação de Agravos de Notificação-Tuberculose)
reported to the World Health Organization (WHO) approximately
74,500 new TB cases in 2001, whereas WHO’s own estimate of TB cases
in Brazil for that year was approximately 110,500 — a substantial
difference (WHO estimates total TB cases based on TB incidence reappraisals
in reporting countries).1,2 In recognition of the importance
of accurate surveillance data, the Brazilian Ministry of Health
invited the International Research and Programs Branch (IRPB) of
DBTE to participate in an evaluation of SINAN-TB.
In order to accomplish the evaluation of the Brazilian TB surveillance
system information was gathered from six different sources: (1)
SINAN-TB; (2) Livro Preto (the local TB registries kept at
each treatment facility; (3) patient records; (4) Sistema de
Informação de Mortalidade (SIM), a national mortality reporting
system; (5) interviews with treatment center staff; and (6) interviews
with municipal, regional, state, and federal TB program administrators.
A comparison of specific data elements between SINAN-TB and
Livro Preto (and patient records), and between SINAN-TB and
SIM was designed to assess the more quantitative aspects of SINAN-TB
including data quality, completeness of coverage, representativeness,
and timeliness. The qualitative assessment of the system relied
on personnel interviews to determine the system’s simplicity, flexibility,
acceptability, stability and utility.
Relatively new to the world of TB surveillance is the capture-recapture
methodology,3 used in this evaluation to estimate SINAN-TB
completeness of coverage. This strategy, first applied in wildlife
management, estimates the number of uncounted TB patients by matching
the number of patients simultaneously recorded by two or more surveillance
systems that capture parallel information (i.e., number of TB deaths
in SIM matched against the number of TB patients in SINAN whose
outcome is death). A two-source model can estimate a system’s completeness
of coverage fairly accurately if the underlying assumptions of the
capture-recapture method are not violated: (1) cases must have the
same probability of appearing in each source, (2) the probability
of a case appearing in any source is independent of the other sources,
and (3) the population under study is closed.3
Data on approximately 1000 TB patients listed in the year 2000
in the Livro Preto, encompassing approximately 15 treatment
centers in 4 Brazilian cities (Rio de Janeiro, Maceio, Porto Alegre,
and Recife) were matched to those of patients from the same geographical
areas and treatment time period as recorded in SINAN-TB. Approximately
100 total TB staff members from these treatment centers and from
TB programs at various governmental levels were interviewed. In
addition, records of patients who died in 2000 or 2001 with a TB
diagnosis (by ICD-9 coding of the primary-, secondary-, tertiary-,
and quaternary-associated diagnoses) as recorded in SIM were matched
with those of patients recorded in SINAN-TB for the year 2000 from
the four cities surveyed--over 15,000 names.
The overall evaluation found SINAN data quality to be good with
one exception: follow-up patient information. For example, 80% of
follow-up sputum smears and 64% of patient outcomes were missing
from SINAN-TB. Further, almost one fourth of staff members interviewed
did not feel they were an important part of the system, suggesting,
along with responses to other questions, that SINAN-TB acceptability
was low. Timeliness of the system was also suboptimal, mainly because
the end users received epidemiologic data feed-back unevenly and
infrequently. Consequently the utility of SINAN-TB was felt to be
limited.
Conflicting results were obtained for SINAN-TB completeness of
coverage. When compared to Livro Preto, the system’s
coverage was estimated to be about 71%, a figure much closer to
WHO estimates of TB case detection in Brazil in 2000. When compared
to SIM, however, the estimate of the system’s coverage fell to 24%.
One possible explanation for this latter difference is that deaths
in SIM may have occurred or been reported in jurisdictions other
than the municipality where patients were receiving TB treatment,
or vice versa, thus violating the “closed population” assumption
of capture-recapture methodology. This situation may falsely lower
the estimated completeness of coverage of SINAN-TB (and of SIM as
well). In order to test this hypothesis, the matching of TB patients
in SIM and SINAN-TB is being expanded to include the four states
in which the four surveyed cities are located. Tuberculosis patients
in the SIM and SINAN-TB databases for each state, in its entirety,
will be matched. The goal of this approach is to minimize
geographical reporting artifacts. In addition, log linear modeling
will be applied to the three databases used for this evaluation
in order to arrive at a more accurate completeness of coverage figure
for SINAN-TB.
In conclusion, this evaluation found SINAN-TB to be a useful TB
surveillance system; however, its utility is reduced by (1) low
capture of follow-up TB patient information, thus hindering program
evaluation at all levels; (2) deficient data feedback, limiting
program improvement efforts at local and regional levels; and (3)
relatively low completeness of coverage (range 24%-71%), possibly
leading to underreporting of TB cases.
The following recommendations were made to the Brazilian Ministry
of Health, Secretariat for Health Policy. The Brazilian National
TB Program should consider the following: (1) establishing a federal
SINAN-TB advisory council to assess, develop, and oversee system
changes and serve as a representative body to address end-user interests
and needs; (2) establishing accountable executive positions with
budgetary and oversight authority as well as with responsibility
at the state and municipal levels for the purposes of quality assurance;
and (3) ensuring access to and feedback of epidemiologic TB information
for all programmatic administrative levels, as well as encouraging
the use this information through currently available resources (i.e.,
technology, training, technical assistance).
As of this writing, a number of the recommendations made by the
International Research and Programs Branch (IRPB) of DTBE for the
improvement of SINAN-TB have been implemented, as modified by prevailing
circumstances. The impact of the applied recommendations has not
yet been assessed; however, the evaluation has been instrumental
in raising the completeness of patient follow-up information in
SINAN-TB from approximately 30% to 50%. In addition, with a credible
assessment of Brazil’s TB surveillance system available to them,
public health officials now take into consideration SINAN-TB underreporting
of TB cases in programmatic and policy decisions while the system
is improved.
—Reported by Abe Miranda, MD
Div of TB Elimination
References
1. WHO Report 2003. Global Tuberculosis Control. Surveillance,
Planning, and Financing. Geneva.
2. Corbett EL et al. The growing burden of tuberculosis: global
trends and interactions with the HIV epidemic. Arch Int Med 2003;
163: 1009-21.
3. Hook EB, Regal RR. Capture-recapture methods in epidemiology:
methods and limitations. Epidemiol Rev 1995; 17: 243-264.
|
