Results from main study: agreement between swab and tissue sampling
We compared the reported presence of isolates likely to be pathogens using both techniques. Swab results of 395 patients with DFUs and suspected infection yielded no bacterial isolates at all in 20% of samples. Despite these wounds being clinically assessed as infected, 29.9% of swab reports noted no potential pathogens. For tissue sampling the rates were lower: 10% had no bacterial isolates reported and 13.9% of reports cited no pathogens (only contaminating/colonising flora). Given that wounds are not sterile, the high ‘no isolates’ rate suggests that sample collection and/or transport were less than optimal. In addition, the ‘no pathogens’ rate might indicate that clinicians have a low threshold for suspecting infection in DFUs given the clinical consequences of failing to detect and treat foot infection promptly. This may result in some wounds with suspected infection being merely colonised (i.e. yielding no pathogens). However, the high ‘no isolates’ rate suggests that another explanation is that collection and transport of samples were suboptimal and, hence, pathogens in the wound were either not collected or did not survive transport. Tissue samples were better than wound swabs in collecting and transporting to the laboratory live bacteria from DFUs, with half the chance of reporting no isolates compared with wound swabbing (10% vs. 20%). To improve the collection of microbiology samples from wounds with suspected infection, it may be necessary to improve collection and transport such that fastidious and anaerobic organisms are preserved for subsequent culture and analysis. We had specified that clinicians used the Levine et al.48 technique for swab sample collection, noting that this was identified as superior to two other swab techniques by Gardner et al.;113 however, the pragmatic nature of the study meant that we did not restrict sampling to a subset of clinical staff whose technique had been assessed as producing the sufficient wound surface pressure to express wound fluid from the ulcer bed, nor did our protocol require validation regarding the level of cleansing and debridement performed prior to sampling.
Overall, the most frequently reported pathogens were S. aureus (35.7%), Streptococcus (16.7%), Enterococcus (14.9%), CNS (12.2%), Corynebacterium (9.4%), Pseudomonas (8.6%) and MRSA (8.1%). Considering each of the potential pathogens separately allowed us to determine patterns of agreement across different pathogens. This was important, as combining pathogens would mask potentially important differences. It is possible to have disagreement in identification of pathogens when both methods report the same prevalence: swab reports 10% and tissue sample reports 10%, but there is symmetrical disagreement whereby some tissue results are positive and swab reports are negative for a particular pathogen and vice versa. We therefore reported both prevalence of identification of pathogens and agreement rates by pathogen.
Reported prevalence of pathogens was identical for S. aureus and Pseudomonas and very similar (1% different) for MRSA, whereas for all other groups of potential pathogens the prevalence was statistically significantly higher from tissue samples than from swab samples. In 80% of patients there was agreement in the parameter ‘at least one pathogen identified’, meaning that the disagreement rate was 20% for ‘≥ 1 pathogen’. This does not convey the extent of variation between pathogens, however, and the disagreement was low for MRSA (1.5%), Pseudomonas (4.1%) and S. aureus (8.1%). For anaerobes, by way of contrast, disagreement was 16.5%. Overall, therefore, the agreement between wound swabbing and tissue sampling is high for three of the most prevalent pathogens, S. aureus, MRSA and Pseudomonas. For other pathogens, the swab reported organisms significantly less often than tissue samples. This might indicate that wound swabbing and tissue sampling have comparable yields for these three organisms. When sampling is designed to provide information on the wound biome with no prior expectation of the infecting organisms, however, then using a tissue sample will more often report the presence of organisms than a swab sample. It is not possible from these data to determine whether the higher yield is associated with higher survivability of organisms or better collection.
As well as measuring disagreement (proportion of samples in which the tissue and swab samples did not agree regarding the presence/absence of a specified organism), the commissioning brief determined that we should also report the agreement using the κ-statistic. There is discussion among statisticians about the use of Cohen’s kappa in measuring agreement (which will not be rehearsed here), but, for completeness, we reported percentage disagreement as well as kappa and PABAK. Although the kappa and PABAK for these outcomes were at a level usually associated with ‘fair’ to ‘high’ agreement, researchers have argued that they are imperfect summary statistics for determining the ability to replace one assessment tool with another, as the consequences of disagreement (such as not identifying an aggressive pathogen with one method), and hence the clinical implications of delayed diagnosis and treatment need to be considered. Guidelines for reporting agreement studies note that some authors require high agreement levels when tools are used to make individual and important clinical decisions, for example, suggesting that a tool should have reliability levels of at least 0.9 when being used to make an individual assessment of pressure ulcer risk.114
The number of different bacterial pathogens reported differed between tissue sampling and wound swabbing. Overall, tissue sampling identified more species than wound swabbing, with a median of 1.5 pathogens identified per tissue sample and 1 per wound swab. Given that the wound swab collected wound tissue from a greater area than the tissue sample, it might be expected that wound swabs would report a higher number of potential pathogens; therefore, this finding might be attributable to poorer pathogen survival in swab samples than tissue samples or insufficient pressure used during swab sample collection to allow deeper-wound-fluid-containing pathogens to be expressed.
We also reported the proportions of patients in whom the same bacteria were identified in each report, where additional bacteria were identified in the tissue sample over the swab sample (but all the swab bacteria were also identified), where additional bacteria were identified in the swab over the tissue sample (but all the tissue sample bacteria were reported in the swab sample) and where there was a difference in pathogens reported. Swab and tissue results reported the same pathogens in 42% of patients; a swab reported additional pathogens to those in the tissue sample in 8.1% of patients; tissue samples reported additional pathogens to those in the swab in 36.7% of patients; and, the tissue sample and swab specimens reported different pathogens, with or without overlap, in 13.2% of patients. These differences were unrelated to any clinical characteristics (type of ulcer, etc.) except potentially the wound duration: for wounds open for > 56 days, the odds of their tissue sample reporting additional pathogens (compared with swab sample) was lower than for young wounds, although this relationship was not present when duration was analysed as a continuous variable. This may be due to the change in biome as wounds age and become increasingly polymicrobial, for example, or may be a chance finding; therefore, it is worthy of further study in future research. The chance of identifying additional isolates (likely pathogens) with a tissue sample is therefore potentially clinically significant; however, if these isolates are present in low numbers then it may represent an increase in overall information about the wound biome with no impact on the clinical assessment regarding the likely cause of infection and the appropriate antibiotic regimen. For this reason, we added the substudy whereby a clinical panel review was undertaken to determine whether or not the (usually) higher amount of information from tissue samples translated into different analyses by clinicians reviewing the microbiology reports while blind to source (swab or tissue). Overall, the microbial diversity (number of species) was higher when wounds were sampled using tissue rather than swab.
The microbial load, assessed semiquantitatively, was compared between tissue and swab. Agreement between the two techniques for MRSA (overall κ = 0.73), Pseudomonas (overall κ = 0.58), Streptococcus (overall κ = 0.65) and S. aureus (overall κ = 0.64) was in the ‘moderate’ to ‘substantial’ range, but was lower for anaerobes (overall κ = 0.32). We were unable to conclude overall if the microbial load (as summarised in +/++/+++) was consistently higher for either sampling technique.
The presence of antimicrobial resistance among likely pathogens was low (33 patients overall), and there was disagreement about the presence of resistance in only 6 patients and, hence, detailed analysis was not undertaken.
The rates of sampling-related AEs were higher for tissue sampling than for wound swabbing, both in terms of the pain reported by patients and sampling-related bleeding (6% in cases related to tissue sampling alone, 0.8% cases related to swab sampling alone, and related to sampling of any form in 0.8% of other cases). This equates to a number needed to harm of 20: for every 20 patients undergoing tissue sampling rather than wound swabbing, there would be one additional case of clinically concerning bleeding. We did not have any reports of further sequelae of sampling; we did not collect data at the next visit, for example, but there were no reports of RUSAEs throughout the study.
One-quarter of patients reported pain before sampling, and we recorded sampling-associated pain after swabbing and again after tissue sampling (swabbing always preceded tissue sample collection). A total of 5 patients (1.3%) reported that pain increased (from baseline) on swabbing, and 37 patients (9.3%) reported that it increased after tissue sampling. This equates to a number needed to harm of 13, although overall (in 89.5% of patients) ulcer pain remained the same after sampling. Despite the vast majority of these patients having neuropathy, we cannot assume that these wounds are painless, that patient comfort is not affected by sampling technique or that, overall, swab sampling is associated with a lower risk of sampling pain.
We sought to report the costs of sampling using each of the two techniques but found limited information on the real costs of techniques, with some centres considering such information commercially sensitive and hence being unwilling to provide it. The cost of sampling can include a number of elements, not all of which will have been included in the costs quoted by those centres providing information. For example, the clinical time involved in wound bed preparation (cleansing and debridement) may not be captured. The relative differences between swab costs and tissue sample costs was 4% in one site (£15.55 vs. £16.53) and 49% in another (£3.91/£5.85). These costs do not include sampling equipment, transport or clinician time. Further work is required to understand the economics of wound swabbing versus tissue sampling.
It appears, therefore, that if one wishes to identify the presence of S. aureus, MRSA or Pseudomonas in a clinically infected wound (diagnosed with regard to clinical appearance, signs and symptoms) then tissue sampling is, broadly speaking, comparable to swab sampling for these clinically important pathogens both in terms of identifying the presence of the pathogen and in determining the bacterial load in a semiquantitative manner. By taking a tissue sample instead of a swab, however, it is likely that a larger number of pathogens will be identified. It is not clear whether this difference in yield is due to poorer collection, survival in transport or lower reporting of the bacteria collected using wound swabs. Interestingly, the small number of swabs analysed in the substudy of molecular microbiology (PCR) techniques had a higher number of pathogens identified than either the cultured swab results, or the tissue samples (cultured or PCR), and as PCR is able to identify and multiply dead bacteria, this suggests that swabs collect more bacteria than tissue but that more of the species die in transport.
In a small proportion of cases, swab samples identify pathogens in addition to those identified from tissue sampling, which is likely to be attributable to the heterogeneity in location of pathogens across the wound surface and the fact that a wound swab gathers material from a larger area. This means that choosing one sample technique leads to a trade-off in information for 58% of patients. One approach might be to consider the two techniques as complementary, but performing both procedures would lead to increased costs and patient pain/bleeding complications. Choosing to take a tissue sample over a swab also leads to a small increase in the number of patients experiencing pain and bleeding of clinical concern.
Results from the substudies
Clinical review panel
A clinical panel review substudy was undertaken to identify if any differences identified in the main study in sampling yield were potentially clinically relevant. If one sampling technique provided more information but did not result in any therapeutic changes (compared with a reference method of sampling), then the additional information may not be clinically important. As all patients in this study had both samples taken and clinicians had access to both reports, we were not able to identify whether or not any clinical changes initiated were attributable to either swab or tissue sampling microbiology reports. We presented paired and blinded microbiology reports to clinicians to ask them if patients’ antimicrobial regimes ‘covered’ the pathogens reported, and also if a change in therapy was warranted, based solely on the microbiology results. The decision to change antimicrobial therapy would not, in practice, be taken on the basis of a laboratory test result alone, with clinical assessment of the patients and their wound playing a major part in clinical decision-making; however, as clinicians were presented with imperfect information (i.e. with no clinical data) for both swabs and tissue sample microbiology reports, we propose that this approach allows some analysis of the amount of clinically relevant information provided by the laboratory reports.
We assessed the inter- and intrarater reliability of these assessments and determined that the intrarater reliability for 7 of 13 assessors was ‘moderate to substantial’70 for a change in therapy (κ = 0.59 for tissue and κ = 0.77 for swab results), with inter-rater reliability lower at κ = 0.35 for swab and κ = 0.66 for tissue samples, meaning that there was at least ‘fair’70 agreement between the clinical reviewers for swab samples but substantial agreement for tissue samples. Having established adequate agreement between clinicians (and where agreement was low, this was overcome as each clinician reviewed both patients swab and tissue sample vignettes), we determined the proportion of patients for whom the prescribed antimicrobial regimen was deemed to be insufficient. For 1 in 5 patients (19.8%), a tissue sample report was assessed by clinicians as indicating that the empirical antibiotic regimen did not cover the pathogens reported (whereas the swab sample results did). Conversely, for 1 in 12 (8.5%) patients, the swab sample results suggested that the empirical antibiotic regimen did not cover the pathogens reported, whereas the tissue sample results did. A change of strategy from swab sampling to tissue sampling would result in 11.5% (1 in 9) of patients additionally being deemed to ‘not be covered’ by their current regimen. Any sampling regimen using a single method of ulcer sampling would be associated with potentially missing pathogens and the resultant assessment would be influenced by the source of the laboratory report.
The results for ‘change in therapy required’ were very similar, given the relationship between these two outcomes. For 1 in 12 (8.9%) patients, the tissue sample results suggested no change, whereas the swab sample results did suggest that a change was required. For 1 in 6 (17.8%) patients, the swab sample results suggested no change, whereas the tissue sample results did suggest that a change/initiation was required. A change of strategy from swab sampling to tissue sampling would result in 8.9% (1 in 11) of patients potentially having a change in antimicrobial regimen.
Although it is tempting to conclude that clinicians should use the sampling technique that result in the highest number of pathogens and the highest rate of potential therapeutic changes, the data from this study cannot determine whether or not tissue sampling is necessarily associated with better clinical outcomes. In managing infected DFUs, there may be a trade-off between tissue sampling and swabbing. Swabbing may be less expensive and require less training to perform, fewer patients may experience sampling pain or bleeding, and the reduced report of pathogens may be sufficient to guide therapy changes at clinical reassessment. Indeed, the survey of practice at sites indicated that microbiologists, working with clinicians in the clinics/wards, sometimes used their expertise to filter the content of the laboratory reports so that they included the material most relevant to the clinical condition (and omitting some low number isolates likely to be non-pathogenic) (see Appendix 3). Neither swab result nor tissue sample microbiology results produced by plating and culture arrive in the clinic in sufficient time to guide initiation of antimicrobials for the vast majority of patients and, hence, currently the role of wound sampling is to guide the tailoring of therapy at day 3 and beyond, when a clinical assessment will also be undertaken. Although plating and culture are the mainstay of microbiological analysis in wounds, there are a number of potential sampling scenarios, as outlined below:
Sampling to be undertaken upon first clinical assessment of infection using a swab, and any subsequent sampling to be undertaken by tissue sample. This would be, for example, should the clinical assessment or microbiology report determine that the empirical antimicrobial regimen is inadequate. This would expose only patients for whom additional information was needed to the potential harms of tissue sampling.
Initial sampling with both swab and tissue sample performed to capture the maximum amount of wound microbiology information. This would expose all patients to the harms of tissue sampling.
Sampling undertaken at initial assessment by taking a piece of tissue, knowing that this gives a better yield than swabbing, after determining that the risk of pain and bleeding is warranted. Some organisms are likely to be missed by tissue sampling alone, however.
Sampling undertaken at initial assessment by swabbing using Levine
et al.’s technique,
48 as happens in some sites at present. This gives a lower yield than tissue sampling but reduces the risk of pain and bleeding. A large proportion of potential pathogens are likely to be missed by swabbing alone, however.
For other clinical scenarios (e.g. not DFUs) in which wound samples are taken with the goal of identifying the infecting organisms prior to antibiotic prescription, the trade-offs may be different.
Comparing culture and polymerase chain reaction techniques
For 14 patients, both swab and tissue samples were sent for analysis of bacterial RNA (dead and alive) and the results were compared with culture microbiology results. There was agreement between PCR and culture for one in three patients, but in half the patients, PCR analysis reported additional pathogens. PCR techniques more frequently reported the presence of a higher number of pathogens (in terms of bacterial diversity); for example, for the two swabs taken from each wound, the PCR analysed swab reported a median of 2.5 pathogens and the culture results a median of 1 pathogen. The difference was smaller for the tissue results, with PCR reporting a median of 1.5 pathogens and culture results a median of 1 pathogen.
In contrast to the results found using culture techniques, swab samples reported more pathogens than tissue samples (when both analysed by PCR). For the 12 samples with complete data, the culture results were that the swab and tissue agreed in 50% of patients and that tissue samples reported additional pathogens (i.e. over swab in 33%, with disagreement in 16.7%). Using PCR, however, the agreement rate was lower at 25%, and swabs identified more pathogens than tissue samples in 33% of patients, tissue identified additional pathogens in 16.7% of patients and there was disagreement in 25%. This indicates that both the choice of sampling technique and the method of analysis affects the yield from swabbing versus tissue sampling. The lower yield from swab samples and culture may be related to low viability of the bacteria sampled, secondary to sampling technique, transport media or delay in analysis owing to timing of collection from clinic/ward or at the laboratory. Any change in sampling technique (e.g. from swab to tissue) must be considered with reference to the sample analysis technique as the roll-out of PCR facilities in more hospitals may mean that wound samples can be analysed more quickly and that the differential yield identified using culture techniques is minimised when using PCR. This would influence the trade-off between swabbing and tissue sampling.
Prognosis of foot ulcer infection
We determined, at around 12 months, if patients had experienced ulcer healing, a revascularisation procedure, amputation, reulceration or had died. As there are no validated tools available to make a clinical diagnosis of resolution of infection, we did not collect data on the time to resolution. Over the 12-month period after ulcer infection, some patients experienced multiple events and, therefore, we used competing risks analyses to appropriately consider the risks of events. The relationship between these outcomes and baseline characteristics, including pathogens identified, were assessed.
The ulcer healing rate at around 12 months was 45.5%, indicating the poor prognosis associated with infected foot ulcers. Of those unhealed ulcers, 20% were not healed at 12 months despite an amputation, 57% of patients had an unhealed ulcer, and the remaining 22.7% of patients had died. Accounting for the competing risks of death or amputation, the estimated cumulative incidence of healing at 12 months post sampling was 44.5% (95% CI 38.9% to 50.1%).
With a univariate analysis we identified an association between reduced risk of healing and the following factors: ischaemic (vs. neuropathic) aetiology, increasing Wagner grade, increasing perfusion grade, increased ulcer depth/tissue loss, infection at grade 3 or above (extensive erythema or systemic inflammatory response), ulcer duration ≥ 56 days (vs. < 56 days), presence of an antimicrobial dressing and MRSA.
Factors associated with increased risk of healing included having a single foot ulcer, the presence of CNS and age (> 63 years vs. ≤ 63 years).
Following multivariable analyses, the remaining prognostic factors for healing were as follows (direction of association not specified unless not identified in previous studies): perfusion, wound duration, presence of other ulcers (a single ulcer more likely to heal), patient age, and the presence of CNS (ulcer more likely to heal in presence of this organism, probably because presence of CNS was inversely related to the presence of MRSA). The associations with ulcer duration and perfusion have been identified previously; we believe the higher risk of healing with increasing age to be a spurious association and that antimicrobial dressing may not be independent of the ulcer status, rather than it having a direct effect of delaying healing.
Strengths of this study
This is the largest comparison of the two main methods of sampling and the first study to report detailed data on paired ulcers for each pathogen and to examine the relationship between baseline characteristics and agreement using multivariable modelling. A strength of the study is its external validity – there were few exclusion criteria and patients were recruited in normal practice settings (specialist clinics and hospital wards) resulting in a generalisable study population. In addition, samples were taken by members of the attending clinical teams and processed by local laboratories, providing a ‘real world’ comparison of the two techniques.
All centres received training updates on swab and tissue sampling to minimise sample differences; the overall agreement between the isolates reported from the two types of specimens was relatively high but overall tissue sampling (and subsequent culture) reported more pathogens than swab sampling with culture.
Previous reports comparing swab to tissue specimens have been small, single-centre, studies but generally had findings similar to ours. Bill et al.60 compared culture results for a variety of chronic wounds from a swab with a punch biopsy (the reference standard) and found the sensitivity to be 79%. Their study of only 38 patients with several sources of bias regarded quantitative culture of the biopsy as a gold-standard reference test for infection, rather than the alternative, namely clinical signs and symptoms. In a retrospective study of 89 concomitantly obtained pairs of samples from 54 patients with DFUs (87% clinically infected), Mutluoglu et al.62 found that culture results of superficial swabs did not correlate well with those obtained from deep tissue. Compared with tissue specimens, swab cultures had a positive predictive value of 84.4%, negative predictive value of 44.0% and overall accuracy of 73.0%. The use of the term ‘predictive value’ presumes the presence of a gold-standard for diagnosis, which we have not done. In keeping with our study, 52% of their patients had received antibiotic therapy at presentation. In a study of 50 patients with an infected DFU (who had not recently received antibiotic therapy), Demetriou et al.63 used a tissue specimen culture against which to compare a swab specimen culture. The results obtained from tissue cultures were the same as those obtained by swab in only 50% of patients. Whereas the sensitivity of swab compared with tissue cultures was 100%, the specificity was ≤ 40%, and because of the isolation of ‘contaminating’ flora it was < 20% for ‘true pathogens’. Similar to our results, they noted no important differences in results between patients with neuropathic compared with neuroischaemic ulcers.
The substantial proportion of samples in our study that reported no pathogens may reflect the difficulty in establishing a diagnosis threshold for clinical infection in the diabetic foot, which is related to the frequent presence of peripheral neuropathy and arterial disease.115 Alternatively, it may be related to poor sampling technique, transport media that fail to maintain viability of pathogens, or a choice by the microbiologist to report only those pathogens, according to their professional judgement threshold, necessary for reporting (i.e. they may be assessing ‘necessity for reporting’ when faced with a modest number of mixed flora).
