Topic Development and Refinement

The topic for this report was nominated by a health care professional association in a public process using the Effective Health Care Web site (http://effectivehealthcare.ahrq.gov/). Working from the nomination, we drafted the initial KQs and analytic framework and refined them with input from key informants representing the fields of urology, pediatric care, primary care, and patient advocacy. All members of the research team were required to submit information about potential conflicts of interest before initiation of the work. No members of the review team have any conflicts.

After review from the AHRQ, the questions and framework were posted online for public comment. We made several changes in response to the AHRQ review. We reworded KQ1 and altered the analytic framework to clarify that the review would not address the diagnosis of cryptorchidism. We also developed separate population, interventions, outcomes, timing, and settings (PICOTS) criteria for the workup and intervention KQs. We received no comments during the public posting phase. We prepared final KQs and resubmitted them to AHRQ for review.

We identified technical experts on the topic to provide assistance during the project. The Technical Expert Panel (TEP), representing the fields of pediatric urology, pediatric surgery, and pediatric endocrinology, contributed to the AHRQ’s broader goals of (1) creating and maintaining science partnerships as well as public-private partnerships and (2) meeting the needs of an array of potential customers and users of its products. Thus, the TEP was both an additional resource and a sounding board during the project. The TEP included five members serving as technical or clinical experts. To ensure robust, scientifically relevant work, we called on the TEP to review and provide comments as our work progressed. TEP members participated in conference calls and discussions through email to:

• Help to refine the analytic framework and KQs at the beginning of the project;
• Discuss the preliminary assessment of the literature, including inclusion/exclusion criteria
• Provide input on the information and domains included in evidence tables;
• Help to develop a hierarchy of participant characteristics and outcomes to assess systematically.

Literature Search Strategy

Inclusion and Exclusion Criteria

Table 3 lists the inclusion/exclusion criteria we selected based on our understanding of the literature, key informant and public comment during the topic-refinement phase, input from the TEP, and established principles of systematic review methods.

Table 3

Inclusion and exclusion criteria.

In addition to the overall criteria above, for KQ1a we only included studies that evaluated the accuracy with which pre-operative imaging techniques identified the presence and location of testicles, with confirmation by surgery. If a study had negative imaging for all subjects or if the imaging techniques were not adequately described or if relevant data to calculate diagnostic performance were not available, it was excluded. We required that the study report data on presence or absence of testicles along with identifying the position of testicles (abdominal, inguinal or scrotal) as determined by imaging techniques and surgery.

For KQ1a and KQ1b, we accepted single arm studies because the purpose of those questions was to assess treatment planning, which is based on the success of the approaches for identifying and locating testicles before and after surgical confirmation. For the treatment questions, however, (KQ2 and KQ3) studies had to include an appropriate comparison group (i.e., comparison of different treatments, hormonal vs. surgical therapy, treatment vs. no treatment).

Studies of individuals with disorders of sexual development or ambiguous genitalia were excluded, because the clinical complexity of these patients means that they are typically undergoing multiple treatments for their conditions and results in this population would not be generalizable to the population typically presenting with cryptorchidism.

Study Selection Process

Once we identified articles through the electronic database searches and hand searching, we examined abstracts of articles to determine whether studies met our criteria. Two reviewers separately evaluated the abstracts for inclusion or exclusion, using an Abstract Review Form (Appendix B). If one reviewer concluded that the article could be eligible for the review based on the abstract, we retained it. Following abstract review, two reviewers independently assessed the full text of each included study using a standardized form (Appendix B) that included questions stemming from our inclusion/exclusion criteria. Disagreements between reviewers were resolved by a third reviewer. All abstract and full text reviews were conducted using the DistillerSR online screening application (Evidence Partners Incorporated, Ottawa, Ontario). Excluded studies and the reasons for exclusion are presented in Appendix C.

Development of Evidence Tables and Data Extraction Process

The staff members and clinical experts who conducted this review jointly developed the evidence tables. We designed the tables to provide sufficient information to enable readers to understand the studies and to determine their quality; we gave particular emphasis to essential information related to our KQs. Three distinct evidence table templates were employed to facilitate the extraction of data most relevant to a specific KQ: one for KQ1a, one for KQ1b, and one for KQs 2 and 3. We based the format of our evidence tables on successful designs used for prior systematic reviews.

The team was trained to extract data by extracting several articles into evidence tables and then reconvening as a group to discuss the utility of the table design. We repeated this process through several iterations until we decided that the tables included the appropriate categories for gathering the information contained in the articles. All team members shared the task of initially entering information into the evidence tables. A second team member also reviewed the articles and edited all initial table entries for accuracy, completeness, and consistency. The two data extractors reconciled disagreements concerning the information reported in the evidence tables, the most common of which was study design. The full research team met regularly during the article extraction period and discussed global issues related to the data extraction process. In addition to outcomes related to intervention effectiveness, we extracted all data available on harms. Harms encompass the full range of specific negative effects, including the narrower definition of adverse events.

The final evidence tables are presented in their entirety in Appendix D. Studies are presented in the evidence tables alphabetically by the last name of the first author. Where possible, studies resulting from the same study population were grouped into a single evidence table. A list of abbreviations and acronyms used in the tables appears at the beginning of that appendix.

Data Synthesis

Evidence tables were completed for all included studies, and data are presented in summary tables and analyzed qualitatively in the text. In addition, quantitative analysis was used for KQ1a and KQ3 as follows. For KQ1a, the accuracy of imaging techniques was evaluated for concordance with surgical results (open or laparoscopic), which were considered the gold standard. Testicles were considered present if they were normal or atrophic including testicular nubbins (defined as a small wad of fibrotic tissue at the end of the spermatic vessels). Those testicles described as absent or vanishing were coded as absent testicles. The performance characteristic data for each imaging technique compared to surgery was arranged into a two-by-two table first and the sensitivity (Se), specificity (Sp), positive predictive value (PPV), negative predictive value (NPV) and overall accuracy rate (OAC) were calculated (Table 4). Next, the accuracy of each imaging modality at capturing the position of the testicles was compared to surgical results. Locations of the normal testicles have been categorized as intra-abdominal and inguino-scrotal. Normal testicles located in the inguinal/scrotal regions were grouped as inguino-scrotal. Both abdominal and inguino-scrotal atrophied testicles were classified as “Atrophy.”

Table 4

Performance characteristic measures.

For KQ3, data on proportions achieving testicular descent were pooled and the weighted proportions (sum of all success testicles/total number of testicles in studies) were calculated for each treatment type. Similarly, weighted testicular atrophy rates were derived for each of the surgical techniques.

Rating Quality of Individual Studies

Three existing, validated tools were used to assess quality of individual studies: the Cochrane Risk of Bias tool,¹² the Newcastle-Ottawa Quality Assessment Scale,¹³ and the Quality Assessment of Diagnostic Accuracy Studies-Revised (QUADAS-2) tool.¹⁴ The Cochrane Risk of Bias tool is designed for the assessment of studies with experimental designs and randomized participants. Fundamental domains include sequence generation, allocation concealment, blinding, completeness of outcome data, and selective reporting bias. The Newcastle-Ottawa Quality Assessment Scale was used to assess the quality of nonrandomized studies (e.g., cohort and case-control studies). This scale assesses three broad perspectives: the selection of study groups, the comparability of study groups, and the ascertainment of either the exposure or outcome of interest for case-control or cohort studies, respectively. The QUADAS-2 tool was used to assess imaging studies (KQ1a) and includes items on the representativeness of patient spectrum, selection criteria, reference standard, verification bias, timing, and study withdrawals. All three tools are presented in Appendix E.

Quality assessment of each study was conducted by two team members independently using the forms presented in Appendix E. Any discrepancies were adjudicated by the two team members. Investigators did not rely on the study design as described by authors of individual papers; rather, the methods section of each paper was reviewed to determine which rating tool to employ. The results of these tools were then translated to the Agency for Healthcare Research and Quality standard of “good,” “fair,” and “poor” quality designations using conversion thresholds developed by the team, as no explicit guidance exists. The conversion thresholds are presented in Appendix E.

Strength of Evidence for Each KQ

We evaluated the overall strength of the evidence for the primary outcomes of treatment (testicular descent, atrophy, longer term fertility and cancer) (Tables 27–31). We used the approach to strength of evidence assessment described in the Effective Health Care Program Methods Guide for Effectiveness and Comparative Effectiveness Reviews.¹⁵

We examined the following four major domains:

• Risk of bias (low, medium, or high),
• Consistency of findings (inconsistency not present, inconsistency present, or unknown or not applicable),
• Directness (direct comparison of influence on outcomes in RCT, or indirect information from observational research), and
• Precision (precise or imprecise based on outcomes rates, size of the individual studies and the total number of participants in the studies for the category of intervention).

For retrospective studies without a control group, we used an implicit comparator given the known natural history of disease.

The overall strength of evidence could be graded as “high” (indicating high confidence that the evidence reflects the true effect and further research is very unlikely to change our confidence in the estimate of effect); “moderate” (indicating moderate confidence that the evidence reflects the true effect and further research may change our confidence in the estimate of effect and may change the estimate); “low” (indicating low confidence that the evidence reflects the true effect and further research is likely to change our confidence in the estimate of effect and is likely to change the estimate); or “insufficient” (indicating that evidence is either unavailable or does not permit estimation of an effect). Strength of evidence was applied both to evidence of benefit and to evidence of lack of benefit. This means for instance that for a category of intervention in which there are multiple studies, with moderate bias and direct evidence showing no effect, and a single study reporting an insignificant reduction, the body of literature for the category of intervention could be scored as low evidence of lack of benefit.

Applicability

Finally, it is important to consider the ability of the findings to apply both to other populations and to other settings. Our assessment of applicability included determining the population, intervention, comparator, and setting in each study and developing an overview of these elements for each intervention category (Appendix F).

Peer Review

External experts in the fields of pediatrics, urology, and endocrinology as well as individuals representing stakeholders and patient/parent advocates were invited to contribute feedback of this review. Officers at AHRQ and an associate editor also provided comments. The draft report was posted on the AHRQ Web site for 4 weeks to elicit public comment. All peer review and public comments were addressed and the text revised as appropriate. All comments and resulting edits were documented and synthesized into a disposition report, which will be made publicly available 3 months after the final version of the review is posted on the AHRQ Web site.