QualityAssessment of the Saudi Initiative Guideline for Bronchial Asthma(Clinical Practice Guidelines for thediagnosis and management of asthma in adults and children, 2016) ABSTRACT The Saudiinitiative for asthma (SINA) is a local Saudi guideline adopted from the GlobalInitiative for Asthma (GINA) and the National Asthma Education and PreventionProgram.
The goal of this study is to assess the quality of this guideline byusing AGREE II tool. Methodology: The guideline was evaluated independently byfive reviewers, three of them were family physicians, one was Internist, theother was pediatric pulmonologist. The appraisers assessed the quality of SINAguideline by using the AGREEII instrument (The Appraisal of Guidelines forResearch & Evaluation). A descriptive statistical analysis through thecalculation of the total, standardized-score value by each reviewer, and thevalue per domain was performed.
Results: Wefound that clarity of presentation for SINA performed the best (74%). It scoredaverage on the domain related to scope and purpose, and stakeholderinvolvement. Applicability and editorial independence (both 25%) received thelowest scores of all the AGREE II domains. However, the overall quality of theSINA guidelines was poor.Conclusion: In conclusion, theresults of our studies emphasize the need of addressing the Domains with lowscores including Rigor of development, Applicability and editorialindependence.
Key words: Asthma, Agree II,SINA guideline, clinical practice guideline. INTRODUCTIONAsthmais one of the common disease worldwide. It affects about 334 millions.
(1) InSaudi Arabia, there are many studies assessing the prevalence in differentcities, however, there is no national survey conducted to estimate theprevalence. One study concludes that the prevalence is increasing by comparingthe prevalence in different cities and different years, 1986 and 1995.(2) Theprevalence of asthma is suggested to affect more than 2 million Saudis.(3) Onestudy analyzed data from a national household survey conducted in KSA in 2013and found that the prevalence of asthma in KSA was 4.05 % (95 %confidence interval CI: 3.54–4.62 %).(4) SaudiInitiative for Asthma (SINA) was developed in 2009 to make national guidelinefor asthma management.
SINA guideline was developed by group of Saudi physicianand they are members in Saudi Thoracic Society (STS). The SINA guideline hadthree versions; the first one was published on 2009,(5)then second version on 2012(6)and, the later on 2016.(3) The goal of SINA guidelines is to have updatedguidelines, which are simple to understand and easy to use by non-asthmaspecialists, including primary care and general practice physicians.
(3, 6) Clinical practice guidelines (CPG) have been developed to assisthealth practitioner in making decisions easily on scientific way. CPG has beendefined as “statements that include recommendations intended to optimizepatient care that are informed by a systematic review of evidence and anassessment of the benefits and harms of alternative care options”.(7) Manystudies showed that the methodological quality of many guidelines is not wellestablished and there are variety between the different guidelines.(8) Guidelinesare need to be well structured by using specific quality tool and they shouldbe preferably developed within a structured and coordinated guidelines program.(9) One of the common used tools to assess the quality ofguidelines is AGREE II tool.
AGREE is synonym that stands for “The Appraisal ofGuidelines for Research & Evaluation”. AGREE was published first time in2003 by a group of international guideline developers and researchers, theAGREE Collaboration.(10) lateron, they publish the modified version on 2010; AGREE II. AGREE II was used bymany studies to assess the quality of different guidelines.(11-15) To our knowledge, the quality of asthmaguideline (The Saudi initiative for Asthma) has not yet been systematicallyevaluated. Therefore, we conducted this study to assess the quality of thisguideline by using AGREE II tool. METHODOLOGYThe Saudi initiative for bronchial asthma 2016(guidelines for the diagnosis and management of asthma in adults and children)was chosen for critical appraisal.
This study was conducted on September untilDecember 2017. Appraisal process:The guideline was evaluated independently by five reviewers, three of them werefamily physicians, onewas Internist, the other was pediatric pulmonologist with more than 10 years’experience in there relevant specialty respectively. All the reviewers are familiarwith the principles of guidelines development and research methodology. The appraisersassessed the quality of SINA guideline by using the AGREEII instrument (The Appraisalof Guidelines for Research & Evaluation). Structure of AGREE II instrument: The AGREEII is both reliable and valid tool.(16, 17) The AGREEII is a 23-item tool comprising six qualitydomains followed by two overall assessment items. Domain1″Scope and purpose” isconcerned with the overall aim of the guideline, the specific health questions andthe target population (three items).
Domain2″Stakeholder Involvement” focuses onthe extent to which the guideline was developed by the appropriate stakeholdersand represents the views of its intended users (three items). Domain3″Rigour of Development” relatesto the process used to gather and synthesis the evidence, the methods to formulatethe recommendations and to update them (eight items). Domain4″Clarity and Presentation”deals with the language, structure and format of the guideline (three items).
Domain5″Applicability” pertains to the likely barriers and facilitators to implementation,strategies to improve uptake and resource implications of applying the guideline(four items); and Domain6 “Editorial Independence” is concerned with the formulationof recommendations not being unduly biased with competing interests (two items).A seven-point Likert scale (ranged between score 7 for strongly agree to score 1for strongly disagree) is used to score each domain item. The overall assessmentincludes two items: the rating of the overall quality of the guideline (with a scaleranging from 7 for higher possible quality to 1 for lower possible quality) andwhether the guideline would be recommended for use in practice, with a three choices(not recommended; recommended with modifications; or recommended).The standardized score for the individual domain, which ranged from 0 to100%,was calculated by using the following formula: (actual score –minimalpossible score)/(maximal possible score – minimal possible score)*100%Overall guideline evaluation reached consensus according topartial item assessment and the global judgment by reviewer.StatisticalAnalysis: Adescriptive statistical analysis through the calculation of the total,standardized-score value by each reviewer, and the value per domain wasperformed.
Agreement among the five reviewers was determined by the intraclasscorrelation coefficient (ICC) Descriptive values were provided. An average score of 60% was chosen(standardization of total points as a percentage over maximum points) toestablish the proportion of guidelines that scored points above this level ineach domain. The ICC with a 95% CI as an overall indicator of agreement amongthe reviewers for each of the 23 items of the AGREE II instrument was calculate RESULTSOne of the good points in SINA is the clarityof presentation. The reviewers found that clarity of presentation domainperformed the best (74%). In this domain, the recommendations were specific andunambiguous with different options for management. Key recommendations wereeasily identifiable and they were clearly presented. SINA scored average on the domain related toscope and purpose, and stakeholder involvement, 50% and 49%, respectively.
Inscope and purpose domain, the population to whom the guideline is meant toapply is described in a good way. However, the health questions need to be moreobvious. In stakeholder involvement domain, the guideline development groupdoes not include other stakeholders such as patient, nurses…etc. In thisdomain, the target users are mentioned but is not easy to find inside theguideline. Applicabilityand editorial independence (both 25%) received the lowest scores of all theAGREE II domains. In applicability domain, the potential resource implicationsof applying the recommendations need to be considered to make the guidelinemore applicable.
While in editorial independence domain, the competing interestof guideline development group members should be clear for the guideline’susers. The overall quality of the SINA guidelines wasaverage (43%). However, it scored good on the domain related to clarity ofpresentation but get less than 50% in other domains.
Table 1 summarizes theresults of the appraisal and the mean score for each domain. The reviewers recommend using SINAguideline for asthma management with taking in consideration the modificationsthat were noted and suggested during the appraisal. Table 1.
Individual AGREE II domain results for SINA guideline and mean score of all domains. Domain Mean Score (%) Scope and purpose 50 Stakeholder involvement 49 Rigor of development 36 Clarity of presentation 74 Applicability 25 Editorial independence 25 Average AGREE II score (%) 43 AGREE II = Appraisal of Guidelines for Research & Evaluation. SINA = Saudi Initiative for Asthma.
DISCUSSIONThe AGREE instrument is generic and can be applied toguidelines in any disease area including those for diagnosis, health promotion,treatment or interventions.(18) Using AGREE II tool wedecided to critically evaluate SINA guideline, as best of our knowledge notmuch studies analysing Asthma guideline in the region are available. It is notmentioned which tool is used to build up this guideline. This study shows thatthe methodological quality of the current guideline for asthma was low whenusing the AGREE II instrument. The domains like rigor of development,applicability, and editorial independence have lower scores but clarity ofpresentation score very good. The scope and purposedomain addresses the overall aim of the guideline, the clinical question, andthe target population.
Our study have 50% score in this domain which is closeto other studies as only two CPGs scored 60% in this domain. This was relatedto lack of details in the description of overall objective, health question,and target-population definition.(19, 20)The scores forstakeholder involvement remained, 48.88% in our study. Apparently, they did notinvolve patients, GP or methodologist apparently. Involving patients indecisions regarding their management has shown promising results. Developmentof a guideline requires multidisciplinary panels that include physicians,Methodologic experts, policy makers, and patient representatives.
The stakeholder involvement domain addresses thedegree to which the guideline represents the views of its intended users.(19, 20)The score rigor ofdevelopment domains remained, 35.83%. The low scores of several of theguidelines were related to the poor performance in the domain of the rigor ofdevelopment.
This finding reflects a lack of methodologic expertise, which isoften related to a lack of adequate resources. No document described itsprocedure for updating guidelines.The score clarity ofpresentation domains remained, 74.44% the clarity of presentation domainsaddress potential, language and format respectively. An established goldstandard for diagnosing asthma used these important issues were addressed onlyby one of the included guidelines.The lowest score ofall six domains was in applicability and editorial independence. Theapplicability domain addresses the potential organizational, behavioural, andcost implications of guidelines implementation.
In our study, this domain scorewas 25%. A previous review reported similar results for this domain in otherconditions, arguing that it could happen because the guideline panelsconsidered the development and implementation of the guidelines as separateactivities.(21) Editorial independence domain also scored 25%, as withonly 11 guidelines scoring above 30% in a study.
(12)In conclusion, theresults of our studies emphasize the need of addressing the Domains with lowscores including Rigor of development, Applicability and editorialindependence. As many guidelines have improved over time and most couldcontinue improving, particularly in the areas of development. Patientparticipation is needed for enhance applicability. There is a need to involvespecialist from other areas also to review the clinical guidelines for morereliable and valid results.