|Year : 2018 | Volume
| Issue : 2 | Page : 52-59
Assessing the evolution and level of evidence of spine surgery research in Saudi Arabia
Saleh Baeesa1, Yazid Maghrabi1, Sadek Bajoh1, Suhail Bajammal2
1 Department of Surgery, Division of Neurosurgery, King Abdulaziz University Hospital, Faculty of Medicine, King Abdulaziz University, Jeddah, Saudi Arabia
2 Department of Surgery, Division of Orthopedics, Faculty of Medicine, Um Al-Qura University, Makkah, Saudi Arabia
|Date of Web Publication||2-Apr-2018|
prof. Saleh Baeesa
Division of Neurosurgery, Faculty of Medicine, King Abdulaziz University, P. O. Box: 80215, Jeddah 21589
Objectives: To measure the impact of Saudi publications on global spine surgery literature, to evaluate the quality of these papers by measuring the level of evidence (LOE) and then to compare it with similar international studies done on the topic of spine surgery.
Materials and Methods: The study design was a systematic review, where a systemic online search using PubMed, Google Scholar and Embase was carried out using search terms related to spine surgery. The impact of each article was measured by the citation number and was reviewed by Oxford Centre for Evidence-Based Medicine LOE scale.
Results: We identified 128 articles that met the inclusion criteria, of which 58% were Level IV studies. The most common study design was case reports (45%). We compared our study with the similar or different methodology of international studies for spine surgery research.
Conclusion: This first study, for the analysis of spine surgery literature in Saudi Arabia, shows that Saudi publications in spine surgery have little impact on the global spine surgery research. The LOE was low and that there is insignificant change throughout 25 years. A national multicentre or international collaborative research is recommended to produce high LOE research.
Keywords: Evidence-based medicine, level of evidence, neurosurgery, orthopaedics, quality, Saudi Arabia, systemic review
|How to cite this article:|
Baeesa S, Maghrabi Y, Bajoh S, Bajammal S. Assessing the evolution and level of evidence of spine surgery research in Saudi Arabia. J Health Spec 2018;6:52-9
|How to cite this URL:|
Baeesa S, Maghrabi Y, Bajoh S, Bajammal S. Assessing the evolution and level of evidence of spine surgery research in Saudi Arabia. J Health Spec [serial online] 2018 [cited 2018 Dec 12];6:52-9. Available from: http://www.thejhs.org/text.asp?2018/6/2/52/229020
| Introduction|| |
The impact of a paper on the international literature of a respective field is measured by the number of times the article has been cited, which provides insight on the influence and academic strength of the publication and the progress of the field itself., Moreover, high-quality papers influence clinical decision-making and improve patients' care significantly. The quality of a certain publication is measured by the level of evidence (LOE) that paper provides. There have been many international studies that quantified the impact of certain publications in the field of spine surgery, neurosurgery and orthopaedics.,,, Moreover, only one Saudi study by Jamjoom et al. dealt with such a topic. Regarding the quality of the spine publications, two international studies have discussed the quality of spine surgery publications. National publications were reported in three Saudi studies with different specialties but with a similar methodology to our study.,,
The population of Saudi Arabia is roughly 30 million, with a rapidly developing health-care system seeking to achieve the high standards set by health-care systems of developed countries. The estimated number of practicing neurosurgeons in Saudi Arabia is 475, while there are 2163 practicing orthopaedic surgeons. This necessitates a method to measure the development of these specialities in Saudi Arabia. A review of the impact and quality of spine surgery publications in Saudi Arabia achieves such a goal.
To our knowledge, there has been no study that combined the issues of impact and quality of Saudi spine surgery publications. We intend to measure the impact of Saudi publications on global spine surgery literature and to evaluate the quality of these papers by measuring the LOE. We will then compare it with local studies that have a similar methodology and similar international studies that were done in spine surgery.
| Materials and Methods|| |
Information sources and search strategy
This study was conducted at King Abdulaziz University Hospital, Jeddah, Saudi Arabia, between June and August 2016. Several databases were used including PubMed, Google Scholar and Embase using search phrases that are related to the area of spine surgery [Table 1]. The search was conducted by the following protocol: (Search term) AND (Saudi Arabia). After screening abstracts of related articles, we retrieved the articles that met the inclusion criteria for final analysis. The impact of each article was measured by the number of times the article had been cited, which was data imported from Google Scholar. Two reviewers reviewed and graded each article using Oxford Centre for Evidence-Based Medicine-LOE Scale. A schematic illustration of the review process is provided in [Figure 1].
The inclusion criteria for articles were as follows: (1) spine surgery-related clinical articles; (2) published in English; (3) published between January 1990 and January 2016; (4) the first author must be affiliated with a Saudi institution; (5) study's population must be based on a Saudi institution, either fully or partially; (6) in case of systematic reviews and meta-analyses with international collaboration at least one author must be affiliated with a Saudi institution and (7) the full-text article must be available for analysis. The exclusion criteria were as follows: (1) editorials, literature reviews, animal studies, cadaveric studies or laboratory studies; (2) clinical studies published earlier than 1st January 1990; (3) study's population based in a geographical area outside Saudi Arabia and (4) a clinical study with no available full text.
Data collection process and data items
Several parameters were collected from each article and entered in a Microsoft Excel Spreadsheet such as journal name, the impact factor (IF) of the journal, year of publication, first author affiliation, the number of authors, city and study design. Moreover, each article was then categorised into different categories related to spine surgery, such as degenerative pathology, developmental spine pathology, oncology, trauma, rehabilitation, deformity, rheumatologic pathology and general spine care.
Microsoft Excel (Microsoft, Redmond, Washington, USA) was used for the statistical analysis. Data were compared using F-test. A P < 0.05 and a confidence interval of 95% were considered statistically significant. Kappa score was calculated to determine the degree of agreement between the two reviewers. Frequency was expressed regarding percentage, and mean and median were the only measures of central tendency utilised for data analysis. Pearson's coefficient was calculated to detect the degree of correlation.
| Results|| |
We initially identified 2230 articles, of which only 128 articles met the eligibility criteria. The strength of agreement between the two reviewers was good (Kappa = 0.65). Citation numbers ranged from 1 to 214 (median 10). Thirty-one articles (24.22%) had no recorded number of times cited. [Table 2] illustrates the top 25 most cited Saudi publications in spine surgery. The LOE of the articles in this study are as follows: 0.78% Level I, 7.81% Level II, 32.81% Level III and 58.6% Level IV. The most commonly encountered study design was case reports in 57 (44.35%), followed by retrospective studies in 31 (24.21%), prospective studies in 13 (10.15%), case series in 13 (10.15%), narrative and systematic reviews in 8 (6.25%), randomised controlled trials in 3 (2.34%) and cross-sectional studies in 3 (2.34%). [Figure 2] describes the frequency of publication from 1990 to 2016.
Regarding cities, 86 (67.18%) articles originated from Riyadh; 15 (11.72%) from Al-Khobar; 13 (10.16%) from Jeddah; 5 (3.91%) from Makkah; 3 (2.34%) from Madina; 2 (1.56%) from Dhahran and one article (0.78%) from each of the following cities: Abha, Dammam, Tabuk and Taif. Regarding provinces, Riyadh Province published the largest number of articles with 86 (67.18%) articles, followed by Makkah Province with 19 (14.84%), Eastern Province with 18 (14.06%), Madina Province with 3 (2.34%), Asir Province with 1 (0.78%) and Northern Border Province with 1 (0.78%).
Academic institutions produced the majority of papers with 61 (47.66%) articles, followed by Ministry of Health Hospitals (MOH) with 38 (29.69%), Military Health Institutions with 26 (20.31%) and the Private Sector with 3 (2.34%). Analysis of LOE according to sectors revealed that 16.4% (n = 10) of the academic sector's research output was at Level II; 36.1% (n = 22) was at Level III and 47.5% (n = 29) was at Level IV. In regard to MOH hospitals, of their total publications, 28.9% (n = 11) were at Level III and 71.1% (n = 27) were at Level IV. Military institutions produced one study at Level I, 34.6% (n = 9) at Level III, 57.7% (n = 15) at Level IV and one publication at Level V. Private institutions produced only Level IV studies. [Table 3] summarises the top 10 Saudi institutions that contributed significantly to the Saudi spine surgery literature.
|Table 3: Top 10 Saudi institutions and their contribution to spine surgery literature|
Click here to view
In terms of the spine pathology categorisation of the published articles, 24 (18.75%) articles dealt with general spine care, 21 (16.41%) with spine infections, 18 (14.06%) with traumatic spine injury, 14 (10.94%) with spinal neoplasms, 12 (9.38%) with developmental spine pathologies, 7 (5.47%) with degenerative spine diseases, 9 (7.03%) with rheumatologic pathologies, 9 (7.03%) with spinal neurovascular conditions, 7 (5.47%) with spine deformity and 5 (3.91%) with spinal rehabilitation. Cervical spine was the primary anatomic focus of 32 (25%) of the papers, along with 13 on lumbar spine (10.16%), 13 on thoracic (10.16%) and 6 on sacral (4.69%). Regarding anatomic junctions, the thoracolumbar junction was addressed in 6 (4.69%) articles, craniocervical in 4 (3.13%) and lumbosacral in 2 (1.56%). Thirty-seven articles (28.91%) dealt with an anatomic focus that is mixed with different areas, whereas in 15 (11.72%), the anatomic focus was not specified.
The IF of journals ranged from 0.089 to 3.973 (median 1.256). Twenty-three articles (0.18%) were published in journals with unrecorded IF. The number of journals used for publication was 50. The 10 most frequently used journals were as follows: spine with 14 (10.94%), neurosciences (Riyadh) with 12 (9.38%), Saudi Medical Journal with 12 (9.38%), surgical neurology with 8 (6.25%), British Journal of Neurosurgery with 8 (6.25), Annals of Saudi Medicine with 7 (5.47%), International Orthopaedics with 5 (3.91%), spinal cord with 4 (3.13%), European Spine Journal (ESJ) with 4 (3.13%) and Asian Spine Journal with 3 (2.34%) and the 51 (39.48%) remaining articles were published in 40 different journals [Figure 3].
The majority of the articles included in this study (49.22%) dealt with an adult population, whereas 25% considered paediatric cases, and in 6.25%, the population was not specified. There was international collaboration in 7.03% of the articles. One hundred and six (82.81%) articles were retrieved from PubMed, 18 (14.06%) from Google Scholar and 4 (3.13%) from Embase. In regard to search terms, 'spinal cord' yielded 23.44% of the results, 'spine surgery' 21.88%, 'spine fusion' 16.41%, 'spinal surgery' 13.28, 'spine fixation' 5.47%, 'spinal fusion' 3.91%, 'spondylolisthesis' 3.13%, 'spondylitis' 3.13%, 'spinal injury' 3.13%, 'herniated disc' 2.34%, 'discectomy' 1.56%, 'spinal rehabilitation' 1.56%, 'spinal cord AND King Saud University' 0.78%.
| Discussion|| |
The primary aims of our study were to measure the impact of Saudi spine publications on the global spine literature using the number of times each article was cited as data imported from Google Scholar. Moreover, we aimed to quantify and evaluate the quality of those papers. There have been many publications that dealt with the impact of publications of certain specialties and the quality of articles separately.,,,,,,, However, there has been no study that combined the two concepts despite the strong relationship between quality and impact.
Studies that measure the impact of publications confined to a particular geographical area are lacking. In our opinion, such knowledge provides invaluable information about the progress of an individual field in a certain country. Our study is considered the first that deals with the impact of spine surgery publications in Saudi Arabia.
Ponce and Lozano have defined citation classic as the articles that get cited more than 400 times. Citation numbers of our study ranged from 1 to 214 (median 10). We found no single article that has reached the level of being a citation classic. When comparing citation rates of our study with that done by Dong et al., we found that the results are comparable. This suggests that Saudi publications in spine surgery are still at a modest level regarding the times they are cited. Hence, quantification and evaluation of the quality of the publication are must for better analysis of the progress of spine surgery literature in Saudi Arabia. An argument might arise in regard the reliability of citation rates generated by Google Scholar; although there might be some unreliability, it provides quantifiable data concerning the impact of individual articles. Moreover, since much of the included articles are published in Thomson Reuters non-indexed journals, Google Scholar offers an excellent, uniform assessment of impact.
We have found that many of the published articles were Level IV, which is comparable to several national and international studies.,,,,, The most encountered study design was case reports, which is consistent with local research but differs from international studies in orthopaedics and spine surgery, where there is a tendency to publish retrospective studies.,,,,, Moreover, there has been no change over 26 years in the pattern of LOE. There was only a slight increase in the frequency of publications in the last 5 years (P = 0.9) [Figure 4]. Comparing the time intervals, 1990–2002 and 2003–2016, we have found that there was no statistically significant difference in LOE (P = 0.3). However, there was a statistically significant difference in frequency between the two time intervals (P = 0.03) [Figure 5].
|Figure 4: Level of evidence pattern of included articles from 1990-2016 using Oxford Centre for Evidence-Based Medicine Scale|
Click here to view
|Figure 5: Comparison between two decades worth of Saudi spine surgery publications in terms of frequency and quality|
Click here to view
Contrary to the belief that correlation would exist between LOE and citation numbers, we have found that there is a weak correlation (r = −0.13) (P = 0.04). On the other hand, we found a somewhat good correlation between journals IF and number of times cited (r = 0.42) (P ≤ 0.00001). This would imply that publishing in high-impact journals would lead to an article being cited more, regardless of the LOE. Amiri et al. found good correlation between LOE and IF, which is comparable with our previously mentioned correlation.
Looking more in depth, the topic of LOE in clinical spine surgery has been discussed by many authors. Wupperman et al. were the first ones to address this issue. In their study, they reviewed articles published in the journal spine between January and June 2003. Included articles were rated using a rating scale published by the Journal of Bone and Joint Surgery-American Volume. They found that 16.10% of the 112 included articles represent Level I, 22.30% belong to Level II, 8% belong to Level III and 53.60% belong to Level IV [Table 4]. Multiple methodological differences exist between our current study and that of Wupperman et al. First, they have included only articles published in one journal, which is not representative of all clinical spinal publications' LOE or represent multiple countries. Furthermore, they used a scale that is different from the one used in our current study and in a narrower time frame. Even though there are methodological differences, their results are consistent with the results of our present study, as Level IV studies predominate in both studies.
|Table 4: Comparison of level of evidence of current study with other international studies|
Click here to view
Amiri et al., on the other hand, tried to fix the methodological flaw in Wupperman et al. by creating a representative sample of clinical spine publications. In their study, they evaluated clinical publications in five leading spine journals including The Spine Journal, Spine, ESJ, Journal of Neurosurgery: Spine and Journal of Spinal Disorders and Techniques, and they only included articles published in 2010. They graded the included articles using the Oxford LOE scale., A total of 703 articles were included in their study, 4.7% of which were of Level I, 23.2% of Level II, 12.5% of Level III and 59.6% of Level IV [Table 4].
Only two methodological differences exist between our study and that of Amiri et al., the first of which being that they restricted their review to a few journals, rather than collecting as many clinical spine surgery publications as possible to get a more representative sample for comparison; the second is their narrow time frame. They overcame this issue by obtaining a broad cross-section of 703 articles. A uniform consistency is present as Level IV publications represent most of the included sample.
In the previously mentioned studies, the authors did not focus on the design of each included article, and their primary focus was mostly on the type of article: therapeutic, prognostic, diagnostic or economic.
Falavigana et al. were the first to attempt LOE evaluation studies confined to a particular geographical area. They conducted a literature search to collect publications published by Brazilian spine surgeons using PubMed. They restricted their search to the interval between January 2000 and December 2011. They found that of the 206 included articles, 1.5% qualified to be of Level I, 14.9% of Level II, 6.8% Level III, 54.9% of Level IV and 21.8% Level V [Table 4]. In their study, they noted the design of the included publications as follows: clinical studies, without the specification of the type, represented 37.4%, whereas case reports represented 31.1%, case series represented 7.8% and meta-analyses represented 1.5%. Contrary to the other mentioned studies, Falavigana et al. included other article types that were excluded from the other two studies, such as narrative reviews 6.8% and technical reports 4.9%.
This study shares some similarities and differences with these aforementioned studies. Methodologically, the search strategy used in our study is similar to that of Falavigana et al., but the main difference is that they have used only one database, whereas we have used three databases. The other issue is that they only included articles published by spine surgeons; in our opinion, many spine surgery-related articles are overlooked since not all will be published by a spine surgeon. We can see, as observed in the previous two articles, that Level IV articles predominate, and this is consistent with our study. Moreover, we cannot evaluate if case reports are the most numerous study design since the authors did not clarify the types of clinical studies included.
The fact that Level IV publications predominate most of the spine surgery publications in the previously mentioned studies and our current study accounts for an actual weakness in the spine literature, not only in Saudi Arabia but also in the global surgical community.,,,],, According to Falavigana et al., there is a tendency for producing more case reports, which represent a low LOE due to funding constraints, lack of time and the fact that it is an excellent way to convey personal experiences in both clinical and operative aspects of spine surgery. We have to take into account that citation rates of included articles were not discussed in Wupperman et al. and Amiri et al., yet it was mentioned briefly by Falavigana et al.
The problem of impact and quality of spine surgery publications is multifaceted and must be discussed as such. It has been suggested that research in Saudi Arabia faces many difficulties ranging from finical, ethical and logistical issues. Moreover, inadequate training and integration of evidence-based medicine (EBM) in medical school curricula could have contributed to this problem. Regarding health policy-makers, Koutras et al. have proposed that more spending in research would lead to a higher rate of publications. The modest rate of Saudi publications in spine surgery could be explained by less research funding in this surgical field. Moreover, education of the public regarding clinical trials plays a vital role in producing high-quality research.
Regarding academics, medical schools' curricula give limited attention to EBM and research methodology. On the other hand, the new requirements of the Saudi Commission for Health Specialties for post-graduate programs applicants emphasise the importance of published papers during the selection process. Hence, medical schools should initiate EBM courses coupled with courses in research methodology, ending with encouraging medical students to pursue their projects to get published.
Program directors in post-graduate programs have a pivotal role in promoting the involvement of residents in research activities and offering incentives for such activities. This should be coupled with organising courses for residents to be trained in research methodology. Moreover, it was a great move when Saudi Commission for Health Specialties (SCHS) placed a minimal research requirement for all residents to be Saudi board certified. We should take into account that the quality of research should also be promoted for better progress in the field of spine surgery in Saudi Arabia.
The system of promotion for academic doctors in Saudi Arabia is defective in a way that it encourages quantity and single authorship rather than quality. Some of the senior doctors may not have had formal training in research methodology, so academic institutions are encouraged to offer faculty development courses that may improve this area. Systematic reviews of randomised trials should be supported and done by senior doctors since it is of high quality. It is important to note that to expedite the change in the quality of spine surgery research in Saudi Arabia, clinical trial units should be established in all major medical institutes in the country, coupled with rigorous ethical supervision, to ensure the production of high-quality research that applies to the population living in Saudi Arabia.
One major problem to be solved is English proficiency. The ability to communicate in formal written English could be one of the factors causing health-care professionals to be hesitant to do research in the first place. Hence, institutions should require English proficiency standardised tests for any new applicant, as well as provide English academic writing courses.
The limitations of our current study include several factors. First, the accuracy of citation rates imported from Google Scholar is a rough estimation without giving the frequency of citations and the citations' full details. This makes it hard to interpret the data only in light of these numbers. Full citation analysis is a must. Moreover, discussing the quality of publications and comparing it with publications from other developed countries are unfair. The comparison should have been done with countries having similar characteristics and with studies similar in methodology. Unfortunately, such studies are lacking.
| Conclusion|| |
The impact of spine surgery publications is modest, and the quality and frequency of publications have not changed considerably over the past 26 years. We encourage Saudi spine surgeons through the established Saudi Spine Society and Saudi Association of Neurological Surgeons to have institutional collaboration with their counterparts to produce high-quality research.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Dong F, Fan M, Jia Z. Fifty top-cited fracture articles from China: A systematic review and bibliometric analysis. J Orthop Surg Res 2016;11:71.
Jamjoom AA, Jamjoom BA, Jamjoom AB. The most cited Saudi neurosurgical publications. Neurosciences (Riyadh) 2013;18:83-6.
Jamjoom BA, Jamjoom AA, Jamjoom AB. Level of evidence of clinical neurosurgery research in Saudi Arabia. Neurosciences (Riyadh) 2014;19:334-7.
Murray MR, Wang T, Schroeder GD, Hsu WK. The 100 most cited spine articles. Eur Spine J 2012;21:2059-69.
Skovrlj B, Steinberger J, Guzman JZ, Overley SC, Qureshi SA, Caridi JM, et al.
The 100 most influential articles in cervical spine surgery. Global Spine J 2016;6:69-79.
Ponce FA, Lozano AM. Highly cited works in neurosurgery. Part I: The 100 top-cited papers in neurosurgical journals. J Neurosurg 2010;112:223-32.
Lefaivre KA, Shadgan B, O'Brien PJ. 100 most cited articles in orthopaedic surgery. Clin Orthop Relat Res 2011;469:1487-97.
Makhdom AM, Alqahtani SM, Alsheikh KA, Samargandi OA, Saran N. Level of evidence of clinical orthopedic surgery research in Saudi Arabia. Saudi Med J 2013;34:395-400.
Samargandi OA, Makhdom AM, Kaur M, Awan BA, Thoma A. Level of evidence of plastic surgery clinical research in Saudi Arabia. Saudi Med J 2013;34:1197-8.
Ponce FA, Lozano AM. Highly cited works in neurosurgery. Part II: The citation classics. J Neurosurg 2010;112:233-46.
Wupperman R, Davis R, Obremskey WT. Level of evidence in spine compared to other orthopedic journals. Spine (Phila Pa 1976) 2007;32:388-93.
Obremskey WT, Pappas N, Attallah-Wasif E, TornettaP3rd
, Bhandari M. Level of evidence in orthopaedic journals. J Bone Joint Surg Am 2005;87:2632-8.
Amiri AR, Kanesalingam K, Cro S, Casey AT. Level of evidence of clinical spinal research and its correlation with journal impact factor. Spine J 2013;13:1148-53.
Falavigna A, Botelho RV, Teles AR, da Silva PG, Defino HL. Assessing the evolution of publications by Brazilian spine surgeons in the last decade. Eur Spine J 2013;22:2084-8.
Baeesa SS, Maghrabi Y, Msaddi AK, Assaker R. Quality of spine surgery research from the Arab countries: A Systematic review and bibliometric analysis. Biomed Res Int 2017;2017:7560236.
Koutras C, Antoniou SA, Heep H. Geographic origin of publications in major spine journals. Acta Orthop Belg 2014;80:508-14.
Alfaleh K, Bajammal S. Enhancing biomedical research in Saudi Arabia. Saudi Med J 2005;26:897.
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5]
[Table 1], [Table 2], [Table 3], [Table 4]