• Users Online: 299
  • Home
  • Print this page
  • Email this page
Home About us Editorial board Search Ahead of print Current issue Archives Submit article Instructions Subscribe Contacts Login 


 
 Table of Contents  
ORIGINAL ARTICLE
Year : 2013  |  Volume : 1  |  Issue : 1  |  Page : 38-42

A study of resident's attitude, knowledge and barriers towards the use of evidence based medicine


1 Department of Health Information Technology (HIT), Faculty of Para-Medicine, Tabriz, Iran
2 Department of Medical Library and Information Science (DMLIS), Faculty of Para-Medicine, Tabriz; Iranian center for Evidence Based Medicine(ICEBM), Isfahan, Iran
3 Department of Physiotherapy, School of Rehabilitation, Tabriz University of Medical Sciences, Tabriz, Iran
4 Health Management and Ecinimics Research Center, Isfahan; Department of Health Information Technology (HIT), School of Health management and information Sciences, Isfahan University of Medical Sciences, Isfahan, Iran
5 , Iran
6 Department of Health Information Technology (HIT), School of Health management and information Sciences, Isfahan University of Medical Sciences, Isfahan, Iran
7 Chancellor for Treatment Affaires, Mazandaran University of Medical Sciences, Sari, Iran

Date of Web Publication30-Nov-2013

Correspondence Address:
Esmaeil Rezazadeh
Medical Records Education, Chancellor for Treatment Affaires, Mazandaran University of Medical Sciences, Sari
Iran
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/2347-9019.122456

Rights and Permissions
  Abstract 

Objectives: To survey the attitudes, awareness, and practice of evidence based medicine (EBM), and to determine the barriers that influence application of EBM in therapeutic process among clinical residents in Iran. Materials and Methods: A cross sectional survey was conducted during September to December 2012 at the teaching hospitals of Tehran University of Medical Sciences among 79 clinical residents from different medical specialties. A valid and reliable questionnaire consisted of five sections and 27 statements were used in this research. Most respondents (90%) completed the questionnaires voluntarily and anonymously. We inserted the data into Microsoft Excel 2007. Data were analyzed using Statistical Package for Social Sciences (SPSS) 16.0 software. We applied Spearman and Mann-Whitney test for correlation between variables. Results: Knowledge of residents about EBM is low and their attitude towards EBM was positive, but their knowledge and skills in regard with the evidence based medical information resources were mostly limited to Google Scholar. Lack of enough time to practice EBM and unperfect access to data bases were the main barriers. There was no significant correlation between residency grade and familiarity and use of electronic EBM resources (Spearman, P = 0.138). Conclusions: It is essential to improve their knowledge, skills, and capabilities to practice EBM. They can conduct EBM that should reduce medical errors and faulty diagnoses. Integration of training approaches like journal clubs or workshops with clinical practice is suggested.

Keywords: Attitude, clinical residents, decision making, evidence based medicine, questionnaire


How to cite this article:
Hachesu PR, Gavgani VZ, Salahzadeh Z, Ehteshami A, Piri Z, Kasaei M, Rezazadeh E. A study of resident's attitude, knowledge and barriers towards the use of evidence based medicine. Int J Health Syst Disaster Manage 2013;1:38-42

How to cite this URL:
Hachesu PR, Gavgani VZ, Salahzadeh Z, Ehteshami A, Piri Z, Kasaei M, Rezazadeh E. A study of resident's attitude, knowledge and barriers towards the use of evidence based medicine. Int J Health Syst Disaster Manage [serial online] 2013 [cited 2019 Mar 26];1:38-42. Available from: http://www.ijhsdm.org/text.asp?2013/1/1/38/122456


  Introduction Top


One of the most critical steps in medical practice is a medical diagnosis. When a physician takes some level of data from different sources (such as the patient's history, his\her own background knowledge, literature search, and tests) and convert these data into the information the medical diagnosis takes place. Therefore, without information the full benefit of clinical decisions and treatment of clinical case is unlikely to be realized. A medical therapeutic process conducted through data collection (observation), diagnostics, and therapy; [1] it starts with the patient history and proceeds with diagnostic procedures which are selected based on available information. Thus, an important question that must be answered is that, how it can be determined which data are relevant? In traditional medicine, the decision-making approach was relied more on physician's background knowledge, clinical experiences, consultation with seniors, and textbooks rather than reading current literature. However, now decision-making process in medical practice focus on evidence based practice and convergence of both individual clinical expertise and best available evidence. An evidence based approach (EBA), gathering, analyzing, and applying up-to-date evidence from the reliable literature and researches integrated into medical practice and patient's care. Evidence based medicine (EBM) which was defined by Sakett as "the conscientious, explicit, and judicious use of current best evidence in making decisions about patient care", [2] is a tool for proper and efficient use of the results of research works in decision making. The most essential components of EBA are to access, assess, and apply the evidence. To access the best available evidence it is essential to know how and where to search and retrieve the best evidence, which databases are rich and reliable, and how to conduct a systematic and purposeful search strategy? The lack of knowledge about the best available evidence is one of the main causes of applying various and controversial medical treatments. [3] EBM is conducted through guidelines, which include general description based on available evidence and experts' experiences. [4] Evidence based guidelines can be found from numerous sources, and the overall goal is to aggregate the best data and use the conclusions to help come to a clinical decision. In past years, attempts have focused on the use of computer guidelines to support the clinical care and resource management based on the updated medical information. [5],[6],[7],[8],[9] EMB provides a common framework for problem solving, improving process, communication, and understanding between people from different backgrounds, such as clinicians and patients. [10] The evidence based clinical guidelines can provide access to information at the point of care. [11],[12],[13]

Todays therapeutic procedures are not acceptable only based on specialists' information. [14] There is a large gap between the information published in evidence based clinical guidelines and the knowledge required to perform diagnostic and therapeutic process. An efficient way to fill this gap is to adhere medical informatics research. [15],[16] It is obvious that the physicians who have established their actions based on scientific evidence, could make better decisions. [17]

Therefore, it was essential to identify if the clinicians' medical practice and clinical decision-making are evidence based. What is the attitude of clinicians towards EBM? Moreover, do they have acquired proper knowledge about EBM practice? Are there any barriers for practice of EBM with physicians?

This study investigate the attitude and knowledge of clinical residents towards EBM and the barriers that they face in practice of EBM.


  Materials and Methods Top


A cross sectional survey was conducted during September to December 2012 at the teaching hospitals of Tehran University of Medical Sciences.

A total number of 79 clinical residents and postgraduate students of physiotherapy (MS and PhD students) which were attending clinical practice learning in the teaching hospital settings participated in this study. Participation in this study depended on the residents' willingness. The informed consent was taken from participants to ensure the confidentiality of collecting data.

Questionnaire

The questionnaire comprised of five sections and 27 statements. Section 1: Questions included demographics feature. Section 2: Covering attitudes and familiarity towards EBM. Section 3: Items about the EBM practice. Section 4: One item used on the most databases/resources applied and two open questions gave participants the opportunity to state their views of the major barriers and encouraging mechanisms. Section 5: Respondents that have applied EBM in clinical actions were asked to answer the other parts related to opinions and viewpoints and about the implication of EBM in the therapeutic tasks. In order to reduce the number of unanswered questionnaires, the authors personally distributed all the questionnaires. We categorized questions on a 1-4 or 1-3 scale based on Likert scale. Some questions were derived from valid and reliable questionnaire. [3],[18],[19],[20],[21],[22],[23],[24]

Analysis

Spearman and Mann Whitney test was used to analyze the statistical differences for correlation between variables. The statistic is performed utilizing a two-tailed test and significance level of 0.05. Answers were reviewed and categorized with Microsoft Office Excel and the data collected were analyzed using Statistical Package for Social Sciences (SPSS) software version 16.0 (SPSS Inc., Chicago, IL, USA).


  Results Top


The questionnaire was distributed to 87 residents, and the total number of respondents was 79 (response rate was 90%). Of the respondents, 28% were PhD students of physical therapy.

Demographics data of the respondents and their level of grades and specialties are given in [Table 1].
Table 1: Characteristics of respondents (N=79)

Click here to view


Knowledge and awareness of evidence based medicine

To determine the familiarity with EBM, we asked respondents to answer the three point scale questions that are presented in [Table 2].
Table 2: Self-reported attitudes and awareness of respondents toward EBM (N=79)

Click here to view


Out of total 79 respondents, the major number 32 (40%) that is less than half of the total respondents stated that they have moderate familiarity with EBM. Our findings showed that the knowledge about all parts of EBM was low in the respondents (39%), 18% of respondents had formal research experience, and 44% had prior EBM training in workshops or other places.

Findings analysis showed that 53%, of those residents who had a good familiarity with EBM approach could apply EBM in their therapeutic tasks.

Resources to locate evidence

The PubMed and Google Scholar were the most used sources of information for clinical decision making as shown in [Figure 1]. Pediatrics residents referred to EBM sites such as www.paediatricevidencebasedmedine.com which was up to date the most popular source used by internal residents.
Figure 1: Clinical resident's awareness of resources relevant to the practice of evidence based medicine

Click here to view


Attitude towards evidence based medicine

Respondents about their attitudes toward EBM were asked. Of 79 respondents, we exclude the 21 (27%) with a poor rating familiarity asserted in [Table 2]. We considered that residents knowledge is low about a specific EBM process in clinical decisions, but there was an overall positive attitude towards EBM. More resident and physical therapists believed that EBM should be used in diagnosis and treatment affairs because of its importance to reduce medical errors; increase the diagnostic accuracy, and to update medical information. However, in the case of time access to evidence and the time taken to search and find accurate and appropriate information, they have relatively negative attitudes [Table 3].
Table 3: Clinical resident's attitudes toward EBM (N=58)

Click here to view


Barriers to evidence based medicine

[Figure 2] shows the reasons that residents perceived to be barriers involved in EBM. The main barrier was the lack of enough time to practicing EBM and unperfect access to data bases. Lack of the time to practice EBM and the belief that EBM is not useful was the most and least important factors respectively. Lack of suitable and complete guidelines on diagnosis or treatment led to decrease of electronic resources on behalf of practitioners.
Figure 2: Main barriers to the use of EBM

Click here to view


We found that there was no significant correlation between residency grade and familiarity and use of electronic EBM resources (Spearman, P = 0.138). Same finding was found in others study. [23] This study also revealed that familiarity and use of electronic EBM resources did not differ between male and female residents (Mann-Whitney, P = 0.317).

Evidence based medicine application: Lessons learned

As are given in [Table 3], of the 58 completed questionnaires with positive response, most respondents (89%) agreed that EBM could improve and update decision-making. Comments on open questions showed that respondents (58%) would prefer to get answers through professors and colleagues' consultation rather than EBM sites during diagnosis or treatment process. According to participants' statement, accuracy in decision making, reduce of medical errors, and finding new drugs or diagnostic and therapeutic methods were the most important reasons to use EBM. Besides, encouragement mechanisms have not been adopted through healthcare or hospitals policy makers.


  Discussion Top


The attitudes, awareness, and barriers to EBM among clinical residents were surveyed. The finding of this study was consistent with other studies related to attitudes of physicians towards evidence-based medicine; for example, Amini found that 56% of the physician respondents described the attitude towards EBM in their institution as positive. [25] It was shown that residents are interested in increasing their skills and the amount of evidence used in their practices.

Catherine revealed that all professional groups welcomed and supported evidence based practice. [26] Mittal and Perakath stated that the attitudes of surgical trainees toward EBM were desired. [21] Mayer and Piterman showed that general practitioners (GPs) welcomed EBM and found that it could support their information needs. [27] Although the positive attitude towards EBM forms the cornerstone for its acceptance, it does not ensure EBM adoption and its widespread usage. Despite these positive findings, fewer residents actually reported practicing EBM. Barghouti et al., stated that Jordanian family physicians welcomed to learn and implement EBM in their daily practice, but there is knowledge gap and lack of awareness. [28] Similarly, Abeysena et al., revealed that knowledge and practices of EBM among Sri Lanka doctors were poor. However, attitudes toward EBM were relatively good. [20] This knowledge gap requires more guidance and training to ensure successful implementation and application of EBM. [28],[29] Another study found that students' knowledge, confidence in knowledge, familiarity with, and confidence in the use of EBP skills increased following education. Prior to the training session students earned a mean knowledge score of 50% which raised to 66%. [30]

Lack of time was one of the major barriers to EBM learning in our study. Other studies [19],[20],[21],[25],[31],[32] confirm this finding. Risahmawati et al., presented a lack of time to access EBM sources, a lack of sources in the native language, and insufficient skills as the main barriers to EBM adoption. [23]

Other commonly reported barriers to EBM included insufficient access to information and negative patients or associate's attitude towards EBM. Despite the increasing emphasis on the importance of EBM in clinical decision-making, some respondents would consider insignificant use of it. Some respondents (41%) believed that current clinical guidelines lack sufficient support for the practice of EBM and they do not apply in clinical decision-making.

We believed learning the use of information technology, methods of search and evaluation of evidence for the diagnosis, treatment, and medical education is necessary. Computerization of medical guidelines in comparison with traditional methods potential analysis led to advances in physician's decision-making. [33] Several official websites such as www.guidlines.gov for sharing medical knowledge among physicians has been created, many of these sites can create text with HTML and text format and useful links which is very useful and valuable. [34]

Our findings concerning knowledge of practicing EBM are conforming to results of a similar study by Ahmadi-Abhari et al., [35] They argued that the majority of trainee physicians at Tehran University hospitals lack adequate knowledge about basic concepts of EBM. Furthermore, most of them keep employing traditional sources of knowledge rather than evidence based sources.

All of residents had access to Medline and word web wide (www) in workplace, but their use is not desirable. In accordance with Moghadami and Amini, [29] few respondents thought that they were competent or skilled to critically evaluate clinical literatures. This is because of that comprehensive practical trainings are not given. Kouhpayehzadeh showed that an EBM workshop may improve clinical teachers' abilities and skills in using EBM. [36]

Although residents were familiar with common databases used in EBM, some database related to EBM such as Cochrane Database of Systematic Reviews were used poorly. In our study, similar to Risahmawati et al., [23] the most understood EBM resources were PubMed and Google Scholar. Few of them used the Cochrane Library. Similarly, only 8.6% of doctors in Sri Lanka [20] were aware of the use of Cochrane Library and in Iran it was the least used source. [35] Despite ease of access to electronic resources, residents' physicians do not use these sources appropriately. Findings show that respondents use both EBM and non-EBM methods such as referring to textbook and colleagues to find the answers during daily clinical problem solving. It is a necessary to increase knowledge among residents about these available resources and appropriate use.

The respondents in our study were not sure that EBM could take into account patient's personal preferences and improving of clinical process' accuracy. This finding is similar to other studies. [31],[37] It shoud be noted that EBM was seen as a necessary and important method, and research findings were useful in clinical practice and decision-making. Similar results have previously presented these results. [19],[31]


  Conclusion Top


This study advises that EBM and literature search skills should be inserted into the medical curriculum. Integration of teaching EBM process and skills in clinical practice with classroom or workshop lessons are an applicable solution. According to studies agreed that students who had completed the course about evidence based medicine concepts had better skills in applying evidence-based medicine to patient care than other students. [38] Physicians and other healthcare professionals can conduct EBM that should reduce medical errors and faulty diagnoses.


  Limitation Top


The questionnaire relied on the participants' self-rated assessment of their own knowledge and beliefs. Another limitation is that we conducted this research among a small portion of residents with few specialties. Hence, this study cannot be representative of all specialties and hospitals. Lack of sources in the native language (Persian) is a barrier to EBM application.


  Acknowledgments Top


The authors would like to thank the clinical residents and physiotherapists of the TUMS for their help and commitment during the completion of this research.

 
  References Top

1.Bemmel JH, Musen MA, Helder J. Handbook of medical informatics. Bohn Stafleu Van Loghum; Heidelberg, Germany: Springer Verlag; 1997.  Back to cited text no. 1
    
2.Komatsu RS. Evidence based medicine is the conscientious, explicit, and judicious use of current evidence in making decisions about the care of individual patients. Sao Paulo Med J 1996;114:1190-1.  Back to cited text no. 2
    
3.Sadeghi M, Khanjani N, Motamedi F, Saber M, Rad GS. Familiarity of medical residents at Kerman Medical University with evidence based medicine databases. J Res Med Sci 2011;16:1372-7.  Back to cited text no. 3
[PUBMED]  Medknow Journal  
4.Park M. Development of evidence based nursing practice guideline document model for electronic distribution. J Korean Soc Med Inform 2004;10:147-59.  Back to cited text no. 4
    
5.Panzarasa S, Maddè S, Quaglini S, Pistarini C, Stefanelli M. Evidence-based careflow management systems: The case of post-stroke rehabilitation. J Biomed Inform 2002;35:123-39.  Back to cited text no. 5
    
6.Lenz R, Reichert M. IT support for healthcare processes-premises, challenges, perspectives. Data Knowledge Engin 2007;61:39-58.  Back to cited text no. 6
    
7.Shiffman RN, Brandt CA, Liaw Y, Corb GJ. A design model for computer-based guideline implementation based on information management services. J Am Med Inform Assoc 1999;6:99-103.  Back to cited text no. 7
    
8.Rome ES, Ammerman S, Rosen DS, Keller RJ, Lock J, Mammel KA, et al. Children and adolescents with eating disorders: The state of the art. Pediatrics 2003;111:e98-108.  Back to cited text no. 8
    
9.Hope S, Javaid MK, Rance K. Secondary prevention of fracture: Time to work together. Pathways in Practice. Primary Care Woman's Health J 2011;3:33-5.  Back to cited text no. 9
    
10.Rosenberg W, Donald A. Evidence based medicine: An approach to clinical problem-solving. BMJ 1995;310:1122-6.  Back to cited text no. 10
    
11.Peleg M, Tu S, Bury J, Ciccarese P, Fox J, Greenes RA, et al. Comparing computer-interpretable guideline models: A case-study approach. J Am Med Inform Assoc 2003;10:52-68.  Back to cited text no. 11
    
12.de Clercq PA, Blom JA, Korsten HH, Hasman A. Approaches for creating computer-interpretable guidelines that facilitate decision support. Artif Intell Med 2004;31:1-27.  Back to cited text no. 12
    
13.Fischer S, Stewart TE, Mehta S, Wax R, Lapinsky SE. Handheld computing in medicine. J Am Med Inform Assoc 2003;10:139-49.  Back to cited text no. 13
    
14.Collins J. Evidence-based medicine. J Am Coll Radiol 2007;4:551-4.  Back to cited text no. 14
    
15.Reilly BM. The essence of EBM. BMJ 2004;329:991-2.  Back to cited text no. 15
    
16.Sackett DL, Rosenberg W, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: What it is and what it isn't. BMJ 1996;312:71-2.  Back to cited text no. 16
    
17.Habai S, Rezaei P, Tabaghi R. Enhancing information literacy as a base of developing evidence-based nursing. Health Inform Manage 2010;7:371-8.  Back to cited text no. 17
    
18.Oliveri RS, Gluud C, Wille‐Jørgensen PA. Hospital doctors' self‐rated skills in and use of evidence‐based medicine: A questionnaire survey. J Eval Clin Pract 2004;10:219-26.  Back to cited text no. 18
    
19.Heiwe S, Kajermo KN, Tyni-Lenné R, Guidetti S, Samuelsson M, Andersson IL, et al. Evidence-based practice: Attitudes, knowledge and behaviour among allied health care professionals. Int J Qual Health Care 2011;23:198-209.  Back to cited text no. 19
    
20.Abeysena C, Jayawardana P, Wickremasinghe R, Wickramasinghe U. Evidence-based medicine knowledge, attitudes, and practices among doctors in Sri Lanka. J Evid Based Med 2010;3:83-7.  Back to cited text no. 20
    
21.Mittal R, Perakath B. Evidence-based surgery: Knowledge, attitudes, and perceived barriers among surgical trainees. J Surg Educ 2010;67:278-82.  Back to cited text no. 21
    
22.Al-Baglalic N, Al-Almaie SM. Physician attitudes towards evidence-based medicine in eastern Saudi Arabia. Ann Saudi Med 2004;24:425-8.  Back to cited text no. 22
    
23.Risahmawati RR, Emura SS, Nishi TT, Koizumi SS. Japanese resident physicians' attitudes, knowledge, and perceived barriers on the practice of evidence based medicine: A survey. BMC Res Notes 2011;4:374.  Back to cited text no. 23
    
24.Hachesu PR, Ahmadi M, Rezapoor A, Salahzadeh Z, Sadughi F, Maroufi N. Clinical care improvement with use of health information technology focusing on evidence based medicine. Health Inform Res 2012;18:164-70.  Back to cited text no. 24
    
25.Amin Z, Marion Aw, Soo R, Ooi S, Sivaraman P, Fei YJ, et al. Attitudes, practice and educational preferences towards evidence-based medicine among physicians in a large teaching hospital. Med Educ Online 2009;12.  Back to cited text no. 25
    
26.O'Donnell CA. Attitudes and knowledge of primary care professionals towards evidence‐based practice: A postal survey. J Eval Clin Pract 2004;10:197-205.  Back to cited text no. 26
    
27.Mayer J, Piterman L. The attitudes of Australian GPs to evidence-based medicine: A focus group study. Fam Pract 1999;16:627-32.  Back to cited text no. 27
    
28.Barghouti F, Halaseh L, Said T, Mousa AH, Dabdoub A. Evidence-based medicine among Jordanian family physicians. Can Fam Physician 2009;55:e6-13.  Back to cited text no. 28
    
29.Moghadami M, Amini M. Internal medicine residents' views and understanding of of evidence based medicine in Shiraz Medical School. J Med Educ 2009;8.  Back to cited text no. 29
    
30.Manspeaker SA, Van Lunen BL, Turocy PS, Pribesh S, Hankemeier D. Student knowledge, attitudes, and use of evidence-based concepts following an educational intervention. J Athl Train Educ 2011;6:11.  Back to cited text no. 30
    
31.Jette DU, Bacon K, Batty C, Carlson M, Ferland A, Hemingway RD, et al. Evidence-based practice: Beliefs, attitudes, knowledge, and behaviors of physical therapists. Phys Ther 2003;83:786-805.  Back to cited text no. 31
    
32.Byham-Gray LD, Gilbride JA, Dixon LB, Stage FK. Evidence-based practice: What are dietitians' perceptions, attitudes, and knowledge? J Am Diet Assoc 2005;105:1574-81.  Back to cited text no. 32
    
33.Rezaei P, Habibi S, Fozonkhah S. Information technology, an effective tool in reduccing and preventing medical error. Health Inform Manage 2007;4:90-7.  Back to cited text no. 33
    
34.Smith RG, Farquhar A. The road ahead for knowledge management: An AI perspective. AI Magazine 2000;21:17.  Back to cited text no. 34
    
35.Ahmadi-Abhari S, Soltani A, Hosseinpanah F. Knowledge and attitudes of trainee physicians regarding evidence‐based medicine: A questionnaire survey in Tehran, Iran J Eval Clin Prac 2008;14:775-9.  Back to cited text no. 35
    
36.Kouhpayehzadeh J, Baradaran H, Arabshahi KS, Knill‐Jones R. Clinical teachers' attitudes toward the efficacy of evidence‐based medicine workshop and self‐reported ability in evidence‐based practice in Iran. J Contin Educ Health Prof 2006;26:210-4.  Back to cited text no. 36
    
37.Retsas A. Barriers to using research evidence in nursing practice. J Adv Nurs 2000;31:599-606.  Back to cited text no. 37
    
38.Bookstaver PB, Rudisill CN, Bickley AR, McAbee C, Miller AD, Piro CC, et al. An evidence-based medicine elective course to improve student performance in advanced pharmacy practice experiences. Am J Pharm Educ 2011;75:9.  Back to cited text no. 38
    


    Figures

  [Figure 1], [Figure 2]
 
 
    Tables

  [Table 1], [Table 2], [Table 3]


This article has been cited by
1 Knowledge, attitudes and practice of physicians toward evidence-based medicine: A systematic review
Farzaneh Barzkar,Hamid Reza Baradaran,Jalil Koohpayehzadeh
Journal of Evidence-Based Medicine. 2018;
[Pubmed] | [DOI]



 

Top
 
 
  Search
 
Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
Access Statistics
Email Alert *
Add to My List *
* Registration required (free)

 
  In this article
Abstract
Introduction
Materials and Me...
Results
Discussion
Conclusion
Limitation
Acknowledgments
References
Article Figures
Article Tables

 Article Access Statistics
    Viewed2888    
    Printed56    
    Emailed0    
    PDF Downloaded272    
    Comments [Add]    
    Cited by others 1    

Recommend this journal


[TAG2]
[TAG3]
[TAG4]