Lyubov Tmanova
Darnall Medical Library
Walter Reed National Military Medical Center
Bethesda, MD

Abstract

Objective: The study objective was to evaluate research skills and the impact of educational instruction on enhancing these skills among first-year graduate medical education (GME) trainees. Another objective was to design the research course focusing on developing essential research skills for conducting research projects and publishing findings in scholarly journals.

Methods: The Research Information Skills Proficiency Scale (RISPS) was designed and used in this study to assess both perceptive and objective research information skills in GME trainees. Descriptive statistics were used to analyze the pre-test and post-test data collected after two 45-minute lectures on information and literature literacy. Additionally, trainees’ learning methods and approaches were assessed using a learning assessment and the clinical research appraisal inventory to evaluate their learning and skills perception.

Results: The RISPS scale was beneficial in assessing trainees’ skills and influenced the design of an elective course for the radiology curriculum. Demographic and educational factors, like research experience and major, have a significant impact on radiology residents’ skills, perceptions, and research abilities. Radiology residents have varying levels of understanding of study conceptualization, design, and collaborative research skills.

Conclusions: The study’s design of the RISPS scale enables the assessment of GME trainees’ perceived and objective research information skills. Implementing the RISPS perceived and objective scale each year for a new cohort of GME trainees equips GME program directors with essential educational data to tailor instructional designs according to the cohort’s learning proficiency. Furthermore, RISPS and the clinical research appraisal inventory (CRAI) are crucial in assessing research information skills and integrating trainees’ learning styles and preferences for research efficiency. The GME faculty can use the data from these questionnaires to guide the design of learning interventions aimed at enhancing the research skills and competencies of GME trainees. The course developed in the study can be added as an elective to the GME radiology curriculum to improve research skills and meet the scholarly activity requirements for the Accreditation Council for Graduate Medical Education (ACGME).

Introduction

The GME program’s educational curriculum is critical to the development of subject knowledge and skills required for graduates’ clinical career and practice. Researchers have found that the radiation oncology curriculum prioritizes theoretical and specialty-focused disciplines over research-oriented instruction ^1-8. Furthermore, they noticed the influence of the trainee’s knowledge on the research process, on the conduct of research, development, design, and allocation of research time on the quality of research and scholarly productivity. ^9-11 Emerging 21st-century medical practices influence the need for designing and implementing a research curriculum, as well as fostering research interest among GME radiology trainees. ^8,12-14 Integrating research-centered instruction into the curriculum could improve the abilities of GME trainees to design research studies and advance radiology research and scholarship.¹² ¹⁴

Nevertheless, excellence in radiology practice and research depends on the quality of post-graduate education, curriculum, independent learning, and academic skills (e.g., research literacy, information literacy, analytics, scholarly writing, and communication). Evaluating trainees’ learning skills during the first semester of trainees’ residency or fellowship is beneficial for designing research-centered instruction and educational processes focused on enhancing research skills and competencies. Similarly, trainees’ assessment of their research skills and competencies could be associated with their research self-efficacy and learning styles, allowing reflection on their learning progress, attainment, and improvement as well as their research skills and competencies.

Researchers have studied how graduate students perceive their abilities in effectively navigating academic literature and research using the Perception of Information Literacy Scale. ¹⁵ Similarly, Gowri and Padma, ¹⁶ using the Seven Pillars of Information Literacy: Core Model (SCONUL), studied undergraduate engineering students’ information literacy competencies. However, the research on evaluation of GME trainees’ information and research literacy in radiology programs is scarce. Also, the effect of GME trainees’ skills and competencies on their clinical, practical, and research training as well as the design of curriculum and instruction in radiology programs requires further understanding.

Assessment of clinical and research proficiency and learning styles during Post-Graduate Year One (PGY-1) is valuable for radiology professors as they inform the design of research lectures and the implementation of teaching methods aimed at enhancing research skills and competencies. Also, the assessment of research information proficiency in trainees can help program directors and faculty gain insights into trainees’ learning needs. This understanding can inform the design of instruction and lectures whilst including trainees’ learning preferences, styles, and pedagogical methods to enhance trainees’ learning. Furthermore, selecting educational resources and implementing research lectures based on trainees’ aptitudes to promote learning, information, and research literacy could improve radiology trainees’ information and research literacy.

The objectives of this study were: 1) design the research information proficiency scale (perceived and objective) to measure GME trainees information and research literacy skills, 2) evaluate trainees’ perceptions of their own research proficiency, clinical skills, and learning preferences in first-year radiology trainees, and 3) design research-oriented instruction centered on the acquisition of research skills required to carry out research projects and communicate research findings in radiation oncology for radiation oncology residents and fellows.

Methods

An observational study approach allows researchers to analyze and draw inferences from a sample population. ^17-19 This study was approved by the Walter Reed National Military Medical Center Institutional Review Board. This research studied the learners’ perception about their learning efficiency, preferences, clinical, and research skills in a cohort of ten first year radiology trainees. Perceived research information and research literacy was determined by designing and administering the Research Information Skills Proficiency Scales (RPIPSs) (both perceived and objective). Inferences were drawn from study questionnaires obtained from the cohort. The quantitative analysis was carried out using descriptive statistics, with educational and research variables. ²⁰ The qualitative analysis was also conducted to analyze the responses obtained from the Research Information Skills Proficiency Scale (objective) questionnaires. The research objective was to identify the effect of educational interventions on perceived skills in conducting research and clinical practice, as well as to understand trainees’ perceptions about their skills and learning preferences.

Population. Ten GME radiology trainees participated in study. Due to a limited enrollment in GME in Diagnostic Radiology, the study sample size was limited the number of students enrolled in the program.

Table 1. GME Trainees Demographics, Educational, and Research Experiences
Education, Research, and Information Support

Educational Track	Residents
Highest Degree Achieved	MD/PhD, MS/PhD, BS/DO, MD, BS
Education Major	Biomedical Engineering, Biology, Life Science, Biochemistry, Cell Molecular Biology
Age	25-45
Years of Medical Practice	2-4
Research Experience	50% with experience, 50% with no experience
Publication Experience	30% with publication experience
Library Visits	Infrequent (e.g., once a month, two-three weeks, never)
Librarian Assistance in Research projects	50% requests
Research Interest	Varied (e.g., little, minimal, moderate, no interest)

Instruments. Research Self-Efficiency Scale by Holden et al. was used to measure research efficiency in radiology trainees (Table 1, 2). ²¹ Learning Preferences Inventory by Rezler was used to assess radiology trainees learning preferences (Table 1, 2). ²² Clinical Research Appraisal Inventory by Mullikin et al. was used to evaluate radiology trainees clinical and research skills (Table 1, 2). ²³

The Perceived Research Information Skills Proficiency Scale (PRISPS) and Objective Research Information Skill Proficiency Scale (ORISPS) were designed in this study to understand information and research literacy skills in radiology trainees (Table 1, 2). The PRISPS comprised of 37 questions and used a Likert scale to measure perceived skills in identifying a research problem and developing a research question, conducting a literature review, research data management, and scientific writing and scholarly communication. ORISPS comprised of 37 questions and used a Likert scale to test research information skills objectively. Both scales included demographic questionnaires. The PRISPS and ORISPS were based on Association of College and Research Libraries Standards for Libraries in Higher Education. ²⁴ The study questions were validated through a pilot testing with clinicians (GME trainees, faculty) and library professionals (librarians, library staff). After the pilot testing, discrepancies were addressed. Then, PRISPS and ORISPS were administered to study participants (Table 1, 2).

Table 2.  Study Instruments and Questionnaires

Pre-Assessment	Post-Assessment
Research Self-Efficiency Scale	Research Information Skills Proficiency Scale (Perceived)
Clinical Research Appraisal Inventory	Research Information Skills Proficiency Scale (Objective)
Learning Preferences Inventory
Research Information Skills Proficiency Scale (Perceived)
Research Information Skills Proficiency Scale (Objective)

Research-Oriented Intervention. Two 45 minute lectures were given (Conducting Research: Research Process, Methods, and Design; and Scientific Communication) and assessments were used to study the effect of information lectures on the improvement of information skills in GME trainees.

The Conducting Research: Research Process, Methods, and Design lecture introduced basic concepts: 1) research principles and ethics, 2) the research process, 3) literature review planning, designing, and conducting, 4) research information resources, 5) literature search design strategy, 6) literature analysis, synthesis, and evaluation, 7) information and citation management, 7) principle of clinical and research question development and evaluation, 8) hypothesis development and in literature context, 9) research types (qualitative, quantitative, mixed) and study designs (cohort-observational, experimental, synthesis), 10) clinical research data ecosystem, data management cycle and plan, 11) intellectual property and copyright, 12) research data collection principles, 13) data analysis tools, 14) research management principles (academic leadership, funding, grants, collaboration, teaching, and project management principles).

The Scientific Communication lecture aimed to review elements of scientific communication and presentation. The lecture topics were 1) the design of scientific presentations, style, and methods, 2) elements of scientific communication (content, methods, tools), 3) the poster presentation, 4) the power point presentation, 5) research articles, 6) scientific writing, methods, and tools, 7) academic publication, guidelines, types, 8) authorship, and 9) the peer-review process.

Data Collection & Analysis. Data was collected from participants using study questionnaires (Table 1). The study questionnaires (in print) were distributed to study participants. The research self-efficiency, clinical research skills, and learning preferences questionnaires were distributed one day before the first lecture. The PRISPS and ORISPS were distributed to students one day before and after the two lectures. All questionnaires were collected one day after they were distributed.

Responses obtained from study participants were confidential, anonymized, and stored on password protected facility computers. Study results were not used for grading residents’ performance in the radiology program. Obtained results during the study did not affect rights, privacy, and confidentiality of residents and ACGME resident performance.

Descriptive statistics were used to analyze study participants’ responses obtained from pre-and-post questionnaires to determine perceived information and research skills. The study demographic variable included the education level, major, years of clinical practice, and educational rank. The study research variable consisted of a prior research experience, scholarly publication, and publication type. In this study we used the measure of frequency, central tendency, and variation to evaluate trainees’ responses of perceived research skills obtained from pre- and post-assessment questionaries, research efficacy, learning styles, and clinical skills. Obtained data was analyzed to determine the correlation between the self-efficacy and post-intervention learning performance in diagnostic radiology trainees.

Stem and leaf plot was used to evaluate GME trainees’ perception of research confidence frequency distribution among studied groups (conceptualization of study; designing a study; collaborating with others; funding a study; planning and managing a research study; protecting research subjects and responsible conduct or research; collecting, recording, analyzing data, interpreting data; reporting a study and presenting a study). SPSS software was used to analyze trainees’ responses obtained from administered questionnaires.

Results

Clinical and Research Effectiveness Perception. Results of the CRAI assessment indicated the dispersal of GME of trainee’s perception regarding their research confidence, especially in tasks associated with study conceptualization (S.D. 16.1), designing a study (S.D. 17.8), collaborating with others (15.1), and funding a study (S.D. 10.1) (Fig. 1).

Figure 1. GME trainees’ perception on clinical and research abilities in conducting research. Groups: CS-conceptualization of study, DS – designing a study, CO-collaborating with others, FS-funding a study, PM-planning and managing your research study, PR-protecting research subject and responsible conduct or research, CR-collecting, recording, analyzing data, interpreting data, RS-reporting study and presenting study. Stem and leaf plot: 0-125.

Learning Preferences. Teamwork, self-evaluation, and theoretical preferences are less preferred among GME trainees. Learning by reading, facts, and directed by teachers were more desirable by trainees.

Table 3. GME Trainees Learning Preferences

Learning Preference	Mean	Standard Deviation
Factual	4.4	.726
Teacher-directed	3.89	.782
Teamwork	2.78	1.202
Reading	3.89	.782
Self-evaluation	2.89	.928
Theoretical	3.11	1.167

Information Research Proficiency Instrument Design. The Research Information Skills Proficiency Scale was designed in this study to evaluate research information skills in radiology residents. This scale consisted of two sets of questionnaires to measure perception and actual research information skills in trainees. The Research Information Skills Proficiency Scale (perception) consisted of 37 questions which measured trainees’ skills in following: 1) identifying a research problem and developing a research question, 2) conducting a literature review, 3) research data management, and 4) scientific writing and scholarly communication (Table S1). Demographic information, education, and research expertise was assessed by education level, years of practice, and prior research experience. The Research Information Skills Proficiency Scale (objective) consisted of 33 questions measured trainees’ skills in following: 1) understanding research, research process, and ethics, 2) writing grant application, 3) identifying research gaps and problem, 4) planning an effective exploration of literature and search design, and identifying relevant information resources, 4) research data cycle, management, and plan design, 5) authorship and publication. Information about prior experience of librarian assistance have been inquired additionally to prior demographic, education, and research expertise (Table S2). ORISPS and PRISPS were administered before and after two lectures to assess trainees’ perceptions about their information literacy proficiency. Obtained data suggested an advanced level of information and research experience in residents.

Table 4. GME trainees perceived and objective perceptions of their information literacy skills using PRISPS and ORISPS.

	Mean	S.D.	Correlation	Significance
Pre-test	193.1428	24.56
Post-test	199.5714	30.52
Pre-test and Post-test			-.716	.070

Educational Intervention. The design of an elective course, Introduction to Research and Scholarly Communication, is described in this study. This course offers theoretical and foundational knowledge of designing research while exploring published literature and developing research skills and competencies. The course consists of five modules: research and scientific method, literature review and search design, information management, academic and medical writing, and academic ethics and publication. Course activities such as quizzes, discussions, assignments, and projects were used to enhance students’ learning. The course was conducted in Canvas and the course syllabus was distributed to a liaison GME program (S3).

The Radiology and Radiology Sciences GME programs serve as the foundation for residents and fellows’ scholarly success. Through teaching, learning, and mentorship from experienced faculty, residents and fellows can apply their knowledge in real-world clinical settings. Prior research experience, years of clinical practice, and education influence radiology trainees’ research self-efficacy. Prior research and publication experiences of half of study participants contributed to their research confidence and efficacy (Fig 1). There was no significant correlation between learning intervention and trainees’ perceptions about their research skills (Table 4). However, the research information proficiency scales designed in this study allows for the identification of gaps in students’ information skills pertaining to study conceptualization and conducting literature reviews, with less research and publication experience in a cohort with doctoral degrees. Furthermore, 45-minute lectures were oriented towards students with a moderate level of skills and concepts by integrating thinking skills. These lectures also aimed to recall prior knowledge about studied concepts.

This study used PRIPS and ORIPS to assess how prior information skills, subject knowledge, and research skills were applied to the identification of research gaps, the design of studies, and application of these skills by post-graduate trainees. Furthermore, ORIPS evaluated the application of the principles of literature review and research analysis. This evaluation enabled the assessment of skills for the synthesis and evaluation of published literature, facilitating its effective utilization throughout the research process. The ORIPS questionnaires were designed to enable GME trainees to assess and improve their information skills by applying their prior knowledge, skills, and information acquired during the educational intervention.

Although previous studies explored information literacy assessments, ^{15, 16} information literacy for clinical research practitioners is lacking in the scholarly literature. This research explored trainees’ perceptions and objective research skills, specifically evaluating trainees’ skills in problem identification and question development, understanding the literature review beyond databases searching to include the analysis, synthesis, and evaluation of information resources. Moreover, PRISPS offers a broader assessment of trainees’ skills in research project design, data management, literature review, and the intricate process of scholarly writing and publication. The objective scale, ORISPS, allows trainees to evaluate their skill perceptions and identify an improvement area. Furthermore, PRISPS and ORISPS are differentiated from other scales due to their ability to evaluate trainees’ perceived and objective research skills for the development of GME scholarly projects. Though CRAI has been used for clinicians to assess the self-efficacy in conducting clinical research, this instrument lacks the capacity to evaluate skills from an information science-specific perspective. Additionally, the instruments in this study were designed to evaluate postgraduate trainees’ perceived skills, identify their learning gaps, design appropriate course, and implement effective teaching methods to enhance their skills. Teachers can use ORISPS to understand trainees’ advancement, identify lacking skills and knowledge, and implement instruction to mitigate missing skills.
GME trainees with research and educational backgrounds are prepared to gain advanced information, research skills, and competencies, as well as start their research during the first year of residency. Screening PGY-1 trainees results in designing targeted information and research support. Furthermore, the results of PRISPS and ORISP equip instructors with insights to design lectures that improve and cultivate new information and research skills and competencies. Building on the results of these assessments, the elective course, Introduction to Research and Scholarly Communication, was designed to enhance trainees’ research and scholarly communication skills. The course is based on information and research theories, focusing on advancing skills through the application of acquired knowledge to meet ACGME graduation requirements.

Although the GME program focuses on clinical competencies and faculty support, there is a noticeable absence of courses dedicated to scholarly activities. The course developed in this study can be integrated as an elective within the radiology curriculum. Given the variability in research time among GME programs, the elective course presents a valuable opportunity for trainees to hone their research skills and competencies.
Proficiency in research skills, including comprehension of research projects, conceptual reasoning, data interpretation, and research ethics, are fundamental components of clinician competencies. By integrating research information expertise, the scholar-practitioner model guides trainees through a transformative process, enhancing their preparedness for engaging in research activities. It is strongly argued that acquiring information and research literacy skills through elective courses empowers trainees to augment their existing knowledge and skills, which can be directly applied to their graduate program studies and clinical responsibilities. Additionally, trainees can refine their research literacy skills through critical thinking, scholarly writing, research conceptualization, data analysis, synthesis, and evaluation of research outcomes. They can also apply their acquired knowledge and skills to publish literature reviews and develop research protocols, studies, or projects.

Knowledge gap assessment, research problem identification, and literature review analysis and synthesis are research milestones post-graduate trainees are experiencing when meeting GME program research requirements. The elective course offers both theoretical and practical guidance in acquiring research literacy skills and competencies. PGY-1, PGY-3, or PGY-4 trainees can enhance their skills by exploring the research ecosystem, discovering, and integrating information to develop research studies, and fulfilling scholar or clinical research prerequisites. The annual screening of GME trainees’ information and research proficiency allows faculty to design courses based on their skills and competencies.

The GME program also provides opportunities for research and scholarly activities, allowing trainees to contribute to the advancement of medical knowledge. Radiology residents need to learn research skills to meet the scholarly requirements of the GME program and advance their careers as radiology physicians or researchers. Therefore, educational curriculum, centered on multidisciplinary and interdisciplinary training along with research-centered instruction is vital for the development of clinical and research skills in radiology residents in meeting GME program scholarly and clinical requirements as well as for their professional endeavors as clinicians and scholars.

Furthermore, the application of research and literature in clinical training and education in GME programs contributes to the development of scholar practitioner qualities as thinker, learner, communicator, and leader in future radiology physicians.^25,26

The limitation of this study is the small sample size due to a low GME residency enrollment on average of 4-5 trainees. Additionally, the data obtained from ORISPS is scarce due to low response representing a small population that provided answers. The findings of this study can’t be generalized due to demographic and educational variables. The study cohort consisted of almost 50% of trainees having graduate degrees, research, and publication experience. Furthermore, trainees educational and research proficiencies might vary during every year of enrollment. However, implementation of PRISPS and ORISPS to screen trainees’ information and research literacy is valuable for instructional design of curriculum to enhance trainees’ research skills and competencies.
This study used 45 min of instruction to investigate a methodological approach for teaching research skills. The content conveyed in 45 min was based on expected trainees’ prior knowledge and experience gained during their undergraduate education and medical education and was designed to explore the pedagogical approach in teaching research skills. The typical format of library instruction, which consists of a single session, allowed to teach the entire lecture in 45 min. However, this study emphasizes that a 45-minute session is not adequate to assist GME trainees in developing research skills they need for their GME projects, especially in understanding research process. Independent learning, collaborative learning, project-based learning for postgraduate medical trainees highlights an approach for the course structure designed in this study (S3. Course syllabus). This research suggests implementing a course structured around five modules, focusing on students’ learning approach. The findings of this study indicate that the 5-model course, centered on foundational concepts of designing, conducting, and publishing scholarly research, can mitigate the deficiency in research skills and competencies, transforming traditional library instruction into an academic course emphasizing the research and scholarly communication. Furthermore, this research suggests for the integration of this course into GME didactics to enhance research skills of scholar-practitioners, particularly withing fellowship programs.

The strategy involves collaborating with faculty to promote research readiness in GME programs, specifically through the implementation of elective courses in GME radiology didactics. Furthermore, collaboration with program faculty strengthens the trainees’ research competencies through educational research. Research librarians, as future leaders, identify problems, gather information on best methods and practices, evaluate research evidence, and contribute to decision-making in higher education based on education. Research librarians significantly contribute to enhancing the quality of GME programs while collaborating in educational research. Their proficiency in information literacy and research methods enables trainees to evaluate evidence critically and make informed decisions in their scholarly pursuits.

References

1. Baumann M, Leer JWH, Dahl O, et al. Updated European core curriculum for radiotherapists (radiation oncologists). Recommended curriculum for the specialist training of medical practitioners in radiotherapy (radiation oncology) within Europe. Radiotherapy and Oncology. Feb 2004;70(2):107-113. doi:10.1016/j.radonc.2003.12.004

2. Coleman CN, Stone HB, Alexander GA, et al. Education and training for radiation scientists: Radiation Research Program and American Society of Therapeutic Radiology and Oncology Workshop, Bethesda, Maryland, May 12-14, 2003. Radiation Research. December 2003;160(6):729-737. doi:10.1667/rr3096

3. Dynlacht JR, Dewhirst MW, Hall EJ, Rosenstein BS, Zeman EM. Toward a consensus on radiobiology teaching to radiation oncology residents. Radiat Res. May 2002;157(5):599-606.

4. Dynlacht JR, Zeman EM. Recent initiatives for radiation oncology resident education in radiation and cancer biology. Radiat Res. Aug 2007;168(2):262-5.

5. Eriksen JG, Beavis AW, Coffey MA, et al. The updated ESTRO core curricula 2011 for clinicians, medical physicists and RTTs in radiotherapy/radiation oncology. Radiotherapy and Oncology. Apr 2012;103(1):103-108. doi:10.1016/j.radonc.2012.02.007

6. Klein EE, Gerbi BJ, Price RA, Jr., et al. ASTRO’s 2007 core physics curriculum for radiation oncology residents. Int J Radiat Oncol Biol Phys. Aug 1 2007;68(5):1276-88. doi:10.1016/j.ijrobp.2007.01.068

7. Mattes MD, Kharofa J, Zeidan YH, et al. Results of the 2012-2013 Association of Residents in Radiation Oncology (ARRO) Job Search and Career Planning Survey of Graduating Residents in the United States. International Journal of Radiation Oncology Biology Physics. Jan 1 2014;88(1):25-32. doi:10.1016/j.ijrobp.2013.09.033

8. Turner S, Seel M, Berry M. Radiation Oncology Training Program Curriculum developments in Australia and New Zealand: Design, implementation and evaluation – What next? Journal of Medical Imaging and Radiation Oncology. Dec 2015;59(6):728-735. doi:10.1111/1754-9485.12337

9. Aggarwal S, Kusano AS, Carter JN, Gable L, Thomas CR, Jr., Chang DT. Stress and Burnout Among Residency Program Directors in United States Radiation Oncology Programs. Int J Radiat Oncol Biol Phys. Nov 15 2015;93(4):746-53. doi:10.1016/j.ijrobp.2015.08.019

10. Morgan PB, Sopka DM, Kathpal M, Haynes JC, Lally BE, Li L. First author research productivity of United States radiation oncology residents: 2002-2007. Int J Radiat Oncol Biol Phys. Aug 1 2009;74(5):1567-72. doi:10.1016/j.ijrobp.2008.10.022

11. Shakespeare TP, Back MF, Lu JJ, Wynne CJ, Bloomfield L. Design of an internationally accredited radiation oncology resident training program incorporating novel educational models. International Journal of Radiation Oncology Biology Physics. Jul 15 2004;59(4):1157-1162. doi:10.1016/j.ijrobp.2003.12.009

12. Poetter R, Eriksen JG, Beavis AW, et al. Competencies in radiation oncology: A new approach for education and training of professionals for Radiotherapy and Oncology in Europe. Radiotherapy and Oncology. Apr 2012;103(1):1-4. doi:10.1016/j.radonc.2012.03.006

13. Thiruthaneeswaran N, Turner S, Milross C, Gogna K. Promoting a Research Culture among Junior Radiation Oncologists: Outcomes from the Introduction of the Australian and New Zealand Research Requirement in Training. Clinical Oncology. Mar 2014;26(3):162-173. doi:10.1016/j.clon.2013.12.007

14. Wallner PE, Shrieve DC, Kachnic LA, Wilson LD. Raising the Next Generation of Physician-Scientists: The Chairs’ Perspective. In Regard to Formenti et al. Int J Radiat Oncol Biol Phys. Nov 1 2015;93(3):727-8. doi:10.1016/j.ijrobp.2015.06.032

15. Doyle M, Foster B, Yukhymenko-Lescroart MA. Initial Development of the Perception of Information Literacy Scale (PILS). Communications in Information Literacy. 2019;13(2):205-227.

16. Gowri P, Padma P. Information Literacy Skills of Female Engineering Graduates: A Spotlight Study. Journal of Advances in Library and Information Science. 2018;7(3):232-245.

17. Miroshnychenko A, Zeraatkar D, Phillips MR, et al. Cohort studies investigating the effects of exposures: key principles that impact the credibility of the results. Eye (Lond). May 2022;36(5):905-906. doi:10.1038/s41433-021-01897-0

18. Song JW, Chung KC. Observational studies: cohort and case-control studies. Plast Reconstr Surg. Dec 2010;126(6):2234-2242. doi:10.1097/PRS.0b013e3181f44abc

19. Setia MS. Methodology Series Module 1: Cohort Studies. Indian J Dermatol. Jan-Feb 2016;61(1):21-5. doi:10.4103/0019-5154.174011

20. Mishra P, Pandey CM, Singh U, Gupta A, Sahu C, Keshri A. Descriptive statistics and normality tests for statistical data. Ann Card Anaesth. Jan-Mar 2019;22(1):67-72. doi:10.4103/aca.ACA_157_18

21. Holden G, Barker K, Meenaghan T, Rosenberg G. Research Self-Efficacy. Journal of Social Work Education. 1999/10/01 1999;35(3):463-476. doi:10.1080/10437797.1999.10778982

22. Rezler AG, Rezmovic V. The Learning Preference Inventory. Journal of allied health. Feb 1981;10(1):28-34.

23. Mullikin EA, Bakken LL, Betz NE. Assessing research self-efficacy in physician-scientists: the clinical research appraisal inventory. Journal of Career Assessment. 2007;15(3):367-387.

24. Iannuzzi PA, Abbott T, Brown J, et al. Standards for Libraries in Higher Education. Association of College & Research Libraries. 2011.

25. Brookfield SD, DiStefano A, Rudestam KE, Silverman R, Taira S. Encyclopedia of distributed learning. Los Angeles, London, New Delhi, Singapore, Washington DC: SAGE Publications, Inc. 2004.

26. Dean K. Part II: The scholar practitioner. What defines a scholar practitioner? A way of thinking and doing. ACPA Developments. 2010.

Supplements

S.1. PRISPS.

Research Information Topics & Questions

Identifying a Research Problem and Developing a Research Question

When given a research project for a presentation, a research paper or a residency graduation project, I feel confident in:

• determining what topic I need to research and find knowledge gaps in the literature, biomedical science, and practice.
• developing a research question while identifying a contribution to the subject area and research topic or problem.
• developing a research question while identifying knowledge gaps within the discipline.
• developing a research question(s) using the FINER criteria (Feasible, Interesting, Novel, Ethical, Relevant).
• developing a research question to test the research hypothesis.

Conducting a Literature Review

When given a research project for a presentation, a research paper or residency graduation project, I feel confident in:

• developing, planning, and managing literature review and analysis
• having skills to use research information and resources to effectively design my research project
• identifying and selecting information resources that are most useful for efficient and effective literature analysis
• developing effective and efficient search strategy methods to identify a relevant information resource
• conducing literature search in electronic databases: PubMed, EMBASE, Web of Science, The Cochrane Library, OVID MEDLINE, and others
• conducting information search by using NCBI molecular databases and tools (MedGen, GTR, Gene, ClinVar, OMIM, BioSystems, PubChem BioAssay, PubChem Substance, and others)
• gathering and organizing information resources from various literature databases by using citation management tools such EndNote, Zotero
• having skills to synthesize main ideas, critically evaluate the extracted material, and synthesize idea into new concepts
• having information evaluation skills to access these resources and identify bias, prejudice, deception, manipulation, and understand the impact of these factors in the selected resources
• having the ability to evaluate prior and current information and detect contradictions, values, and the uniqueness of the information
• determining the impact of new knowledge, identifying differences between priori and current knowledge, and assess various viewpoints
• having skills in the analyzing, appraising, and synthesis of biomedical literature
• citing resources by applying appropriate citation styles
• utilizing the knowledge of legal and ethical issues while planning the research study when appropriate

Research Data Management

When given a research project for a presentation, a research paper or residency graduation project, I feel confident in:

• planning and writing the research data management plan for the study
• employing the best practices of data management during the experimental and analytical phases of the project
• having knowledge of intellectual property and copyright
• having knowledge of data sharing guidelines and policy
• having knowledge of the security, privacy, and confidentiality regulations and policies for experimental research data
• having knowledge and skills in analysis of the experimental data using statistical analytical methods and tools such as: SPSS, SAS, Minitab, Statistica
• having skills using the following databases and programming software: MS Excel, MS Access, R, Python
• having skills in analyzing experimental data using visualization methods

Scientific Writing and Scholarly Communication

When given a research project for a presentation, a research paper or residency graduation project, I feel confident in:

• being familiar with the best practice and research ethical standards in the conduct of reporting of research and other scholarly material published in medical journals
• establishing a research team and defining the role and responsibilities of the authors and contributors
• communicating the research findings within the subject areas or specialty using following: peer-reviewed journal, poster, presentation
• writing a scientific article of these types: journal article, case-report, review, systematic review, meta-analyses
• writing a research grant
• selecting a journal for manuscript submission
• knowing how to comply with the DoD Public Access Policy
• understanding the legal, social, and economic issues that are pertinent to the information and information resources
• understanding intellectual property and its value in scholarly publication, open access, and public domain

S2. ORISPS

1. 1. What is research? Please define.
   2. What are the main steps of the research process?
   3. Which research regulations are employed in biomedical research?
   4. The protection of human subjects is described in which regulation?
   5. Respect for person, beneficence, justice – is part of what regulation?
   6. What is scientific research integrity?
   7. An outbreak of the Zika virus was reported in Latin America. A junior clinical scientist is interested in conducting research on the Zika virus and applying for a federal grant supporting this research. Please outline:
      a. a plan for the research study
      b. the steps that need to be taken to identify available grants in order to prepare a competitive research grant
   8. What type of grant should a junior clinical scientist choose for the clinical trail and research grant?
   9. How can one find an opportunity for the Mentored Research Scientist Career Development Awards (K01) for postdoctoral and clinical residency research?
   10. Please select on excerpt.
       A. “An outbreak of the Zika virus, which is transmitted by mosquitos, was detected in Brazil in May and has since moved into more than 20 countries in Latin America, including two new ones announced Monday: Costa Rica and Jamaica. The main worry is over the virus’ possible link to microcephaly, a condition that causes babies to be born with unusually small heads and, in the vase majority of cases, damaged brains. Reported cases of microcephaly are rising sharply in Brazil, ground zero for the disease, though researchers have yet to establish that Zika causes the condition.”
       B. “The findings were reported in a study of 13 babies in Brazil that was published Tuesday in Morbidity and Mortality Weekly Report. At birth, none of the babies had heads small enough to receive a diagnosis of microcephaly; months later, 11 of them did. For most of these babies, brain scans soon after birth showed significant abnormalities, and researcher found that as the babies aged, their brains did not grow or develop enough for their age and body size. The new study echoes another published this fall, in which three babies were found to have microcephaly later in their first year.”
       Based on a chosen excerpt, please describe the strategy and approach the clinical junior scientist should take to:
       a. Explore the research topic and find knowledge gaps in biomedical literature and practice pertaining to the chosen research excerpt.
       b. Assess biomedical scholarly publications and other relevant resources in order to analyze the published information.
       c. Identify available resources in order to develop a research project leading to a competitive research grant.
   11. Please indicate what research resources, relevant to the topic, should a junior scientist use to assess published studies relevant to the research on Zika virus?
   12. Please outline an effective and efficient plan to explore published literature on the research of interest.
       a. Please outline the search strategy and methods clinical junior scientist needs to develop in order to assess published literature on the research topic of interest.
   13. Please describe how a clinical junior scientist should manage identified studies obtained from electronic databases and print collections.
   14. What legal resources and information will be used in your grant application and research pertaining to the research on the Zika virus?
   15. Please choose one except and develop a research question(s) and/or hypothesis for your research study.
   16. Please define the appropriate approach a junior clinical scientist should take to construct the study design.
   17. What is the research data?
   18. Describe the steps of the research life cycle.
   19. What are the main components of data management the clinical junior scientist should consider when preparing the data management plan for the research project?
   20. Please describe the main data management practices the clinical junior scientist should employ in the research to ensure research data reproducibility.
   21. Please list the main DoD guidance that is used for the protection of research study subjects.
   22. In the “Plan to Establish Public Access to the Results of Federally Funded Research” the DoD requires a supplementary Data Management Plan. Which of the following should be included in the Data Management Plan?
       a. data, software, and material produced during the research or course of the project
       b. standards for the data and metadata format and content
       c, access and sharing policy (including privacy, confidentiality, security, intellectual property, and other)
       d. information on the re-use, re-distribution, and creation of derivative works
       e. data archiving and the preservation provision and conditions for data samples and digital formatted research data
       f. information on data restriction release for public access
   23. The clinical junior scientist is running a Diabetes Hundred project that is centered on analyzing 2500 samples from the Utah region. The samples will be analyzed using the Illumina HiSeq 200 sequencing system by two postdoctoral researchers and two graduate students from the Biosystem Laboratory. The analyses will be performed three times per week during the first quarter of 2017. These analyses will be validated three times per week during the first quarter of 2017. These analyses will be also validated by two postdoctoral researchers from the PerkinElmer Laboratory using the Ion Torrent ION SEQUENCER (508-U001) machine following the same analysis scheme as researchers from the Biosystem Laboratory.
       The clinical junior scientist is developing the Standard Operation Procedure for the data management for this project. Define and describe the data management for these two projects for the Biosystem Laboratory and PerkinElmer Laboratory researchers.
   24. What is the most common problem with data collection and analysis?
   25. What are the differences between metrics, dimension, distribution, relationships, outliers, and variance?
   26. What is bias?
   27. Describe the main criteria that are used to determine authorship in scholarly publication.
   28. What is the study that attempts to collect all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question? It uses explicit, systematic methods that are selected with a view to minimize the bias, thus providing more reliable findings from which conclusions can be drawn and decision made.
   29. What are the main components of the research manuscript?
   30. From the listed sources, indicate which citations are from books and which are from journal articles.
   31. What citation style is required in your R01 NIH grant application or research manuscript?
   32. What is the most restrictive license of the Creative Commons?
   33. What are steps the clinical junior scientist needs to take to comply with DoD Public Access Policy?

S3. Syllabus

Introduction to Academic Research

Syllabus

Part 1: Course Information

Course Description

This course provides a solid foundation for designing research in clinical sciences and sharing research findings with healthcare scholars and providers. The course is built on the fundamental theories and concepts of information sciences and knowledge management. Attendees will learn to formulate a hypothesis or develop a research question for designing a study in clinical medicine or healthcare. The course discusses the methods and tools to review literature, manage citations, and collaborative writing. A component of the course is the course project to review the literature, identify research gaps, and design the research study. Learners will develop knowledge and skills in the literature review, academic writing, and ethics of scholarly communication. This course includes individual and active learning when reasoned by the course instructor.

Textbook & Course Materials

Required Text

• Course books to be available in the course lecture, Instructure LMS

Recommended Texts & Other Readings

• • Course articles to be available in the course lecture, Instructure LMS
  • Other readings to be available in the course lectures, Instructure LMS Course Requirements

• Access to DoD Teams, Instructure learning management systems.

• MS Office Word and EndNote are required by the student for success in this course.

Course Structure

This course is comprised of lectures, discussions, quizzes, activities, and course projects. Students must devote one hour to each module to read course articles and complete course quizzes, assignments, and projects. Students are also required to participate in discussions in course Instructure.

Online Resources

Darnall Medical Library databases, books, journals, web resources, guides, and Instructure LMS.

Part 2. Student Learning Outcomes

To successfully complete this course, you will be expected to:

• Identify and evaluate information resources and citations to develop a research study or literature review.
• Analyze, synthesize, and evaluate information resources and published research to identify gaps.
• Appraise published research to develop a research question or hypothesis.
• Communicate ethical, scholarly, and professional research that is congruent with the expectation of professionals and scholars, including research publication.

You will meet the objectives listed above through a combination of the following activities in this course:

• Attend course lectures.
• Complete the course quizzes, assignments, and course projects.
• Participate in lecture discussions.

Part 3: Module/Topic Outline

• Module 1: Conducting Research
  • Lecture: Research & Scientific Method
• Module 2: Literature Search
  • Lecture: Literature Search & Search Design
• Module 3: Citation Management
  • Lecture: Introduction to Citation Management & Ethics
• Module 4: Scholarly Communication
  • Lecture: Academic & Medical Writing
• Module 5: Scholarly Communication
  • Lecture: Academic Ethics & Publication

 

COURSE FORMAT:

Module	Topic	Readings	Activities	Due Date
1	Conducting Research	Assigned article	Quiz, Project, Discussion
2	Literature Review	Assigned article	Quiz, Project, Discussion
3	Citation Management	Assigned article	Quiz, Project, Discussion
4	Writing	Assigned article	Quiz, Project, Discussion
5	Publication	Assigned article	Quiz, Project, Discussion

Note about calendar/schedule. Course assignments are posted on Instructure.

Modules Outline

Module 1. Conducting Research

Objectives: Upon successful completion of this learning module, you will be able to:

• Identify information resources for developing the research in biomedical specialties to detect research problems and/or develop a research question.
• Analyze the information resources to select the information to design the research study through the research process.
• Evaluate selected information resources and determine their information value for research gaps assessment (e.g., credibility of information resources and sources).
• Apply learned skills to design a research log of information resources to design a study or research question.

Introduction:

Research in biomedical sciences requires an assessment of published literature to design a research study and improve clinical care or enhance the body of scientific knowledge or clinical specialties. Theories and scholarly research are components in the process of designing the research. The research problem and research questions are intertwined. Knowledge about historical development in clinical specialties and present research is learned from the published research stored in literature search databases. Effective search strategy and design enhance the literature discovery published in biomedical databases. The evaluation of published citations and articles is improved using features available in search databases, critical appraisal guidelines, and information management tools. Research gaps, problems, and questions in the context of research study design are discussed during the lecture.

Identification of research data, methods, techniques, and theories will be discussed to enhance their discovery, analysis, and application from the literature search databases. Research process and information equivalently contribute to research success.

Module 1. Activities: Research & Information.

Objectives: Upon successful completion of these activities, you will be able to identify, find, evaluate, apply, and evaluate the sources of information in order to design a research question or study.

• Module 1. Assignment 1. Research & Information.
• Module 1. Discussion 1. Research & Information.

Module 2. Literature Search

Objectives: Upon successful completion of this learning module, you will be able to:

• Identify information resources required to assess research development in your specialty to identify the research topic of interest and gaps.
• Analyze research topics and published literature on research topics in literature databases.
• Select and evaluate information resources (e.g., publications, methods, techniques, tools)
• Design literature search to identify research gaps.

Introduction:

Research and medical literature and information resources are driving forces in scholarly research. Published research, information resources, and information sources contribute to the quality of the research development. Literature search terminology, controlled vocabulary, and strategies are used to design a literature search of information resources in various literature sources to identify citations relevant to the studied topic or phenomenon. Information dictionaries, thesauri, classification categories (Dewey, NLM, and others), ontologies, and search-retrieval systems are used to explore information resources, search design, and access to information resources. A controlled vocabulary is vital in health sciences for search design in various biomedical databases hosting trillions of citations. Thus, knowledge of controlled vocabulary, health science concepts, and terminology is crucial in developing research questions to review published literature in biomedical databases. A literature search, designed by applying the search concepts, vocabularies, and strategies, contributes to updating researchers with medical knowledge discoveries and enhancing their scientific knowledge. Information resources and literature search design will be discussed to develop research or clinical questions, identify research or clinical gaps, and apply information literacy skills to enhance the discovery and evaluation of published citations.

Module 2. Activities: Research & Information.

Objectives: Upon successfully completing module activities, you will be able to identify and find information resources to identify research topics using primary and secondary literature to design research questions or study and manage the literature review project.

• Module 2. Discussion 1. Research Topic & Literature
• Module 2. Discussion 2. Information and Literature Discovery
• Module 2. Quiz. Literature Search

Module 3. Citation Management

Objectives: Upon successful completion of this learning module, you will be able to:

• Gather information resources from information resources using various information management tools.
• Understand principles of citation analysis and style.
• Apply learned skills to design a citation library using EndNote.
• Apply learned skills to develop best practices for citation management.
• Apply citation ethics in scholarly communication.
• Understand plagiarism in citing research.

Introduction:

Citations are used in scholarly communication to reference literature, either for an argument or statement, and support or disapprove of the research statement, argument, or statement. Citations are used in the research article’s introduction, methods, materials, and discussion. Citations are placed in the context of the research, research question, or hypothesis in the prior research. Citations are used in the research to understand the state of knowledge about the studied topic or phenomenon in published literature. Various information resources, publication types, and citation styles are used in scholarly publications of science or clinical research. Citation management tools such as EndNote, Zotero, Mendeley, RefWorks, and others are used by researchers. Predatory information resources (databases, search engines, predatory publication databases, and journals), including citations and published articles, affect the quality of research citations. Selecting, citing resources, predatory publication resources, and research citations are discussed.

Knowledge of plagiarism, intellectual property, and copyright is necessary for researchers and scholars.

Module 3. Activities: Citation Management.

Objectives: Upon successful completion of this learning module, you will be able to:

• gather information resources from information sources: literature databases, search engines, repositories, and other sources.
• apply the citation principle to citing various information resources in scholarly (research and medical) publications.
• apply best data management practices to design your research citation EndNote library.
• understand the variety of citing sources, styles, and citing ethics.
• understand the citation delineation process as well as research citing delineation.
  • Module 3. Quiz. Citation Management
  • Module 3. Discussion. Citation Analysis
  • Module 3. Citation Management Project

Module 4. Academic & Medical Writing

Objectives: Upon successful completion of this learning module, you will be able to:

• Identify information resources for designing literature ontology and literature review outline.
• Analyze the information resources to outline the research article or case study.
• Evaluate selected citations and research.
• Apply learned skills to outline the literature review of an academic article or case report and methods and material.
• Apply research ethics to writing and communicating research.

Introduction:

The writing process consists of the following steps: developing, planning, outlining, drafting, revising, and editing an article. Research articles published in scientific journals are based on scientific research methods, hypothesis testing, methodology used, data analysis, and interpretation. These articles enhance the fundamental science knowledge through research design to answer research questions.

Information presented in published articles contains an analysis of the research development. In contrast, each research contributes knowledge by creating a new understanding of their research areas or specialties. Academic or clinical writing requires conducting a literature search, gathering published literature to understand the literature published before the researcher’s question, and new perspectives emerging in the research. Thus, scientific knowledge or research phenomenon involves an analysis of published literature. This process requires asking questions about published research, information, meaning, and implication of published research. Thus, during the writing process of academic or clinical articles, researchers ask questions based on – what, why, when, where, and who – to provide insight for analysis of published literature, evaluation, and interpretation. This lecture discusses research writing: purpose, research evidence, research design, evaluating literature, annotating literature, outlining the research manuscript, arranging research ideas, drafting, revising, and editing to write clinical or research articles.

Research and scholarly communication ethics are also introduced.

Module 4. Activities: Research & Information.

Objectives: Upon successful completion of these activities, you will be able to:

• identify, analyze, and evaluate selected citations and research for introduction, methods, materials, and discussion.
• analyze, synthetize, and evaluate published research about your research and study results.
• annotate the selected information resources, citations, and research.
• outline an article or case report (IMRAD)
• understand the concepts of research and publication ethics and researcher’s ethics.
  • Module 4. Assignment 4. Literature in Research or Case Studies.
  • Module 4. Discussion 4. IMRAD – Writing Approaches and Styles.

Module 5. Publication

Objectives: Upon successful completion of this learning module, you will be able to:

• Understand the ethics of scholarly publication.
• Identify information resources to identify authors’ publication guidelines.
• Apply learned skills in intellectual property, plagiarism, and research integrity to submit an article for publication.

Introduction:

Scientific research and scholarly communication are governed by principles of scientific integrity and shared values in scientific research. Authorship and intellectual property are crucial elements of scholarly publication. Author’s journal publication and submission guidelines vary among scholarly journals and publishers. Thus, understanding publication requirements and academic ethics is crucial for scholars to select academic journals and publish their research.

Module 5. Activities: Research & Publication.

Objectives: Upon successful completion of these activities, you will be able to:

• Understand the publication process and policies, research ethics, and consent.
• Identify information resources that are under Intellectual Property & Copyright laws.
• Understand the concepts of research and publication ethics and researcher’s ethics.
• Identify, analyze, and evaluate selected journals to select a potential journal for manuscript submission.
  • Module 5. Assignment 5. Research & Publication Ethics.
  • Module 5. Discussion 5. Journal Selection.
  • Module 5. Project. Publication Ethics.

Part 4: Grading Policy Graded Course Activities

Visit the Assignments, Quizzes, Discussions, and Projects link in Instructure for details about each assignment listed below. Click on Assessments to access quizzes and exams. The online grades are posted three days following the completion of course assignments. Grades are posted on your Instructure home page.

Points	Description
# 5	Quiz 1. Conducting Research.
	Quiz 2. Literature Search.
	Quiz 3. Citation Managing.
	Quiz 4. Publication.
# 20	Assignment 1. Research & Information.
	Assignment 2.
	Assignment 3.
	Assignment 4.
# 25	Discussion 1. Research & Information.
	Discussion 2.1. Research Topic & Literature. Discussion 2.2. Information & Literature Discovery.
	Discussion 3. Citation Analysis.
	Discussion 4. Scholarly Communication.
	Discussion 5. Publication Ethics.
# 50	Course Project: Article
	Project 1. Literature Review.
	Project 2. Citation Management.
	Project 3. Scholarly Writing.
	Project 4. Publication Ethics.
100	Total Points Possible

 

Course Policy

Submit your course assignments per the course timeline. Assignments submitted beyond the course timeline will not be graded. Course instructor requires a compelling reason to accept your assignments.

Letter Grade Assignment

Course final grades assigned for this course will be based on the percentage of total points earned and are assigned as follows:

 

Letter Grade	Percentage	Performance
A	93-100%	Excellent
A-	90-92%	Nearly Excellent
B+	87-89%	Very Good
B	83-86%	Good
B-	80-82%	Mostly Good
C+	77-79%	Above Average
C	73-76%	Average
C-	70-72%	Mostly Average
D+	67-69%	Below Average
D	60-66%	Poor
F	0-59%	Failing

 

Part 5: Course Policies

Attend Class

Students are expected to attend all course modules as listed on the course calendar.

Participate

Course modules’ attendance will track your attendance. Your participation in the discussion will be tracked in the course Instructure. The course Instructure will be used to track course quizzes, assignments, discussions, and projects to monitor their participation in the course.

Complete Assignments

All course assignments will be submitted electronically through Instructure by course timeline and schedule. Course projects must be completed per the course timeline.

Accommodations Needed

Disability Statement: If you are a student with a disability requesting reasonable accommodations in this course, you are encouraged to discuss approved accommodations with the course instructor.

Commit to Integrity

You are expected to commit to learning and participation in this course. You are expected to remain professional during course participation and communication with the course professor and your colleagues.

Academic Integrity

You are expected to commit to the academic integrity and policy of the ACGME and DoD.

Acknowledgement

I would like to thank Dr. Francois Tuamokumo for the statistical support.