Below is a copy of the Medical Informatics Needs Assessment, given to our Faculty, Residents and Staff at the beginning of our curriculum development. Data from this survey and followup surveys was used to track self-reported competencies and attitudes as we developed our curriculum. You can download a copy of the survey as well. To facilitate data collection and analysis, we converted the original paper survey to an Access database questionnaire, which could be done "on-line". This is also available for download in Access MDE form. If you use any of these tools, please contact Dr. Zelnick, who would be interested in helping you use them as well as discussing results and experiences.
Medical Informatics Needs Assessment
I. Name___________________________________________ Date_________________________
I. Demographics
a. Your age:_________
b. Your gender:
Female p
Male p
c. Training level: R1 R2 R3 MD Faculty Other Faculty FPC Staff FHC Staff |
_______________________________________ |
II. Computer Experiences
a. How frequently do you use a computer (choose one)?
Daily
Weekly
Monthly
Rarely
Never
b. What training or experience with computers have you had? (check all that apply)
p Formal course(s) in computer science or related field
p Formal medical school training in computers
p Formal residency or fellowship training in computers
p Formal workshops or conferences on computers for which I received CME credit
p Workshops or conferences on computers for which I did not receive CME credit
p Self-guided learning about computers
p None
c. On the whole, how sophisticated a computer user do you consider yourself? (choose only one)
p Very sophisticated
p Sophisticated
p Neither sophisticated nor unsophisticated
p Unsophisticated
p Very unsophisticated
d. On the whole, how interested are you in learning more about computers? (choose only one)
p Eager to learn more
p I’d like to learn more
p I’ll learn more it I have to, to do my job
p I’d just as soon avoid the subject.
p I feel hostile when I hear the word "computer".
e. On the whole, how interested are you in learning more about the noncomputer aspects of medical informatics (e.g. evidence based medicine, Problem oriented medical record, etc.) (choose only one)
p Eager to learn more
p I’d like to learn more
p I’ll learn more it I have to, to do my job
p I’d just as soon avoid the subject.
p I feel hostile when I hear the words "medical informatics".
f. Please complete the following table.
1. Where do you have access to a computer or terminal? Complete all rows that apply |
What type of machine? |
Other features does this computer have? Check all that apply: (AMOS is the FPC billing system. Meditech is Mercy’s system, PHAMIS is St. Lukes’. |
How often do you use this computer, and how comfortable you are using it? |
|||||
__No access anywhere |
|
|
||||||
__At work,on my desk |
__PC, WINDOWS __PC, DOS only __Mac __PC or Terminal connected to a mainframe _Other, which?_______ |
__AMOS __Meditech __Phamis __Local Area Network __Internet connection __Medline search |
|
|
||||
__At work,NOT on my desk |
__PC, WINDOWS __PC, DOS only __Mac __PC or Terminal connected to a mainframe _Other, which?_______ |
__AMOS __Meditech __Phamis __Local Area Network __Internet connection __Medline search |
|
|
||||
__Laptop computer |
__PC, WINDOWS __PC, DOS only __Mac __PC or Terminal connected to a mainframe _Other, which?_______ |
__AMOS __Meditech __Phamis __Local Area Network __Internet connection __Medline search |
|
|
||||
__In a hospital teaching area, which hospital? ________________ |
__PC, WINDOWS __PC, DOS only __Mac __PC or Terminal connected to a mainframe _Other, which?_______ |
__AMOS __Meditech __Phamis __Local Area Network __Internet connection __Medline search |
|
|
||||
__In my ambulatory clinical teaching area |
__PC, WINDOWS __PC, DOS only __Mac __PC or Terminal connected to a mainframe _Other, which?_______ |
__AMOS __Meditech __Phamis __Local Area Network __Internet connection __Medline search |
|
|
||||
|
__PC, WINDOWS __PC, DOS only __Mac __PC or Terminal connected to a mainframe _Other, which?_______ |
__AMOS __Meditech __Phamis __Local Area Network __Internet connection __Medline search |
|
|
g. Where do you have access for conducting MEDLINE searches? (Check all that apply) |
How do you connect to the information? (Check one) NLM = National Library of Medicine |
__ No access |
|
__ At work |
__ Grateful Med to NLM __ Modem to library, which?________ __ CD-ROM on site __ Other, how? __ Physicians On Line |
__ In my clinic |
__ Grateful Med to NLM __ Modem to library, which?_________ __ CD-ROM on site __ Other, how? __ Physicians On Line |
__ In thehospital library, which hospital? ________ |
__ Grateful Med to NLM __ I search __ CD-ROM on site __ Someone else searches for me |
__ At home |
__ Grateful Med to NLM __ Modem to library, which? _________ __ CD-ROM on site __ Other, how? __ Physicians On LIne |
h. To what extent do you personally use a computer for each of the following professional tasks? Please circle your answer.
1. Never perform this task
| 2. Perform this task but never use a computer
| | 3. Sometimes use a computer
l | | 4. Often use a computer
l | | | 5. Always use a computer
v v v v v
Documenting patient information (e.g., history & physicals, progress notes) |
|
2 |
3 |
4 |
5 |
Accessing clinical data (e.g., laboratory data, EKGs, radiology reports) |
|
2 |
3 |
4 |
5 |
Communicating with colleagues |
|
2 |
3 |
4 |
5 |
Obtaining advice on a specific patient's diagnosis or therapy |
|
2 |
3 |
4 |
5 |
Scheduling patient appointments |
|
2 |
3 |
4 |
5 |
Writing (e.g., grants, research papers, teaching material) |
|
2 |
3 |
4 |
5 |
Prepping presentation slides or overheads |
|
2 |
3 |
4 |
5 |
Performing statistical analysis on clinical or research data |
|
2 |
3 |
4 |
5 |
Searching the medical literature (e.g., MEDLINE) |
|
2 |
3 |
4 |
5 |
Teaching students and residents |
|
2 |
3 |
4 |
5 |
Financial tools: accounting and analysis |
|
2 |
3 |
4 |
5 |
Internet search and research on topic |
|
2 |
3 |
4 |
5 |
|
|
2 |
3 |
4 |
5 |
Generating reports from a database |
|
2 |
3 |
4 |
5 |
Patient education |
|
2 |
3 |
4 |
5 |
i. Does your current hardware access meet your work needs? _Yes _No If no, why not?
j. Does your current software meet your work needs? _Yes _No If no, why not?
k. Rate the quality of computer/informatics training provided to residents in your program?
__Excellent __Very good __Good __Fair __Poor __None
la. (Faculty only) Rate the quality of computer/informatics training provided to faculty in your program?
__Excellent __Very good __Good __Fair __Poor __None
lb. Rate the quality of computer/informatics training provided to staff in your program?
__Excellent __Very good __Good __Fair __Poor __None
m. For each skill below, we want you to enter ONE code number which reflects both (a) whether or not you can perform the skill, and (b) your attitude about learning the skill. Example: Enter "4" if you can’t do the skill right now, but are excited about learning to do it.
CODE |
Can you do this skill? |
How do you feel about learning this skill? |
1 |
NO |
-- I really don’t want to learn this skill |
2 |
NO |
- I’d just as soon not learn this skill, but will if I need to for the job |
3 |
NO |
= Neutral about learning this skill |
4 |
NO |
++ I’m eager to learn this skill |
5 |
YES |
- No, I know more than I want to already |
6 |
YES |
+ Yes,I’d like to learn more about it |
7 |
YES |
++ Yes, I’m feeling a real need to learn a lot more about it. |
Basic Electronic Mail (Email) Skills
___Log in to the system
___Send messages
___Read messages
___Reply to messages
___Forward messages
___Delete messages
___Print messages
___Use a lookup or address book feature
___Save messages into appropriate folders
___Create folders or mail boxes
___Change password
___Attach files for shipment
___Read or save attachments
___Use online help
___Log out or shut down
Advanced Email Skills:
___Create personal mail groups
___Create/use a signature
___Use/customize button bars
___Create/use rules
___Handle trash (undelete messages, empty trash)
Basic Word Processing Skills
___Create a new document
___Open an existing document
___Save a document
___Save a document to a floppy disk
___Print a document
___Spell check a document
___Simple formatting (bold, underline, center,
margins, fonts)
___Cut, copy, or paste text
___Use online help
Advanced Word Processing Skills
___Use page numbers
___Use headers/footers
___Use the thesaurus
___Create a table
___Use hanging indent and user-defined tabs
___Save a file in a different format (ASCII, other
format)
Operating System Skills
___Format or initialize floppies
___Copy/move files to other locations
___Create directories/folders
___Delete files
___Switch among open applications
___Cut, copy, or paste text
___Log in to a network
___Log out of a network
___Change passwords
___Access files on a shared network
___Use the public folders to share network files
___Remove/delete folders/directories
___Back up files
___Find files
___Undelete erased files
___Customize the desktop
Calendar/Scheduling
___Read calendar information
___Add/delete items in personal calendar
___Print calendars
___Accept/decline appointment requests
___Schedule appointments
___Schedule resources (rooms, department car)
___View other calendars (with permission)
Medline Searches
___Log into the Medline computer
___Select a bibliographic database
___Select the type of search (keyword, author, etc.)
___Perform a search
___Review search results on screen
___Print search results
___Download results onto to a disk
___Disconnect from the system
Clinical Computing Skills
___Use a clinical business billing system
___Use electronic medical records
___Access patient information in a clinical information
system
___Use a decision-support program (eg, QMR)
___Use a CD-ROM (such as Stat-Ref or SAM) to
access information
___Use a drug interaction program (eg, AskRx)
___Use Problem Knowledge Couplers
___Use Clinical Reference Systems (CRSWin handouts on disk)
___Use Instant Medical History
___Use PATIENTED ® (patient education templates)
Analysis Skills
___Use a spreadsheet (like Excel) for financial analysis
___Use a spreadsheet for statistical analysis
___Generate reports from a database (AMOS, Access, etc.)
Programming Skills
___Write or record a Macro
___Write a simple program, in _____________language(s)
___Write a complex program in ____________language(s)
___Build a Web Page
Other Skills
___Use presentation software (eg, PowerPoint) for
transparencies, slides, etc.
___Use a web browser (Netscape or MIE) to
access the World Wide Web (WWW)
___Use a browser to search for information on WWW
___Use a WWW search engine (Yahoo, Altavista, Excite, etc.)
___Access Usenet discussion groups (NEWS://)
___Use an Internet Gopher
___Telnet into a remote computer
___Subscribe to and unsubscribe from a Listserv
(email discussion group)
___Use File Transfer Protocol (FTP) to download files
___Use a personal data base (eg, article file,
document procedures)
___Participate in distance education by phone, TV, etc
k. What computer training would you most like to receive?___________________________________________ ______________________________________________________________________________________________________________________________________________________ ____________________________________________________________________________________________________
n. What software program(s) do you use for:
email?____________________________________
word processing?___________________________
data base?_________________________________
communications (modem)?____________________
spreadsheet?_______________________________
statistics?__________________________________
presentation graphics?________________________
medical information?_________________________
III. Computer Knowledge
Below are a set of paired terms that relate to computers in medicine. Please score your knowledge of the distinction between
the terms in each pair, using the following scale:
1 |
I don't understand the distinction at all.
|
|||||
| |
2 |
I have a general appreciation of the distinction but I couldn't define it. |
||||
| |
| |
3 |
I can define the distinction precisely |
|||
| |
| |
| |
||||
V |
V |
V |
||||
Hardware Û Software |
1 |
2 |
3 |
|||
Images Û Graphics |
1 |
2 |
3 |
|||
Forward chaining Û Backward chaining |
1 |
2 |
3 |
|||
Free textÛ Coded data |
1 |
2 |
3 |
|||
Field Û Record |
1 |
2 |
3 |
|||
Relational database Û Flat-file database |
1 |
2 |
3 |
|||
Data in memory Û Data on disk |
1 |
2 |
3 |
|||
Sensitivity Û Positive predictive value |
1 |
2 |
3 |
|||
ICD9-CM Û SNOMED |
1 |
2 |
3 |
|||
Entities Û Relationships |
1 |
2 |
3 |
|||
Floppy diskÛ Hard disk |
1 |
2 |
3 |
|||
Full-text database Û Bibliographic database |
1 |
2 |
3 |
|||
Interpreter Û Compiler |
1 |
2 |
3 |
|||
Mainframe computer Û Personal computer |
1 |
2 |
3 |
|||
Electronic mail Û Electronic bulletin board |
1 |
2 |
3 |
|||
Client Û Server |
1 |
2 |
3 |
|||
Digital Û Analog |
1 |
2 |
3 |
|||
Database Û Knowledge base |
1 |
2 |
3 |
|||
Don’t Understand distinction |
General understanding, can’t define |
Could define distinction precisely |
||||
RAM Û ROM |
1 |
2 |
3 |
|||
Data Û Information |
1 |
2 |
3 |
|||
Privacy Û Confidentiality |
1 |
2 |
3 |
|||
Identification Û Authentication |
1 |
2 |
3 |
|||
Algorithm Û Flowchart |
1 |
2 |
3 |
|||
Problem Oriented Record Û SOAP Charting |
1 |
2 |
3 |
|||
Internet Û World Wide Web |
1 |
2 |
3 |
|||
Java Û HTML |
1 |
2 |
3 |
|||
Backup Û Archive |
1 |
2 |
3 |
|||
Rule Based System Û Neural Network |
1 |
2 |
3 |
|||
In Statistics: Type 1 Error Û Type 2 Error |
1 |
2 |
3 |
|||
Standard Deviation Û Standard Error |
1 |
2 |
3 |
|||
Evidence Based Medicine(EBM) Û Your unsubtantiated clinical knowledge (YUCK) |
1 |
2 |
3 |
|||
Disease Oriented Evidence(DOE) Û Patient Oriented Evidence that Matters (POEM) |
1 |
2 |
3 |
|||
Recall based systems Û "Just-in-time" systems |
1 |
2 |
3 |
|||
Patient instructions Û Patient education |
1 |
2 |
3 |
|||
Computer Literacy Û Punishment |
1 |
2 |
3 |
IV. Applications of Computers in Medicine
A. Priorities for future development. Listed below are 18 potential functions of computers in medicine. First, circle six functions that you consider to be of highest priority for future development. Then, circle six functions you consider to be of lowest priority for future development.
Priority for future development
Highest (Choose 6) |
Lowest (Choose 6) |
|
Creating electronic medical record systems to replace the paper record |
Highest |
Lowest |
Taking a medical history from a patient |
Highest |
Lowest |
Entering physician orders such as laboratory tests or prescriptions |
Highest |
Lowest |
Controlling of machine tools that assist in surgical procedures |
Highest |
Lowest |
Performing automated interpretations of diagnostic tests (e.g., X-rays, EKGs, and pulmonary function tests) |
Highest |
Lowest |
Assisting in the development of treatment plans for patients with complex problems |
Highest |
Lowest |
Computer assisted patient education |
Highest |
Lowest |
Providing reminders to patients and clinicians of overdue visits, tests, or preventive care |
Highest |
Lowest |
Collecting directly from patients information that is useful for screening or triage |
Highest |
Lowest |
Augmenting medical care in geographic areas where trained personnel are not readily available |
Highest |
Lowest |
Monitoring and adjusting life support systems in intensive-care units |
Highest |
Lowest |
Offering advice in the diagnosis of the patient with an unknown illness |
Highest |
Lowest |
Substituting for cadavers in the teaching of anatomy |
Highest |
Lowest |
Simulating patient encounters to teach clinical skills |
Highest |
Lowest |
Administering a medical licensure examination using simulations |
Highest |
Lowest |
Identifying patients eligible for clinical trials |
Highest |
Lowest |
Assisting in collecting and reporting clinical trials data |
Highest |
Lowest |
Auditing the quality of care provided by hospitals and physicians |
Highest |
Lowest |
Reviewing the utilization of medical resources |
Highest |
Lowest |
B. Capabilities of future computer systems. If you were considering the use of a computer-based system in medical care, how necessary would the following capabilities be? Circle your response using the scale below.
1. Vitally necessary: Any system I would use must have this capability. I would not use a system that lacked it.
2. Generally necessary: I would be much more likely to use a system having this capability, but I might use a system that lacked it.
3. Somewhat necessary: I would be somewhat more likely to use a system because it had this capability.
4. Not necessary: My decision to use a system would be unaffected by the presence of this capability.
5. Unable to respond: The meaning or implications of this capability are not clear to me.
Vitally Necessary |
Generally Necessary |
Somewhat necessary |
Not necessary |
Unable to respond |
|
I can enter information in my own words and not need to know any special codes |
1 |
2 |
3 |
4 |
5 |
I can learn to use the system in less than two hours. |
1 |
2 |
3 |
4 |
5 |
I can access the system at any place in the clinical setting |
1 |
2 |
3 |
4 |
5 |
The system always responds to my queries in less than five seconds |
1 |
2 |
3 |
4 |
5 |
The system always displays X-rays and other images in less than 30 seconds |
1 |
2 |
3 |
4 |
5 |
I can interact with the computer without using a keyboard |
1 |
2 |
3 |
4 |
5 |
The system is always functioning. There is never any "down time." |
1 |
2 |
3 |
4 |
5 |
The system can be implemented with no changes whatsoever to existing clinic routines |
1 |
2 |
3 |
4 |
5 |
When a system provides medical advice on the care of specific patients, it always provides multiple alternative recommendations |
1 |
2 |
3 |
4 |
5 |
When a system provides medical advice on the care of specific patients, it can quantify the level of certainty inherent in its recommendations |
1 |
2 |
3 |
4 |
5 |
The system takes a patient's own preferences into account when giving advice to clinicians |
1 |
2 |
3 |
4 |
5 |
The system can clearly explain the rationale for advice it gives on the care of patients |
1 |
2 |
3 |
4 |
5 |
Users can browse the information in a system as well as asking it to provide advice about care of specific patients |
1 |
2 |
3 |
4 |
5 |
The system has been demonstrated in research studies to make diagnoses at least as accurate as human consultants |
1 |
2 |
3 |
4 |
5 |
The system has been demonstrated in research studies to provide treatment recommendations at least as accurate as human consultants |
1 |
2 |
3 |
4 |
5 |
Help on how to use the program is available on-line |
1 |
2 |
3 |
4 |
5 |
Level of confidentiality and security must be better than the paper record |
1 |
2 |
3 |
4 |
5 |
1 |
2 |
3 |
4 |
5 |
C. Potential effects of computers. Given below are some effects that computers may have on medicine and health care. For each, indicate whether you believe the effect will be beneficial or detrimental using the scale below:
Effect of computers on: |
Highly detrimental |
Detrimental on the whole |
Neither detrimental nor beneficial |
Beneficial on the whole |
Highly beneficial |
Costs of health care |
1 |
2 |
3 |
4 |
5 |
Clinician autonomy |
1 |
2 |
3 |
4 |
5 |
Quality of health care |
1 |
2 |
3 |
4 |
5 |
Interactions within the health care team |
1 |
2 |
3 |
4 |
5 |
Role of the government in health care |
1 |
2 |
3 |
4 |
5 |
Access to health care in remote or rural areas |
1 |
2 |
3 |
4 |
5 |
Management of medical/ethical dilemmas |
1 |
2 |
3 |
4 |
5 |
Enjoyment of the practice of medicine |
1 |
2 |
3 |
4 |
5 |
Status of medicine as a profession |
1 |
2 |
3 |
4 |
5 |
Continuing medical education |
1 |
2 |
3 |
4 |
5 |
Physicians’ stress levels |
1 |
2 |
3 |
4 |
5 |
The self-image of clinicians |
1 |
2 |
3 |
4 |
5 |
Humaneness of the practice of medicine |
1 |
2 |
3 |
4 |
5 |
The rapport between clinicians and patients |
1 |
2 |
3 |
4 |
5 |
Personal and professional privacy |
1 |
2 |
3 |
4 |
5 |
Clinicians' access to up-to-date-knowledge |
1 |
2 |
3 |
4 |
5 |
Patients' satisfaction with the quality of care they receive |
1 |
2 |
3 |
4 |
5 |
Generalists' ability to manage more complex problems |
1 |
2 |
3 |
4 |
5 |
V. Open Response Area (Use back of page if necessary)
1. Regarding medical information in general, is there anything you might like to do or be able to do that you can’t do right now.
2. What barriers or problems do you see us encountering and needing to overcome. What can we do to better address these barriers.
3. What anxieties or worries do you in relation to changes in the way we deal with medical information?
4. What else do you think we should know?
Page 9 Acknowledgments: This survey was adapted and modified from the work of Craig Gjerde, Univ. of Wisconsin who provided the sections on computer access and computer skills. The sections on computer use, knowledge, priorities, capabilites and effects are derived from the Computers in Medical Care Survey, ã 1993 William Detmer M.D., MSc and Charles Friedman, Ph.D.. (http://www.med.virginia.edu/~wmd4n/medsurvey.html, or email: bdetmer@virginia.edu.) Synthesis and expansion of this work accomplished by Charles Zelnick M.D. Many thanks to the above authors.