HCD/OpenMRS/Design Development
From Rockalypse
Contents |
Introduction
The system being evaluated is OpenMRS, an open-source electronic medical records system. It is designed to be used in developing countries, where most patient data is currently stored on paper or in spreadsheets. The system is designed to be customizable so that it can meet the needs of a variety of users. The project also aims to provide health care in resource-constrained environments, and the technological issues that occur in these environments is always a concern. Since one of the OpenMRS project's goals is to design for user needs, the results of our usability testing will most likely be welcomed.
The purpose of this experiment is to identify usability issues in the OpenMRS interface and to find solutions for them. Our initial thoughts on problems and possible improvements can be found in our Needs Report. We hope to design an interface that is more usable and has less of a learning curve than the current system. The OpenMRS developers may then use our design and suggestions in order to create a new interface that better meets the needs of practitioners working in the field.
Prototypes
Original Interface
The first paper prototype was the original web-based interface in low-fi. We used this prototype in order to gather some initial usability data and to confirm our suspicions pertaining to usability issues in the interface. In addition to the usability data we gathered, we also found that it would be difficult to represent all of the features provided by OpenMRS on one paper prototype. This resulted in the reduced interface found in the next section.
Here are a few pages from the interface. The complete collection of pictures can be found here.
When the user first logs in, they see this screen. The web browser is used as a background for the rest of the interface, since we based this mockup off of the web-based demo.
This is the main page for the concept dictionary. This allows users to both search for concepts in the dictionary and add new concepts.
This is the main page for patient management. The user can either search for patients in the database or add new patients.
The patient page for Alison Tomno, the patient we used for usability testing. The selected tab is Overview, but other information about the patient can be viewed by clicking the appropriate tab.
The Composition tab of the Cohort Builder. Allows the user to combine previous searches using boolean operators.
Small Screen Interface
In our second prototype we focused on a small set of usability issues. This interface is designed for a practitioner working in the field, and therefore eliminates many administrative functions. It was designed as an application, and the goal is to display only what is needed on each screen. Another concern is how many clicks it takes to navigate through the interface.
Here are a few pages from the interface. The complete collection of pictures can be found here.
This is the main page of the application. Here the user (a practitioner in the field) can perform one of three tasks: find and manage patients, create new patients, and search for concepts in the dictionary. Because we were designing for a smaller screen, we used large buttons and did not overwhelm the user with too much information on each page.
This page allows the user to search for patients in the database.
This is the main page for the patient we used throughout our usability testing, Alison Tomno. Instead of overwhelming the user with information, we divided the tasks that would need to be performed on a patient into separate screens. In this way, the user only sees what information they actually need for a given task.
In the previous interface, the only way to add an encounter was to find a small link on the Administration page. Now the ability to add an encounter is under the list of encounters for a patient, which our testing shows to be a much more intuitive location for the user.
This page allows the user to add a patient to the database. In the previous mockup the user had to enter partial information and then click in order to enter all of the details pertaining to a patient. Now they can enter data into all of the fields when they decide to create a patient, which is much more efficient.
This page allows the user to search for concepts in the dictionary. A practitioner working in the field will probably not need to add new concepts, so this functionality was removed in order to simplify navigation and the interface in general.
Method
Participants
We had two participants during the course of our usability testing. The first subject was a professor of Global Health and Development who had some experience working in the field. The subject had been using Excel spreadsheets in order to keep track of patient data, and was looking for a simpler way to manage patient information in a fast and reliable way. This test subject represented an expert user, and we valued the usability data and suggestions provided by this subject the most. The second subject was a member of our class with no experience using a medical records system. This subject was able to confirm some of the findings from the first subject, which suggests that the issues we found were with usability in general and were not specific to an expert user.
Task Scenarios
Original Interface
1) Finding a patient and viewing information.
Note: By new navigational elements we mean additional buttons and navigation bars that appear as a result of navigating to a page other then the main page.
- Search for the patient named "Alison Tomno" and pull their information.
- Does the subject get a grasp of the overall structure of the system's interface?
- Does the subject notice the main navigational element?
- What is the date of the patient's last encounter?
- Does the subject notice the static information present on the new page?
- Is the subject focused on the new navigational elements present on this page?
- What town is the patient from?
- Is the subject able to make use of the new navigational elements on this page?
2) Creating a new patient and editing their information.
Note 1: On the patient creation page there is nothing suggesting required fields even though there are a few. Also, one of the identifier types checks for the format of the input but again nothing suggests the proper format.
Note 2: It is possible to edit a patient's information other then their demographic information. But links to this functionality are found under "Demographics" on the patient dashboard. A better placement would be above the navigation bar on the dashboard since this area is visible at all times. Links to this functionality are also found on the administration page.
Note 3: Adding encounters can only be done from the administration page.
- Create a new patient.
- Is the subject able to interact with error messages should they appear?
- The patient has moved, update their location information.
- Is the subject able to find the link to this functionality?
- Return to the patient page to view the updated info.
- Is the subject able to return to the patient dashboard?
- Add an encounter for this patient.
- Is the subject able to find this functionality?
3) Using the concept dictionary.
- Find out what AIDS is.
- Is the subject able to find the concept dictionary?
- Add a concept for Mononucleosis.
- Is the subject able to find this functionality?
4) Using the cohort builder.
- Perform separate searches for "Male", "Between 20 and 30 years", and "ADULTRETURN".
- Can the subject find the cohort builder?
- Does the subject realize the existence of different types of searches?
- Display the results of the first search.
- Does the subject notice that an easily accessible search history is kept?
- Perform a search for patients that are "Male" and "Between 20 and 30 years old".
- Is it clear to the subject that they can combine previous searches?
Small-Screen Mockup
First, have subject login using a provided username and password.
1) Searching for a concept in the dictionary.
- Search for "blood pressure".
- Does the subject know to look up concepts in the dictionary?
- Have the subject view information on both diastolic and systolic blood pressure.
- Is the subject able to successfully navigate through the interface by finding the correct buttons?
2) Creating a new patient.
- Ask the subject to create a new patient with any information.
- Use scrap pieces of paper in order to reflect what they typed on the actual interface.
- Does the subject understand which fields are required?
3) Finding and managing a patient.
- Have the subject search for the patient "Alison Tomno".
- Is the subject able to successfully navigate to the patient's main page?
- Now have them find what country she is from?
- Does the user know to click on "Overview"?
- Have the subject change her mother's name to "Maria".
- Is the subject able to successfully navigate to the correct screen?
- Have the subject add an encounter for the patient.
- Is this better than having it hidden in the Administration page?
- Have the subject add a regimen of Triomune-30, 50mg dose, 1/day, 7 days/week, starting today.
- Have the subject create a graph for Diastolic Blood Pressure.
In general, is the subject able to navigate through the application without difficulty?
Procedure
The first thing we did in this process was to search for possible participants. Professor Jadud told us of two individuals in the college community who had some knowledge in this area, so we tried to contact them. We were ultimately only able to perform usability testing on one of them, but our testing was fruitful nonetheless. A member of our class served as a second test subject. We chose this individual because they had no experience working with a medical records system, and would therefore give us the point of view of a novice user.
After recruiting users, we began the paper prototyping process. We chose to base the original prototype on the current OpenMRS interface in order to gather some initial usability data. We then performed usability testing on our test subjects. Each testing session lasted at most a half hour. After analyzing our results and conferring with Professor Jadud, we decided that trying to make the current OpenMRS interface usable would be extremely difficult. This realization caused us to create our second prototype, which was designed for someone working in the field. This mock-up removed many administrative features, and the interface was designed for a small screen. We then performed usability testing in the same way as before.
After our second round of testing we decided that creating another interface would not be justified by the small number of usability issues we found. We then entered the reporting portion of the process, which involved writing up our findings and summarizing the results of our usability testing.
Test Measures
Our general test measure was intuitiveness. We were interested in whether the user was able to perform the tasks without guessing or asking the facilitator for help. This test measure was chosen because it represents the main problem with the current OpenMRS interface. It is a database management system, and the organization of the interface closely resembles the structure of the database. Another measure was the speed with which the tasks were performed. This was chosen because someone working in the field needs to spend as little time as possible using the interface. Their focus needs to be on the patients they are trying to help.
Results
Test Subject 1 - Original Interface
For the first task, the patient was generally able to use the interface effectively to complete the task. However, the user did not notice that the date of the patient's last encounter was found at the top of every patient page. The subject instead went to the Encounters page, which provided the same piece of information. This suggests that the page is cluttered with unnecessary information.
In the second task, the user was able to successfully create a new patient and edit some basic information about them. However, the subject did not know how to add an encounter for the patient. The user expected it to be on the Encounters page, and only found it on the Administration page after prompting from the facilitator.
For the third task, the user first looked on the Administration page for the concept dictionary, even though there was a link at the top of every page. This may have been due to the fact that the last task involved going to that page. After they found the dictionary, they had no difficulty in completing the rest of the task.
In the fourth task, the user realized that Cohort Builder was the appropriate page for searching and was able to complete individual searches. However, when the subject was asked to combine searches, they did not know that the Composition page was the correct place to go. This task was completed successfully after the user was aided by the facilitator.
Test Subject 2 - Original Interface
In the first task, the subject was able to find the patient's information, but was confused by the "Include Verbose" check mark, since this did not change the search results. Like the first subject, this user did not notice that the date of the last encounter is at the top of every patient page, and instead found it under Encounters. Finally, the subject did not realize that the town a patient was from would be under Demographics.
For the second task, the user was able to create a new patient and edit patient information. However, like the first user, this subject was not able to determine how to add a patient encounter. Only after being prompted by the facilitator did the user find this link on the Administration page.
In the third task, the subject was able to search for a concept and view the corresponding information. However, the user attempted to type the term Mononucleosis into the search box when trying to add a new concept.
For the fourth task, the user did not immediately realize that the Cohort Builder was the place to go for performing searches. Once the subject was directed to this page, however, they had no difficulty in completing the tasks.
Test Subject 1 - Small Screen Interface
In the first task, the subject experienced no difficulty in searching for concepts in the dictionary.
For the second task, the subject was able to successfully create a new patient. However, when asked to return to the main page, the user hesitated because there was no button for doing this. They instead had to press "Back to Previous Page" multiple times.
In the third task, the user was able to successfully complete all subtasks. When determining what country the patient was from, the subject selected "Overview" only because it was the last option left, and not because it made intuitive sense. The user also appreciated the ability to add an encounter from the patient Encounters page.
Test Subject 2 - Small Screen Interface
In the first task, the user experienced no difficulties.
For the second task, the user overlooked adding an identifier type and location. Besides this, the subject did not experience any other difficulties.
In the third task, the user experienced no difficulties.
After testing, we asked the subject whether this interface had a lower learning curve than the first. The user believed that this was the case.
Discussion
We learned a great deal from the first paper prototype. In general, the interface was too cluttered, and the users had difficulty finding what they needed to complete each task. Adding encounters for a patient was another issue. Both subjects were unable to locate this feature without being prompted by the facilitator. Although our first user had difficulty finding the link for the concept dictionary at the top of the page, we believe that the concept dictionary does not need to be changed. The subject was most likely confused because it was necessary to look on the Administration page in the previous task. Finally, the Cohort Builder was easy to find and difficult to use for our user with medical records experience, but was difficult to find and easy to use for our second user with no medical records experience. This only tells us that using the Cohort Builder is confusing, but more tests would be needed to confirm why this is the case.
After the first round of testing we decided to focus on a smaller set of usability issues and to design an interface that could be used on a device with a small screen. We decided to focus on three tasks: creating patients, finding and managing patients, and searching for concepts. This allowed us to focus our usability testing and gather more useful feedback.
The second paper prototype did not give us as much usability information, but this was because the interface was found to be usable. We noticed that needing to click "Back" multiple times to return to the main page is a hardship for the user - a "Back To Main Page" button would greatly improve navigation of the interface. The name of the Overview button was also confusing - the user clicked Overview only because it was the only option left.
The second round of testing confirmed that this interface was much more usable than the first. It focused on only the tasks a practitioner in the field would need, though, so a different interface would be needed for a researcher or anyone removed from the actual site of the clinic. If we were to perform another round of testing, we would only change the few small issues mentioned in the previous paragraph.
Although this evaluation was informative, it could not cover all possible usage scenarios. For example, paper prototyping is not the same as clicking a touch screen or pressing small buttons - this may prove problematic for some users. Weather conditions in the area may also be an issue - low/high light conditions, high winds, extreme heat, etc. This evaluation also does not tell us if the interface would still be usable when the practitioner is trying to interact with patients. Most of these issues are not easily simulated, and testing the interface in the field would be the only way to see if they are indeed serious problems.
Appendix
Raw data from our testing can be found under Testing on the main OpenMRS page.
