Development and Testing of Pictorial Action Instructions for a Basic Humanitarian Engineering Project : A Feasibility Study

The aim of this study was to develop, refine, and assess the usability of pictorial action instructions (PAI) in training low-literacy individuals to build a basic humanitarian engineering project. Health in lowand middle-income countries is disproportionately affected by environmental hazards, often mitigated by such projects, but widespread literacy deficits often necessitate literacy-adapted materials for training. Project-specific PAI were developed, refined, and tested, with the intent of later testing them abroad. Development was based on best practice guidelines for illustrations and easy-to-understand text. The PAI were then reviewed by literacy specialists for content and formatting. Next, a feasibility study was conducted in Utah with low-literacy immigrants who were trained to use the PAI and provided with tools and materials to individually construct the project. Usability metrics gathered included construction efficiency and effectiveness, user satisfaction, and selfefficacy. Effectiveness was 100% and user satisfaction and self-efficacy were both high. Participants provided feedback to improve the PAI’s clarity by changing item proximity and combining or separating steps, and revisions were made. In future PAI development, it would be beneficial to include low-literacy individuals when making the initial sketches to enhance sequence clarity before finalizing the illustrations. Index Terms – humanitarian engineering, literacy deficit, pictorial action instructions, usability


INTRODUCTION
Health in low-and middle-income countries (LMIC) is disproportionally affected by environmental hazards, which often lead to adverse health outcomes and increased mortality rates. 1 Projects have been designed to reduce these hazards, but widespread literacy deficits in the most underserved communities often present a barrier for training individuals in project construction.Therefore, literacy-adapted training instructions are necessary to grant broader access to information, for those with the most need for these interventions.One way to increase understanding for low-literacy individuals is by providing sequential, picture-based instructions, known as pictorial action instructions (PAI).In order to ensure that the instructions are usable, they must be designed based on best practice guidelines and piloted before further testing abroad. 2

OBJECTIVE
The aim of this feasibility study was to develop, refine, and assess the usability of PAI to train low-literacy individuals in building a basic humanitarian engineering project.The project selected was the solar bottle bulb, which is a simple structure that uses refraction of the sun's rays to light indoor spaces during the day.Usability of the instructions was assessed by measuring effectiveness and efficiency of project construction, participants' levels of user satisfaction, and self-efficacy. 3he instructions underwent multiple revisions to improve clarity and participant comprehension, in order to develop PAI that can be used in an upcoming randomized control trial in rural Guatemala.

BACKGROUND
Environmental hazards, such as unsafe water, poor sanitation, indoor smoke inhalation, and unintentional acute poisoning disproportionately have an adverse effect on health, particularly in low-and middle-income countries (LMIC).These hazards can lead to severe health conditions such as diarrhea or respiratory issues, which are responsible for millions of deaths annually. 4umanitarian engineering projects have been designed to mitigate these hazards, and databases of these projects are present online through groups such as Engineering for Change. 5However, many instructions are heavily text-based, and overall literacy rates in LMICs only range from 57-74%, with rates decreasing sharply in rural regions, where the projects are most needed. 6Video options are also available for some projects, but are not accessible in many underserved regions, as nearly 33% of the world lacks mobile broadband or Wi-Fi, and the majority of the 4.8 billion people without reliable internet live in LMICs. 7Therefore, instructions must be adapted to increase usability for those with the greatest need for these projects.
An evidence-based method for addressing literacy barriers in training is the use of pictorial action instructions (PAI). 8,9,10,11Rather than text-based literacy, PAI are a series of sequential pictures used to convey information and communicate ideas 12 .The CDC and the Robert Wood Johnson Foundation recommend PAI for conveying information to low-literacy individuals. 13,14lthough no large-scale studies have been identified that test PAI use in assembly instructions, results from small-scale studies support a high likelihood of the method's success in larger applications. 15,9akob Nielsen's concept of usability will provide the conceptual framework by which success will be determined. 16The International Organization for Standardization defines usability as an assessment of effectiveness, efficiency, and user satisfaction related to achieving a specific goal in a specific environment. 17Furthermore, an assessment will be done for self-efficacy, which is the degree to which one believes in his or her ability to complete a task.This concept has been found to have a strong correlation with usability. 18

Preparation
The PAI were developed based on best practice guidelines for illustrating people and objects, as well as the inclusion of text in easy-to-read materials.A panel of literacy specialists reviewed the instructions to ensure that all criteria for content and formatting were met, and revisions were made accordingly.A feasibility test was then conducted, using the revised instructions, to measure usability and further improve clarity within the illustrations.

Humanitarian Engineering Project Selection
Project selection was based on an assessment of environmental hazards in a community in rural Guatemala.Community selection was based on literacy deficits, with more than 40% of adults in the area never completing primary school 19 , making it an ideal location to test PAI.Furthermore, the community is among the 25% of Guatemala's rural population that does not have electricity. 20ore than 80% of homes in LMICs without electricity, including those within this community, use kerosene lamps or candles indoors for light. 21,22The indoor use of candles or kerosene lamps increases risk for indoor smoke inhalation and burns. 23,24ased on identification of this environmental hazard, the solar bottle bulb was selected for construction as it can provide a safe and affordable daytime indoor light source for those without electricity. 25Each bulb costs less than one U.S. dollar to make, and mimics a 55-Watt bulb 26 , potentially lighting a 40 square meter space. 27fter attaining legal permission from Liter of Light, previously produced instructions for the solar bottle bulb 28 were converted into PAI using best practice guidelines for illustrating, sequencing, and easy-to-read wording.

Instruction Best Practice Guidelines
As the PAI included objects, sequences, people, and words, guidelines for each were followed in designing an initial draft of the instructions.
The guidelines used for the inclusion of objects and sequences in PAI come from the work of Schumacher, 29 who identified three aspects of illustration that optimize comprehension.First, illustrations with simple black and white line drawing, and no shading, are most clearly interpreted.Only necessary items should be included in these line drawings, eliminating decorative or distracting items.Second, to increase comprehension of the transitions between sequential steps, the vantage point must remain consistent, unless intentionally indicating a need to shift perspectives.Finally, comprehension of individual steps and transitions is best when the size of the objects do not change from frame to frame, and the only object that varies is the object of focus.
Guidelines for the inclusion of people in PAI are based on multiple studies conducted by Haaland, Akogun, and Oladepo in various LMICs. 30Their findings include the importance of developing the simplest form that is personally relatable to those who will use the instructions, including features such as hairstyle, jewelry and local dress, or architecture that is similar to that seen in the community.However, too many details in features or architecture are seen as distracting and exclusive of those who have slightly different features or architecture, particularly in societies where these details distinguish between communities or tribes.
Guidelines for wording in the instructions originate from the CDC's Simply Put: A Guide for Creating Easy-to-understand Materials. 31This document provides considerations for the incorporation of text in materials for low-literacy individuals, including font sizes and styles, as well as layout.Recommendations include the use of serif font with emphasized words in bold and avoiding the use of fancy lettering, all caps, italics, or underlining.Furthermore, the CDC recommends conveying action through pictographs with necessary indicator words, instead of text, when possible.Finally, testing of the readability level must be completed to ensure the wording is at an appropriate literacy level for those who will be using the materials.As the instructions are intended for Spanish speakers, the Spanish Lexile Analyzer was selected to test readability. 32

Literacy Specialists Review
The initial draft of the Spanish-language instructions was evaluated by a Spanish-speaking literacy education specialist, using the CDC Checklist for Easy-to-understand Print Materials to ensure that criteria for both content and formatting were met. 30The specialist concluded that all criteria were met, but recommended consulting three other, literacy specialists for further review of formatting criteria.Feedback from these specialists indicated that the materials met "easy-tounderstand" criteria for content and format, but two specialists stated that words were not necessary in the sequential instructions, as the illustrations clearly conveyed the message.When this feedback was shared, all four specialists agreed that the text was redundant; thus, the text was removed from the sequential illustrations.Words remained on the page to indicate the project's necessary materials and tools to ensure comprehension of the items seen in the illustrations.

FEASIBILITY STUDY
The revised PAI (Figure 1) were tested in a single-day feasibility study at a community center in Utah in March 2017.The study focused on assessing the usability and improving the clarity of the PAI before use in a randomized control trial in rural Guatemala.

Sample
As per guidelines set forth for piloting usability testing, recruitment was done to select five participants 33 via an announcement made by a city councilmember during a community meeting over Hispanic affairs.He informed the attendees that five people, meeting the inclusion criteria below, were needed for a one-day study to learn to build a basic project, and that no construction background was necessary.Interested adults were instructed to contact a Spanish interpreter hired by the investigator.The first five individuals to call, who also met the criteria, were selected.
All participants met the following inclusion criteria: 18-64 years-old, native Spanish speakers, no physical impairments that would preclude the use of basic hand tools, less than a seventh-grade education, and able to correctly identify at least 8 of 10 pictures of basic hand tools used in rural building by name and function.This sample was as similar as possible to those who will participate in a future randomized control trial in the rural Guatemala.

Methods
To mimic the randomized control trial as closely as possible, a demonstration was given by the investigator, in conjunction with a Spanish-English interpreter, to show participants step-by-step how to construct the solar bottle bulb using the PAI.After the demonstration, there was a one-hour break, to decrease participants' reliance on immediate memory recall for construction, and increase reliance on the instructions for guidance.Following the break, participants were provided the tools and materials necessary for the project, as well as the PAI, and were asked to construct the project individually.Initial plans to video-record construction were altered, as all of the participants were undocumented immigrants and stated that, with the current sociopolitical climate, they feared being recorded could increase their risk of deportation.Monitoring, in the absence of videorecording, was done by the investigator and an interpreter trained to observe project construction.

Usability Metrics and Results
Testing of the PAI was done by assessing the three components associated with the concept of usability: effectiveness, efficiency, and user satisfaction. 34The secondary concept of self-efficacy was also measured, as evidence indicates that usability problems can decrease self-efficacy, and successful step completion maintains or increases self-efficacy levels. 35ffectiveness was determined by calculating the completion rate (# task steps successfully completed / # total task steps X 100%) of the twelve steps involved in this task.All five participants were able to achieve 100% effectiveness by completing all twelve steps correctly.Furthermore, efficiency was calculated by recording the task time (start time subtracted from end time) rounded to the nearest whole minute.Construction times ranged from 25-40 minutes.
User satisfaction was calculated using a one-time administration of the validated three-item After-Scenario Questionnaire to assess overall satisfaction with completing the project. 36dministration and response recording were completed verbally by the interpreter, who was trained by the investigator to use this tool.All five participants rated each of the three items a 5 (on a scale from 1 [strongly disagree] to 5 [strongly agree]).
Self-efficacy was measured using an adaptation of the validated Online Learning Value and Self-Efficacy Scale, administered upon construction completion. 37Ten of the eleven original items were applicable, and were thus retained and adapted.The word "project" replaced references to an "online course" in the first five items and "method of instruction" in the remaining five items.Administration and response recording were done verbally by the interpreter, who was trained by the investigator to use this tool.Each item's rating averaged between 4.8 and 5 (on a scale from 1 [strongly disagree] to 5 [strongly agree]).

Instruction Clarity
Upon completion of the project, the interpreter asked the participants five questions about clarity of the instructions.Although all five participants stated that, overall, the instructions were clear to follow; two recommendations were made to improve the proximity of items in illustrations for steps two and seven.For step two, participants requested clarification that the second circle should be drawn within, not around, the first circle.As depicted in Figure 2, illustrations for step seven were revised after a request was made to illustrate the bottle being placed in the sheet metal from below, not above.Furthermore, participants felt that step eight, applying rubber sealant around the FIGURE 2 STEP SEVEN ORIGINAL AND REVISED bottle, was unnecessary and could easily be included with step twelve, applying rubber sealant around the lid.All other steps and transitions between steps were reported to be clear.Participants additionally stated that the materials page and illustrations clearly conveyed all materials and tools.

Participant Statements
In addition to the formal data gathered on usability and instruction clarity, the participants provided positive feedback about their involvement in the study.One woman, unable to read or write anything more than her name and whole numbers, expressed overwhelming feelings of empowerment and gratitude as she, for the first time in her life, could independently follow a set of written instructions.All five participants asked to take their completed projects home.Although they all have electricity in their homes, participants stated that they were incredibly proud of their finished products and wanted to show them to friends and family.Moreover, three participants asked permission to take pictures of the instructions on their phones and planned to send the pictures to family members in rural areas of Mexico.All of the participants felt confident that, with a copy of the PAI, they could reconstruct the project in the future, even years later, and could teach others to successfully construct the project, as well.

Limitations
The inability to video-record participants meant that construction effectiveness observations had to be done in real time.Accuracy in observation could have been increased by the ability to watch the construction video multiple times, focusing on a different participant each time.Furthermore, although the inclusion criteria were the same for the participants in Utah as those for the future study in Guatemala, those in the feasibility study all lived in a suburban area, whereas the study in Guatemala will be conducted in a rural community.Finally, as no required gender distribution was indicated in the study design, the first five individuals to contact the interpreter were all women, resulting in no gender variation within the study.

Implications
The process of development, revision, and testing used in this study produced PAI that are anticipated to be useful in training low-literacy individuals in LMICs.One recommendation for future application would be to include low-literacy individuals earlier in the process, when doing preliminary sketches, to determine the clarity and comprehension of items, steps, and sequences before finalizing the illustrations.Considering this, the following steps are recommended when creating PAI for future projects: 1. Conduct an environmental hazards assessment in the community 2. Select a humanitarian engineering project that addresses the hazard (taking into account availability of tools and materials) 3. Use best practice guidelines to sketch initial drafts of PAI a. Have low-literacy individuals assess step sequencing and item proximity for clarity b.Use a text analysis tool to determine literacy level of words in PAI 4. Finalize PAI using feedback from low-literacy individuals and text analysis tools 5. Test the usability of the PAI for the specific humanitarian engineering project Following these steps, PAI can be created and implemented to enhance training comprehension among low-literacy individuals.These PAI can be used by service learning groups who travel abroad to construct basic humanitarian engineering projects.By using these instructions during training, and leaving copies in the communities after the group finishing their project, those living in the community will have a resource that can promote both project sustainability and scalability.Individuals could use these instructions to help replicate a project in other areas, and potentially share the information with surrounding communities.These PAI can also be shared with, and used by, local health workers or non-government organizations (NGOs) that oversee communities or larger districts in areas that may be in need of such projects, even in regions with literacy deficits.

CONCLUSION
Evidence provided through the feasibility study indicates a strong potential for PAI to be used in training low-literacy individuals to construct basic humanitarian engineering projects.Construction effectiveness was 100%, and user satisfaction and self-efficacy were both high.Additionally, feedback from the participants led to revisions that are anticipated to improve the PAI's clarity and comprehensibility.Thus, the final version of the instructions should be ready for use in a randomized control trial in which usability will be compared between training with PAI and demonstration-only training, the most common method of instruction implemented for these projects. 38he process of developing, refining, and testing the PAI for the solar bottle bulb resulted in recommended guidelines that can be used to create PAI for other humanitarian engineering projects in the future.If usability of these instructions can be established for training low-literacy individuals in LMICs to construct their own projects, then this approach can potentially be empowering, enhancing the sustainability and scalability of such projects in the world's most underserved regions.