Using Pictorial Action Instructions to Build a Basic Humanitarian Engineering Project: A Randomized Control Trial

Humanitarian engineering projects mitigate environmental hazards disproportionately affecting health in lowand middle-income countries. However, widespread literacy deficits can create barriers in training low-literacy adults to construct these projects, indicating a need for literacy-adapted training materials. A randomized control trial in rural Guatemala tested the usability of pictorial action instructions, compared to demonstration-only methods, in training low-literacy adults (N = 60; n = 30 per group) to construct a solar bottle bulb. Fourteen days after the training, participants individually constructed the solar bottle bulb. The intervention group received pictorial action instructions to guide them, and the control group completed construction based on memory. Usability was evaluated by measuring the effectiveness and efficiency of construction, as well as user satisfaction and self-efficacy levels. Effectiveness and selfefficacy were significantly better among those in the intervention group compared to the control group. Considering this, the findings support the use of pictorial action instructions in training low-literacy adults to construct humanitarian engineering projects. This method may allow more individuals in rural regions of lowand middle-income countries to successfully construct their own humanitarian engineering projects in a way that is sustainable and scalable. Further research is needed to test these instructions in different settings, on a larger scale, as well as to test the long-term effects of using pictorial action

of stoves and kerosene lamps can cause various respiratory infections 4,5,6 as well as explosions and burns 7 and elevated neonatal death rates. 8 The field of humanitarian engineering focuses on designing basic technologies to mitigate environmental hazards, in order to promote human welfare and social justice in underserved communities. 9 Humanitarian engineering projects have traditionally been decided upon and implemented by large wealthy organizations using a top-down approach that was often detached from the recipients of these projects, or by smaller non-governmental organizations, using a piecemeal approach, both of which often, " [lack] technical and systemic input... significant planning, and accountability." 10 Although with the advances in technology there are now more repositories of humanitarian engineering projects available to the public 11 , widespread literacy and technological deficits often found in LMICs can present barriers for implementing these interventions.
Currently, instructions for many of these projects are text-based or video-based. Text based instructions limit use among low-literacy individuals. Video-based instructions do not require participants to be literate; however, 4.8 billion people globally (predominately in LMICs) live without reliable internet access. 12 Therefore, projects are not always accessible to those needing these interventions. Adapting training methods to expand access to low-literacy individuals without the internet could help transition knowledge to action. This can promote the sustainability and scalability of such projects in underserved regions, leading to projects that last for prolonged periods of time and can be shared with others. 13 One approach to convey instructions to low-literacy individuals is through sequential pictures, known as pictorial action instructions (PAI). This method has been shown to increase comprehension of materials in low-literacy populations. Currently, existing applications of PAI have primarily focused on medication administration 14 , toy assembly 15 , and safety procedures. 16 To the authors' knowledge, there have been no formal studies conducted on the application of using PAI to grant low-literacy individuals access to usable instructions for constructing basic humanitarian engineering projects.
In this study, participants were trained to use PAI to construct a basic humanitarian engineering project known as a solar bottle bulb. Training assessment was based on a conceptual framework from Jakob Nielsen's 17 concept of usability, which assesses effectiveness, efficiency, and user satisfaction with completing a specific task. 18 Therefore, we measured each construction for how many steps were completed correctly, how long it took to build, and how participants felt about the training. A secondary concept of self-efficacy was also assessed, as how one feels about their own ability to complete a task positively correlates with usability. 19,20 Based on these metrics, it was hypothesized that the intervention group who received training with PAI, compared to the control group wo received demonstration-only training, would have better construction accuracy, shorter project completion times, higher levels of user satisfaction, and increased self-efficacy.

Setting Selection:
A randomized control trial was conducted in Guatemala, which is 127 out of 189 countries and territories in the world on the Human Development Index, and has substantial literacy deficits, particularly in rural regions. 21,22 The rural community selected is in a 44,000-person municipality in the western highlands, where nearly 40% of adults have not completed primary school. 23 The selected community is among the 25% of homes in rural Guatemala without electricity. 24

Project Selection:
As many homes in the area do not have windows, indoor lighting options and limitations for these individuals place them at risk for additional adverse health outcomes. Either, they increase the use of kerosene lamps and candles indoors 25 with poor ventilation, which can lead to burns as well as respiratory complications 26 , risk injury when navigating a poorly lit environment, or they must remain outdoors to do many daily tasks which can present risks and challenges, especially in the rainy season. With little formal tracking of health data in the area, the magnitude of these concerns is unknown, but the local community health workers state that they frequently see persistent coughs, as well as minor and moderate burns or abrasions that they associate with either kerosene lamps or no light source in the homes.
Given the community's remote location and lack of both financial and technological resources, an intervention was needed that could produce affordable, sustainable indoor light, and could be constructed using easy-to-find tools and materials. After a review of humanitarian engineering interventions designed to address this need, the solar bottle bulb was selected as it costs less than one U.S. dollar to make, can be constructed with commonly found scrap materials, has no maintenance cost, and has been shown to have positive outcomes in similar settings. 27 This structure is made from metal lamina, a plastic bottle, water, bleach, and sealant; which when placed in a lamina roof can refract the sun's light to mimic a 55-watt bulb, potentially lighting a 40 square meter space. 28 This would provide a sustainable, affordable, daytime light source that would allow individuals to stay in their home to do their daily tasks, without increased health risks.
Step-by-step instructions for constructing a solar bottle bulb already existed, but were either video-based or required somewhat advanced literacy levels, both of which preclude their use in this community. 29 Therefore, best practice guidelines for including objects, sequences, and people in PAI illustrations, as well as simplified text were applied to these existing instructions to develop PAI for constructing a solar bottle bulb ( Figure 1). 30,31,32 The PAI were tested during a single-day feasibility study in Utah, and participant feedback guided revisions for the final PAI used in this study. 33

Sampling and Recruitment:
Participant recruitment was done via an announcement made by the local health worker at a community meeting. Attendees were invited to participate in a study to learn how to construct a solar bottle bulb if they met the following inclusion criteria: 18-64 years-old, native Spanish speakers, no physical impairments precluding the use of basic hand tools 34 , less than a 7th grade education, and able to correctly identify at least 80% of preselected basic hand tools commonly used in rural building 35 . Power analyses were conducted for a student's t-test and multiple regression for each hypothesis, with an alpha of .05 and power of .85, using effect sizes from prior studies on similar outcome variables. 36,37 The results indicated a desired sample size of 54 participants. To account for possible attrition, 60 participants were recruited, who were then randomly divided into two 30-person groups.

Study Design:
In a randomized control trial, usability was compared between two methods of training lowliteracy adults to construct a solar bottle bulb. The intervention group received PAI-based training, and the control group received demonstration-only training, a common method used when servicebased groups visit LMICs to construct projects. 38 The PAI included 18 steps for project construction and installation as well as a tools and materials page.
Both trainings were conducted in the community center, by the principal investigator, with a local Spanish language interpreter. The intervention group received construction training 24 hours after the control group. Training included a description of how the bulb functions, the tools and materials used, and a construction demonstration. Following the demonstration, each participant completed a text-based literacy assessment and a visual literacy assessment. The intervention group was provided with PAI to follow during the demonstration, and all PAI were collected afterwards to decrease intervention contamination.
Participants returned to the community center 14 days after each training to individually construct a solar bottle bulb, without outside assistance. Participants were provided with the tools and materials necessary for the project's construction, and were assigned pre-designated work areas surrounding the community center. The intervention group was given PAI to use, and the control group completed construction based on memory. Participants were instructed to raise their hand upon completion of their solar bulbs, and their construction times were recorded. Upon completion of construction, each participant brought their finished product to the investigator for inspection. Finally, participants took their assessment form to the interpreter and provided feedback regarding user satisfaction and self-efficacy.

Effectiveness:
Effectiveness was calculated by dividing the number of steps successfully completed by the total number of task steps (12), then multiplying this number by 100. As the exact sequence of steps can vary somewhat and still result in successful construction, effectiveness was determined through inspection of the final product. Results could range 0-100%. The full instructions have 18 steps, but the last six were excluded as they focus on bulb installation, not construction. However, installation instructions were explained after the study to increase project sustainability and scalability.

Efficiency:
Efficiency was calculated as the time lapse between task initiation and completion. All participants in each group had the same initiation time: when they were instructed to collect tools and materials to take to their work areas. Task completion times were recorded, to the nearest whole minute, when each participant finished the 12th step of construction.

User Satisfaction:
Satisfaction was calculated using a one-time administration of the validated three-item After-Scenario Questionnaire (ASQ) 39 , commonly selected in usability testing. The ASQ assessed overall satisfaction with the project based on ease of construction, perception of construction times, and the support information received. Each item was rated between 1 (strongly disagree) and 5 (strongly agree). The ASQ was administered verbally, with responses recorded by an interpreter who was trained to use this tool.

Self-efficacy:
Self-efficacy was evaluated once, upon construction completion, using a ten-item adaptation of the Online Learning Value and Self-Efficacy Scale, designed to evaluate both learning value and self-efficacy. 40 Ten of the initial eleven items were applicable, and were thus adapted so that "online course" was replaced with "project" in the first five items, evaluating learning value, and "method of instruction" in last five items, evaluating self-efficacy. Each item was rated between 1 (strongly disagree) and 5 (strongly agree). The scale was administered verbally, and responses were recorded by an interpreter who was trained to use this tool. Text-based literacy and visual literacy were assessed as either, or both, could have a confounding influence on outcomes. Text-based literacy was measured using an adapted Native Language Literacy Screening Tool-Spanish 41 , widely used to determine literacy levels in an individual's primary language. Visual literacy was assessed with the Santa Barbara Solids Test-3rd Edition 42 to measure spatial reasoning.

Data Analysis:
Data analysis was done with R, version 3.4.2. Each hypothesis was analyzed using a student's ttest, as well as a Wilcoxon rank-sum test, based on abnormal distribution within each outcome variable. Furthermore, each hypothesis was analyzed via multiple linear regression, adjusting for the two covariates.

Results:
Significance of the student's t-test and that of the Wilcoxon rank-sum tests was consistent for each outcome variable. Furthermore, multiple linear regressions adjusting for text-based literacy levels and visual literacy levels did not change the conclusions, and were not associated with any of the outcome variables (Table 1). It was hypothesized that participants in the intervention group, using PAI would have more efficient and effective project construction, and would report higher levels of both user satisfaction and self-efficacy than participants in the control group who received demonstration-only training. Effectiveness was significantly better in the intervention group (95.6% versus 80%, p < .001), indicating a higher rate of accuracy in solar bottle bulb construction for those using PAI compared to those constructing the project based on memory. As effectiveness was determined via inspection of the final product, step order may have varied. The first five steps appeared to have been completed accurately by all participants in both groups. Steps six through eight had the largest amount of variation between the two groups. Within the PAI group, one participant omitted step six, and one participant inaccurately placed the lamina on the bottle in step eight, with the majority of the bottle above the lamina level. Within the control group; step six was omitted by eight participants, two participants placed the lamina upside down from the instructions in step seven, and four participants placed the lamina on visible inaccurate regions of the bottle in step eight, including three bulbs with more than 75% of the bottle above the lamina level. It appears that all participants in both groups completed steps nine through eleven completely, and only one participant, who was in the control group, omitted step 12.
In addition, self-efficacy levels were significantly higher for those using PAI (4.6 versus 4.2, p = .003) than those without. This indicates that the use of PAI is associated with participants feeling more confident in their own ability to construct this project, as well as similar basic projects. The prompts that indicated the largest difference, with the intervention group reporting higher levels of agreement than the control group, were as follows: 5) Even in the face of technical difficulties, I am certain I can learn the material using this method of teaching; 7) I am confident I can learn without an instructor present; and 10) Even with distractions, I am confident I can learn lessons using this method of instruction.
There was no statistically significant difference between the two groups for efficiency (intervention group = 30.8 minutes, control group = 33.0 minutes, p = .31). Also there was not statistically significant difference between the overall user satisfaction among participants within the two groups (intervention group = 4.4, control group = 4.5, p = .46). However, responses to the prompt, "Overall, I was satisfied with the support information received when completing this task" indicated higher satisfaction among those using PAI, while the prompt, "Overall, I am satisfied with the amount of time it took to complete this task" indicated higher satisfaction among those in the control group.

DISCUSSION
The results indicate that using PAI leads to significantly better outcomes for effectiveness and selfefficacy. Both of measures, in addition to being statistically significant, are practically significant in application. When assessing what is needed to successfully replicate this product in the future, the ability to effectively construct an accurate project, and the belief in one's own ability to complete the project are both key.
Construction effectiveness is the most crucial aspect of usability in practical application, as final product accuracy can influence the functionality and sustainability of the project to different degrees. Some inaccuracies in sequence, such as adding the water before the bleach instead of after, would result in no difference in the final product. Other inaccuracies, such as not properly applying the sealant around the lid, or not including bleach, would likely result in an initially functional solar bottle bulb, but could deteriorate slowly over time if the water evaporates or becomes cloudy. However, errors such as cutting the hole or tabs larger than the bottle's diameter could have an immediate effect on the safety of the product, leading to increased hazard in the home, should the bottle fall from the ceiling. Particularly in areas where resources are scarce, it is important that projects be well-constructed to promote project sustainability, improving the health of underserved populations in LMICs.
Based on responses to a specific item within the self-efficacy scale, those who used PAI also reported feeling more capable and confident, even if faced with technical glitches, distractions, difficult tasks, or presented with new projects. This feedback reflects practical significance, as it is likely that individuals replicating this project in the future could face technical glitches and distractions during construction. Confidence with constructing and potentially replicating the project can potentially lead to increased sustainability, as individuals may feel more comfortable with making adjustments to maintain the longevity of bulb function, although it should be noted that there is little risk for needed repairs beyond the potential need for reapplication of silicone if the initial application dries and cracks. Also, confidence in one's ability is likely to increase the changes that they would in turn show others how to construct the same project, leading to increase scalability.
Project completion time was only slightly better when using PAI. One possible explanation for this outcome is that both groups were instructed to focus on construction quality, not speed. In addition, the lack of variation in times is partially attributed to the short overall duration of construction as, even with the inclusion of outliers, there was only a 38-minute difference from the slowest to fastest completion time. In larger projects, the completion time may vary more drastically between the two different methods.
User satisfaction results did not differ when using PAI, but a ceiling effect was noted. Each item within the ASQ received a mean score exceeding four out of five in each group. The health worker explained that in this community, where resources and opportunity are scarce, stating dissatisfaction would have been perceived by participants as ingratitude, leading to apprehension that the community would then be disqualified from future projects and opportunities. Therefore, based on cultural perceptions, the results may not reflect participants' true feelings.

Limitations:
Due to time constraints, the period between training and participant construction was limited to fourteen days. A longer period between demonstration and participant construction could provide a clearer indication of the long-term differences in outcome based on training method. Furthermore, constraints in time, geography, and human resources led to evaluation of only the twelve construction steps and not the six installation steps. Although this was consistent between both groups, evaluating all of the PAI steps could provide greater variation, particularly in effectiveness and efficiency. Although the final steps were not included in the data gathering, after completion of the study a demonstration was provided to the community members, and further training was provided to a selected group, followed by successful return demonstration of installation. Furthermore, as financial constraints make it highly unlikely for the investigators to do follow-up inspections of the installed solar bottle bulbs, the same selected group that received installation training also committed to providing follow-up inspections, particularly if problems were suspected or reported.
Also, based on these same constraints, particularly the limitation of having only one interpreter available, it was not feasible to collect qualitative feedback during this study. Also, a limitation is present within the interpretation of the self-efficacy and user-satisfaction tools. Both tools were validated prior to use in other studies, and a bilingual interpreter verified that the translations were accurate. However, as validation was not able to be tested in a rural Guatemalan community prior to the study, there is the potential that cultural context and subtle wording variation through translation could lead to some shift in participant perceptions. Finally, although not specifically a limitation to the study, it should be noted that while participants completed construction individually for the study, construction within community settings is often also done in small groups, which could influence outcomes of further construction using the PAI.

Implications:
Providing a training method with better usability for low-literacy individuals can help remove a barrier that currently exists between many basic humanitarian engineering projects and the individuals who need to access them to mitigate environmental hazards. As the application of PAI within this study resulted in better project accuracy and self-efficacy levels, it is conceivable that this approach could also be implemented in similar trainings for other basic humanitarian engineering projects. By removing the literacy barrier, individuals who may have previously been unable to access these projects can now learn to construct them by themselves, helping promote sustainability and scalability of such projects in underserved regions.

CONCLUSION AND RECOMMENDATIONS
Using PAI to train low-literacy individuals to construct a solar bottle bulb was found to result in better construction effectiveness and self-efficacy than training individuals using only demonstrations. Although efficiency and user satisfaction levels were not significantly different between the two groups, as they are key components of usability, they should be reassessed in future usability studies.
To further determine the usability of PAI to train low-literacy individuals to construct humanitarian engineering projects, multiple future studies are recommended. First, it would be of value to test PAI for other similar, basic projects. Second, usability testing could be expanded to a larger scale. Third, testing in different settings, such as within urban slums or in other regions of the world would be beneficial. Fourth, better understanding of cultural perceptions of this project could be assessed using qualitative feedback on how the project impacts social, environmental, and economic aspects of individual's lives. Finally, it is important to evaluate long-term sustainability and scalability, including tracking the replication and dissemination of the project by participants in each group.