Reflection-based assessment of service learning in undergraduate engineering

Virtually all engineering education organizations from around the world agree on the critical awareness of the societal context of engineering for graduating engineers. Service learning is one viable way that can bring about students’ awareness of their role in society. However, the body of literature addressing service learning as a pedagogy and strategic ways of implementing it in the already-full engineering curricula, as well as its assessment is relatively small. This paper describes a concise engineering service-learning component in a typical heat transfer course for mechanical engineering students. The service-learning component was used to probe the students’ ability to a) articulate the societal context of engineering, b) explain the importance of pro-active community service, and demonstrate an inclination to continue such service in the future, c) exhibit an appreciation of communication with non-engineers and finally, d) challenge some of the students’ stereotypes regarding other members of the community. Reflections were conducted by the students when answering a set of carefully-phrased questions addressing the above four issues. The responses of all students were analyzed as explained in this paper. The recorded benefits of service learning are described and can be expected from similar service-learning components in other engineering courses. Index Terms – communications, reflection, service learning, societal responsibility


INTRODUCTION
The understanding and awareness of graduating engineering students of the societal context of engineering has become a sought after outcome and skill of engineering programs around the world.Pascail 1 stated that engineers must not separate technical and human problems but rather work on and think about them together.Ravesteijn et al. 2 emphasized that future engineers must have the ability to understanding social dynamics of technology and to communicate facts, values and emotions on different levels.According to Santander Gana and Trejo Fuentes 3 , technology is a human practice and a social activity that develops as a result of various intrinsically-woven socio-cultural circumstances.Engineers Australia 4 and many new science and technology policies in Europe 5 emphasize the importance of societal values connected to engineering.In the US, outcome '3h' in the Accreditation Board for Engineering and Technology's (ABET) Engineering Criteria 2000 deals with the broad education necessary to understand the impact of engineering solutions in an environmental and societal context 6 .The National Academy of Engineers stressed that the years between now and the year 2020 will require that engineers strengthen their leadership role in society. 7The Academy presented the grand engineering challenges, which were not engineered devices, but nearly all address complex social issues that use technology, but cannot be solved by technology alone. 8raditional engineering programs are usually more focused on technical development and not on preparing socially-responsible engineers with a strong sense of citizenship. 9Service learning is a very viable solution for addressing this issue.Service learning is a pedagogy that provides students with structured opportunities to learn, develop and reflect through active participation in community projects. 9It is an opportunity to learn numerous non-technical skills. 102][13][14] Tsang 15 classifies service learning as experiential education, and says that it has elements of drama and dilemma, just like the real world.
7][18] Some of the published literature regarding engineering service learning lack any reflective analysis and assessment of service learning impact on students, 18,19 while more recent work 20,21 includes these key items.
This paper provides a brief description of an engineering service-learning component and its implementation in a senior heat transfer class.In addition, it presents, assesses and analyzes the effects of the service component on students' stereotypes, societal awareness and communication.The paper also presents some of the students' opinions and observations.

BRIEF DESCRIPTION OF THE SERVICE-LEARNING ACTIVITY
A service learning activity was undertaken by undergraduate mechanical engineering students in their heat transfer course at the University of Detroit Mercy.Similar service-learning assignments have been implemented for the past few years.By the end of 2007, about 55 students had participated.These service-learning experiences were assessed and analyzed from a variety of perspectives. 22In 2008, a total of fourteen students participated in this assignment, while in 2009 only four students participated.In the class of 2008, the service-learning component was worth a bonus of 5% of the final grade of the course which explain why most of the class participated (14 out of 16 students).In 2009, the service-learning component was voluntary and did not carry any bonus points.It should be noted that in 2009, 15 students out of 25 signed up for the service component, but only four completed it.The rest of the students did not want to could not handle he added burden of scheduling and following up with the coordinator.In 2010, all 14 students signed up for the service component, but only 13 completed the activity(one could not schedule the home visits).For the cohorts of 2008, 2009 and 2010, a clear outcome-based assessment was designed and conducted and the results and conclusions of the assessment are presented here.The service-learning activity was conducted by the students with help from the Warm Training Center. 23This center is a not-for-profit organization that has been serving the Detroit area for over 20 years.One of the center's programs helps homeowners save on heating fuel and electricity bills by installing plastic storm windows, rope caulk, and door sweeps which reduce heat loss and thus reduce heating cost.There was a need for assistance of some residents in installing the materials, mostly due to a combination of old age, disability, and illness.Student teams of two went to the homes to install the materials and took window and door measurements which they then used in calculating the reduction in heat loss due to modifications and the resulting monetary savings on monthly heating bills.There was a direct and clear link between these calculations and some of the heat transfer course technical outcomes.More details about the service-learning program are given in a previous publication. 22

REFLECTION QUESTIONS AND ASSESSMENT OF OUTCOMES
Reflection is a critical aspect of service learning and differentiates it from simple community service. 9,24It promotes learning about larger social issues. 15The students usually reflect on how issues of race, ethnicity, socioeconomic class and other social factors might affect an engineering problem. 13,24The act of reflection develops evaluative thinking skills 15 and reflective judgment. 13flection and its analysis are relatively new in engineering education. 13,24Sometimes it can become extensive. 25Efficient, concise and easily-implemented reflective components and assessment rubrics can be extremely useful for assessing the impact of service-learning on engineering students.
In this project, the students were required to reflect on their experience considering several reflective questions.The two kinds of questions "reflection questions (RQ)" and "multiplechoice questions (MCQ)", along with the targeted outcomes, are shown in Table 1.There were a total of 14 reflections submitted by the 2008 students, four by the 2009 students and 10 by the 2010 students, which brings the total responses to 28 over the three-year period.The three sets were analyzed by the authors.The assessment was outcome-based as explained below.

Outcome 1:
To gauge student's ability to explain how engineering solutions can improve people's lives, the students answered reflection question 1 (RQ1) and multiple-choice question 1 (MCQ1) of Table 1.For MCQ1, the vast majority of students (22/28) either agreed or strongly agreed that the materials installed significantly improved the lives of home's residents.Two students disagreed while the other was neutral.As for RQ1, the two authors independently rated the explanation of each student.The rating systems ranged from 1 to 5, where a score of 5 indicated an excellent explanation.The scores given by the two authors were averaged for each student.The scores ranged from 1 to 5, with an average of 3.6 or 72.1%.An example of an excellent explanation (5/5) read: "I think the materials will improve the quality of their lives because it will save them money that they are normally wasting on heat bills.With this extra money they will be able to better take care of their family.Also it helps to create a warm area in the house for the family to spend time in, when they may have previously been avoiding the area."While a relatively poor explanation (2/5) read: "Yes, I think that they will help them to save money, as well as gave them ideas as to how they can save money other places.The homeowners seemed excited about doing things in their house to help them out." It should be noted here that probing students with such reflection questions and asking them to explain and elaborate triggers in them the activity of reflection-something not usually done when doing purely engineering work whether it is a homework exercise or a design project.To highlight this we looked for positive, neutral, and negative indicators in the students' responses to RQ1.

FIGURE 1 TOTAL NUMBER OF POSITIVE, NEUTRAL AND NEGATIVE INDICATORS ABOUT THE SERVICE LEARNING
Outcome 2: Outcome 2, as stated in Table 1, has two parts: explaining the importance of community service and showing an inclination to continue it in the future.The first part was assessed by RQ2, while the other parts were assessed by MCQ2.In response to RQ2, a very high portion of students (25/28) said yes, their participation in the service-learning activity was worthwhile.Similar to rating of RQ1, the explanation part of RQ2 was rated by the two authors and the two scores for each student were averaged.The ratings ranged from 2 to 4.5, with an average of 3.3 or 65.8%.An example of an excellent explanation (4.5/5) of as to why the service activity was worthwhile read: "My participation was very worthwhile.I have always enjoyed doing service and I think it should be an active part of everyone's life.I think this experience was unique in the fact that I could apply first hand things I am learning in class to help someone's life.I could also then calculate how much money we were saving the home owners."A poor explanation (2/5) read: "It was certainly interesting and I even learned things about how to combat my $400/month heating bills." To assess the inclination to continue service in the future, which is the other part of this outcome, the responses to MCQ2 were analyzed.A moderate majority of students (18 out of 28) indicated that they either agreed or strongly agreed that they were motivated to make community service a vital part of their future.Nine of the remaining students were neutral while one student disagreed.These numbers are reflected in Figure 2.

Outcome 3:
The third outcome targeted by the service-learning component was to challenge students' stereotypes regarding others.This outcome was assessed by RQ3 and MCQ3 of Table 1.In response to MCQ3 only nine students agreed or strongly agreed that one or more of their preconceived notions regarding low-income people were challenged as a result of participating in service learning.Five students disagreed, while 14 were neutral.
In response to the RQ3, students wrote about several stereotypes.A word of caution is in order here.Some of the students appeared not to have a clear understanding of what a stereotype exactly means while others could not communicate their answers regarding stereotypes.Several students' responses fell under this category and were not discussed any further by the authors of this paper.An example of the former reads: "Well there are mainly African Americans in Detroit.It is a demographic fact.The point that African Americans have been economically suppressed because of the color of their skin from 100's of years ago has held them back."An example of the latter reads: "I think the hardest thing for me was to see how vulnerable she was.Her front door knob didn't work and the door basically just rested shut.The huge holes/ breaks in her windows let in serious drafts and her house was extremely filthy.She had 5 children and only one of them came around ever to help her in her weakened condition.The stereotype that someone could "get out" of that situation was clearly wrong."One of the students said he was not sure while another indicated that he was free of stereotypes.A third said he was "unable to encounter any stereotypes."Table 3 lists the stereotypes that were challenged and others that were reinforced, as indicated in the students' responses.Students' responses were diverse.The same stereotypes could be challenged by one student and reinforced by others.In total seven of the students reported that they changed one or more of their stereotypes as a result of participating in the service.On the other hand, three students had some of their stereotypes reinforced.

Outcome 4:
The service-learning activity was designed to place engineering students in situations where they had to interact with non-engineers.In response to MCQ4, 22 of the 28 students found such conversations to be valuable while five students were neutral and one disagreeing that the conversation was valuable.RQ4 elicited explanation and elaboration on this issue.The explanations of the students were separately rated by the two authors.Scores ranged from 1.5 to 5 out of 5.There were seven students scoring 5, exhibiting a high level of appreciation and understanding of communication with non-engineers.The overall average for students who wrote about communication was 67.9%.As an example of an excellent explanation the following was stated by one student: "I had to use different terminology than what we learned in class.As engineers we have more technical background and understand things differently.The people I worked with came from very different backgrounds so I had to take a different approach.I couldn't use heat transfer terminology such as conduction and convection, instead I had to describe what the process was by using non-technical terms."Five of the students were aware of the fact that they had to alter their communication and be selective in terms of vocabulary in order to reach their audience.
There was a separate reflection question to encourage the students to reflect on the environment in which the service was rendered.The question read: "What was the environment of the neighborhood physically, economically, and ethnically?"Table 4 lists the words assigned as positive, neutral, and negative identifiers for the neighborhoods.Students' observations about the environments in which their visits took place are summarized in Figure 3.
Other Comments: There were many interesting comments recorded by the students.Here are a few examples: "I enjoyed doing this project.I thought it was very interesting to apply what we were learning in class to real world application.It was also interesting to see some "extreme" cases where our application could really benefit a homeowner.I think in the future it would be interesting to show the homeowner how much money we were helping them save.""This service project allowed me the same satisfaction as all of our community service opportunities.An experience like this is necessary periodically throughout one's life to be humbled, and become aware of what is truly important in life.""I also think it is beneficial to stay Detroit centered… This is our home, it's about time people start opening their eyes to what's right under their noses.""Overall it was a good experience that was rewarding.It was nice to be able to help out others less fortunate and also apply concepts from class.I think this is a great idea and it should definitely carry on for future classes.""I enjoyed the service learning.I would like to do it more often.It made me feel helpfulespecially after I saw how much there is to do and how few people there are to do it."One international student suggested that the instructor inform students about safety issues and risk involved when performing the service learning.
The major positive findings of the service-learning experience as obtained from students' responses are summarized in Figure 4.The majority of students to varying degree indicated that the experience benefitted the residents (78.6%) and was worthwhile (89.3%); they will likely continue community service or make part of their lives in the future (64.3%) and that the conversations they had with the residents and the way they had to communicate with them were valuable (78.8%).These results are very encouraging in terms of establishing students' awareness of the importance and benefits of community service.

International
Journal for Service Learning in Engineering Vol. 5, No. 2, pp.32-43, Fall 2010 ISSN 1555 FIGURE 3 TOTAL NUMBER OF POSITIVE, NEUTRAL AND NEGATIVE INDICATORS ABOUT THE ENVIRONMENT Indicators for each of these response types are listed in Table2.In order to quantify the results, positive indicators were assigned the number +1, neutral indicators were assigned 0, and negative indicators were assigned a value of -1.It should be noted that the numbers were obtained by simply summing indicators (for each student first and added up for all 28 students).A graph of the results is shown as Figure1.It is very clear that the positive indicators outnumber the neutral and negative indicators.TABLE 2 WORDS/PHRASES USED AS POSITIVE, NEUTRAL, AND NEGATIVE INDICATORS FOR ASSESSING STUDENTS'

TABLE 4 WORDS
USED AS POSITIVE, NEUTRAL, AND NEGATIVE INDICATORS FOR ASSESSING STUDENTS' REFLECTIONS ON THE