Learning Design through the Lens of Service: A Qualitative Study

Twenty-four senior-level capstone engineering design projects were completed at a large, public, primarily undergraduate university involving 85 students (70 male and 15 female). All projects involved the design of equipment to facilitate physical activity for people with disabilities. The effects on: i) learning design, ii) attitude towards people with disabilities, iii) motivation to complete team design projects and iv) interdisciplinary collaboration were analyzed through 24 one-hour focus groups. We explored the student experience using a constructivist approach and grounded theory. Four major themes (with associated sub-themes) emerged from our data analysis: learning design (project management, iterative design process, and user-centered design), motivation to complete design (engineering, disabilities, user), perceptions of people with disabilities (previous experience, changed attitudes and beliefs), and multidisciplinary collaboration (etiquette presentation, communication between disciplines, defining roles and expectations). Students completing these projects were shown to appreciate user-centered design, exhibit greater motivation when able to meet and develop a relationship with their client in person, discuss altruistic factors regarding their capstone experience, and were able to develop strong multidisciplinary skills. Index Terms – capstone design, design assessment, learning design, user-centered design


INTRODUCTION
People with disabilities constitute a minority group within society and as such are stigmatized in a similar way to other social minority groups. 1 Many people with disabilities can become marginalized through prejudice, stereotyping, and discrimination. 2For people with disabilities, physical activity provides a range of benefits including physiological, emotional, cognitive, and social aspects [3][4] .The genesis of the work presented in this paper began in 1999 with the creation of an Adapted Paddling Program in the Kinesiology Department at California Polytechnic State University, San Luis Obispo (Cal Poly).The Adapted Paddling Program was offered in collaboration with the Central Coast Assistive Technology Center (CCATC), an organization that provides evaluations, trainings and education to address challenges in employment, lifelong learning and independent living for individuals living in California's Central Coast.Cal Poly's adapted physical activity programs grew the following year with addition of a program called The Friday Club, in collaboration with Special Olympics, and EyeCycle, a tandem cycling program for people or are blind or low vision, offered then in collaboration with the now disbanded Central Coast Low Vision Council.
The cadre of activity programs offered through Kinesiology is now collectively known as Activity4All with the mission of providing and promoting physical activity for people with disabilities.The programs continue to provide practical learning opportunities for kinesiology students while serving the local community of people with disabilities.As the activity programs grew they led to the realization that specialized equipment, designed and built in collaboration with Cal Poly's College of Engineering, might facilitate greater levels of inclusion in physical activity for some of the programs' participants.Initial collaborations with faculty in Mechanical Engineering led to a series of engineering student teams designing and building prototype equipment.The richness of these first experiences led to an increased level of sophistication in both the projects and the multi-disciplinary make up of student teams.The collaboration grew to include students and faculty from Kinesiology, Mechanical Engineering, Computer Engineering and Biomedical Engineering.A series of successful projects attracted attention from far and wide which led to an expansion of community partners to include partners from as far afield as Bridge to Sports in North Carolina; Break the Barriers in Fresno, California; the US Adapted Ski Team and Disabled Sports Eastern Sierra.Since 2005, this collaboration has led to the custom design of over 100 adapted physical activity (APA) devices for our partners and the people they serve.
In our service-learning program, engineering teams work with kinesiology students as part of an interdisciplinary approach to meeting individual user needs.Engineering faculty advise teams through a design process ending with functioning prototypes.All kinesiology students in the program have previously taken an adapted physical activity class as part of their curriculum and have at least one quarter of experience running and participating in organized adapted sports activities with the local community.These activities include the Adapted Paddling Program (in which students guide sea kayaking daytrips in Morro Bay for people with spinal cord injuries), the Eye Cycle Program (in which students "captain" a tandem bicycle for people with low vision or who are blind) and the Friday Club (in which students help Special Olympics athletes learn a variety of sports skills).The kinesiology students provide expertise in access and abilities for the engineering students throughout the design process.This begins by giving a Disability Etiquette presentation to the engineering students that discusses abelism and appropriate social and professional expectations for interacting with people with disabilities.Later, they assist with testing prototypes with clients.
The different projects designed in our program are identified by our community partners with whom engineering and kinesiology faculty collaborate to outline the project scope and initial design brief given to students.Additionally, as devices age, faculty recruit students to repair and upgrade them as needed for individuals.These devices have the potential to greatly enhance the quality of life for those individuals with disabilities who utilize the equipment.The research presented here evaluates the impact of working on an APA project on engineering students learning of design, motivation, attitudes towards persons with disabilities and their multidisciplinary collaboration experience.

On Design and Service
A key element of undergraduate engineering education is learning to design. 57] Design is an iterative process that consists of devising a system, component, or process to satisfy a desired need.An effective capstone design experience should motivate students to develop a variety of design skills that are difficult to attain in typical lectures and laboratories.Capstone experiences can also provide opportunities for students to further develop program outcomes as specified by ABET Criterion 3a-k.These include the ability to: 1) function on a multidisciplinary team; 2) communicate effectively; 3) design and conduct experiments; 4) analyze and interpret data; and 5) design a system that is within realistic constraints.The number of team-based and multidisciplinary team-based capstone classes across the U.S. has increased 8 since 1995 9 , likely due to the influence of ABET on U.S. engineering programs. 8Faculty in that study report on the capstone as an ideal experience from which to evaluate students' abilities for Criterion 3a-k.Capstone courses require significant faculty involvement, and from the student's point of view, projects should be motivating and challenging but not overwhelming.This requires a faculty advisor to work closely with each student group and external client as project specifications are determined and design concepts are evaluated. 10ultidisciplinary capstone design provides students an opportunity to integrate domain knowledge with that from other disciplines in a holistic fashion, considered more relevant in real world contexts than "fractionated knowledge". 11From a more pragmatic viewpoint, teams in a multidisciplinary capstone experiences have shown superior performance in a variety of design skills and concepts such as innovation, proof of concept and communication skills than their singular discipline counterparts. 12Yet multidisciplinary design experiences come with challenges that their very form creates. Students may struggle with identifying and valuing the expertise of teammates from other disciplines; they may face uncertainty of each others' roles on the team; and they may face challenges in improving interpersonal skills. 13ervice learning pedagogy is seen in a variety of engineering and design classes, including multidisciplinary capstone design.Early assessment on the impacts of service learning in engineering focused on retention, diversity, and ABET outcomes including both technical and professional skills. 14More recent work shows positive effects on modeling and metacognitive strategies in a service-based design context. 15-16A survey of outcome assessments for projectbased service learning show they span knowledge, skills, attitudes and identity with specific outcomes such as cultural competency, ability to communicate effectively, function on multidisciplinary teams, and creative design. 17he work of Zoltowski et al. uses a qualitative approach to investigate the manner in which students experience human-centered design when "designing for others". 18Participants were situated in a variety of learning context including service learning.The authors uncovered seven descriptive categories in their analyses ranging from a technology-dominated view of design to an empathic one.Meaningful connections to clients are likely for students whose views lean toward empathy.

Motivation and Connectedness
0][21][22][23][24] Ford and Smith describe social purpose as an amplifier for motivation and argue that thriving in the context of larger social meaning has positive transformative implications for learning. 25oyle et al. underscore the learning opportunities that connections of the technical to broader societal contexts may provide. 26roviding students opportunities for social connection through team design experiences and community projects helps to meet a student's basic psychological need of relatedness.] Intrinsic motivation is a desirable learning state in which students actively integrate externally motivated (learning) goals and behaviors.Levels of internalization appear on a continuum of different motivational orientations. 29Whereas intrinsic motivation is a state described inherent satisfaction and goal internalization, amotivation lies at the opposite end of the continuum and is a result of one lacking to identify autonomy, competence, relatedness or value with the learning experience.Between the two extremes lies extrinsic motivation, driven by external factors and rewards.As individuals experience greater autonomy and identify more relevance in a task, they internalize the learning goals and eventually assimilate the learning into their own sense of values and identity. 27Internalized reasons for achievement-related behaviors positively correlate with measures of empathy, moral judgment, and interpersonal relatedness. 30These measures, specifically empathy we contend, align well with engineering design experiences in a service-learning context.

PURPOSE
The purpose of this study was to gain greater insight into the effects of completing a team APA design project on: learning engineering design, student motivation and student attitudes toward people with disabilities.An additional goal was to identify student perceived benefits and barriers related to working with an interdisciplinary team comprised of Engineering and Kinesiology students.Researchers were most interested in the salient characteristics of the Engineer-Kinesiology working relationship; one or two Kinesiology students were assigned to provide engineers with background on disabilities facilitate communication with the client and participate in the design process as needed.
This study examines an answer to the broad question, "What impact does completing a service-learning senior engineering capstone project in designing recreational equipment for people with disabilities have on learning design and motivation to complete such a project?"Secondary aims, that are largely programmatic, of the study have the objectives: to evaluate the overall impact and value of the program, to evaluate process and student collaboration between engineering and kinesiology departments, to improve the existing program by adopting student ideas that reinforce the success of the program experience for future students, and to understand if engineering students participating in this project value the experience and confirm the necessity of such a program.

METHODS
Focus groups are exploratory forms of qualitative research and are considered a useful and credible method to assess and evaluate engineering student learning outcomes. 31][35] A qualitative research framework analyzes data in a constructivist (e.g., naturalistic, hermeneutic, ethnographic, participatory action, symbolic interaction, and phenomenological) fashion from in-depth, semi-structured interviews (focus groups) and written documents (transcripts). 39It is with this framework that researchers engage in naturalistic inquiry in realworld settings to inductively generate and analyze rich narrative descriptions and construct themes across groups (in our case 24 groups over 5 years).] The primary purpose of the inductive approach is to allow research findings to emerge from the frequent, dominant, or significant themes inherent in raw data, without the restraints imposed by structured methodologies.In this type of evaluation research, researchers wish to describe the actual program effects, not just the planned effects.The identification of any significant unplanned or unanticipated effects or side effects arising from course implementation is regarded as an important evaluation task. 42ost-project focus groups were conducted with design teams to determine engineering students' perceptions of learning design.Previous experience with people with disabilities and the amount of client interaction were compared qualitatively to examine if those two factors played a role in the project outcome.Narrative evidence of increased technical skill, social responsibility, systems thinking, and communicating as a team effectively, were determined by the team's response to a semi-structured script.According to Lathen et al., it is important to assess if student engineers have "assimilated" the professional roles and responsibilities required to meet the ABET Criterion 3 guidelines. 43ocus group questions in this study were designed to: (1) explore if student perceptions (i.e., attitudes, beliefs, and knowledge) of people with disabilities changed as a result of their capstone experience, (2) identify specific design concepts or design processes learned, (3) identify motivating factors for students.(4) identify any academic and project design barriers, and (4) determine if students could identify a natural connection between 'engineering' and 'inclusion'.

Participants
Engineering students were purposefully selected to understand their academic and social experiences from participation in one of twenty-four adapted design projects (n = 85, 70 male and 15 female).All assigned projects had the intent of promoting inclusion for people with disabilities in a specific physical activity.Twenty-four focus groups were conducted from spring 2009 to spring 2013.There were twelve teams of three engineers, eleven teams of four, and one team of five.Projects enabled clients with disabilities to participate in a number of activities, including video games (e.g., Wii), kayaking, bocce, and bowling (See Table I).Criterion sampling was used (i.e., assigned individuals to a given project were selected to attend an approximately one-hour team focus group at the end of their capstone project).

Data Collection & Analysis
5] The interview script (see the Appendix), which consisted of a checklist of topics prepared by the moderator, was discussed with the members of the research team and revised according to their comments.
Data were collected at a large, U. S., four-year comprehensive polytechnic public university.Each focus group lasted 50-60 minutes in duration.After each focus group interview, debriefing reports from each session were discussed and recorded by the moderator and two note takers.These reports covered logistics, group dynamics, the moderators' performance, the participants' comprehension, emerging themes and unanticipated findings. 46The data analysis process for this study was guided by the principles of grounded theory.8] The value of qualitative research to public health professionals and others engaged in multidisciplinary health-related research is recognized and used in this study. 49rounded theory is particularly well suited to the analysis of this data in that the goal was to understand the issues that could serve as barriers or assets to the development of an experiential community-based design project.
Table I provides an overview of the composition of each focus group.The project title, year the focus group was run (coinciding with the end of the project; typically at the end of the spring quarter), the number of female and male students from the team present at the focus group, and the number of students who report selecting their project as their top choice.Students were asked about the amount of client contact they had during their project.Their responses were fit to a 5point scale ranging from low (minimal contact) to high (frequent, weekly or biweekly contact).The aspect of client contact was not explored in the findings of this study.

Coding and Agreement
Focus group transcripts averaged 10 single-spaced pages in length with a range of six to 15 pages.Because of potential confidentiality issues with respondent validation and time demands, this study utilized several methods of data analysis to ensure the rigor of the evaluation research. 50To ensure reliability, the analysis process involved three coders who read the transcripts and independently wrote a summary of the main issues that emerged for each of the domains.Coders then were instructed to first make notes in the margins defining the main points and identified the main themes for each of the questions.After coders developed these summaries independently, they met to discuss the emerging themes.Coders then reached consensus on the major themes and sub-themes (Table II).According to previous research, inter-coder agreement must be 0.90 or greater, although 0.70 is considered acceptable for most exploratory studies. 51When different interpretations arose, coders had extensive discussions until an agreed-upon final interpretation was established.Inter-rater reliability between lead author and two coders were 0.90, 0.93, and 0.92.Once the categories and themes were established, computer supported MAXQDA 10 TM (VERBI Software, 2010, Marburg, Germany) qualitative data analysis of the transcripts was conducted.Utilization of both human coding and software coding is useful for triangulation of codebook consistency, dependability, and stability. 52Software coding also helps to "red flag" or identify responses that were unanticipated (i.e., "other") so researchers can go back and add those responses for inclusion into the codebook if warranted.The researchers identified 17 major codes and 139 subcodes.The software matches these codes to phrases in the transcripts and reports on the number of hits for each.The researchers then reviewed all matches for verification.The selected quotes International Journal for Service Learning in Engineering Vol. 9, No. 1, pp. 1-23, Spring 2014 ISSN 1555-9033 presented in the Findings section are representative of the set of quotes for each theme/subtheme.

FINDINGS
Four themes and nine sub-themes emerged from the focus group analysis.Each theme and subtheme is described below, along with supporting quotes from engineering students (names have been changed, but not the gender and project of each student).These themes have evolved and expanded from our prior work. 53Student responses are referenced using the themes identified in Table II.The design process involves phases of analytical understanding, critical thinking, and creative decision making; and often times these elements of the design process do not occur linearly.Constant continuous team member interaction is required to complete their design and meet project submission deadlines.The value of most capstone projects is that the projects provide students with an iterative design process with a built-in set of purposive problem-solving procedures over several project phases.Arguably learning design wasn't so much about the end product as it was about the going through the iterative design process as evidenced by the following quotes.
Strider II (2011) Daniel: "It was a lot more involved than normal projects.It's three quarters long and you go through the whole process of getting the user requirements and designing it and analyzing it and making sure it's not going to break, and of course working with the client was just a lot more involved than any other [project] I have done." Strider II (2011) Daniel: ""Just the whole process from going to an idea to an actual product was a huge learning experience.It's a lot more work than I… I mean I expect to be a lot of work which is the whole process…you see a product and you think it's not that hard to make and design but it really is."

b. Attitudes Toward Client
User-centered design is a term used to describe design processes in which a client influences how a design project takes shape.The end-user is not an afterthought, but rather an important participant in the entire design, development, and fabrication process.8] Furthermore, individuals who have experience and contact with people with disabilities outside of their work/school setting (e.g., a close friend or family member) tend to have the most positive attitudes toward people with disabilities. 59The capstone experience is the first chance that most students have to think deeply about a specified user, and many of their comments reflect meaningful learning and altruistic patterns of thought by the following comments.
Sit Ski (2010) Bradley: "We are just trying to solve problems that people have… with people with disabilities it is looking for products to allow them to regain all of their independence.So if engineers can develop something for that, it would be good."typically present in capstone projects.After noticing several comments regarding the lack of guidance in terms of the budget in earlier focus groups, our team decided to provide more concrete initial budget constraints.The occurrence of budget-related comments reduced in latter focus groups.A second topic in learning design that was common among all projects was recognizing the importance of the entire design process, including background research, conceptualization, system integration, and the iterative nature of product design.The students working on the APA projects, however, appeared to have a much deeper understanding of the importance of the user in the design process and a greater appreciation for universal design (design for all users).Having a client with needs that may differ substantially from the general population encouraged students to recognize how important client interaction is during the design process.Students were required to consider mobility impairments, impaired vision, and developmental delays when designing their products.A few of the students discussed the need to consider all potential users from the beginning of the design process for all products.
Many of the engineering students working on APA projects were motivated to develop an idea into a final product and apply engineering skills to a real-world problem.These students also tended to comment on helping people, and specifically helping those with disabilities.Some students developed deep relationships with their end-users, and often discussed how they wanted to develop a good product for that specific individual.This points to a benefit in having a real local person as a client, with whom students can interact.
With respect to interaction with people with disabilities, most students who chose the APA projects had some prior experience with people with disabilities.More females (26.6%) had "extensive" experience working with people with disabilities than males (5.71%).Students tended to self-report their attitudes toward people with disabilities as having not really changed as a result of working on the APA projects.However, many of their quotes demonstrated a profound new appreciation for inclusivity and universal design.Students mentioned noticing wheelchair ramps and door openings, focused on the abilities of their clients rather than their disabilities, and commented on how their clients were simply unique human beings like the rest of us.
Our thematic analysis of student responses aligns with the work of Zoltowski et al. 18 Our theme on Learning Design aligns with the Category 2 (Service), Category 3 (user as Information Source Input to Linear Process) and Category 4 (Keeping the users' Needs in Mind).Representative quotes from our theme on Motivation to Complete Design is more aligned with Category 4, Category 5 (Understanding the Design in Context), and Category 6 (Commitment to Involving Stakeholders to Understand Perspectives).Evidence indicates a more powerful experience as relationships with clients deepens and supports the work of Ryan and Connell on a correlation between intrinsic motivation and empathy. 60e were very interested in the teaming between the kinesiology and engineering students.In the early stages of the project (2008-2009) students discussed their frustration as to the roles of the kinesiology students; similarly, the kinesiology students often felt like the engineering students were not involving them in the design process.Evidently, faculty advisors did not provide the students with enough scaffolding on roles and expectations.As a result of early focus group feedback, we made a number of changes in how we run the APA projects.The kinesiology students are now required to be available during the lab sessions of the capstone design teams; in the past the students decided when and how to communicate, which often resulted in sporadic email correspondence.Additionally, the majority of the APA teams are now placed in the same laboratory sections together; previously these teams were dispersed into

CONCLUSIONS
All of the engineering teams interviewed perceived a successful capstone experience in terms of learning design and valuing a human-centered project for their client.Motivation to successfully move through the design process was also highly evident for each team based on responses given during the semi-structured interview.As evidenced by our outcomes, focus groups were a valuable methodology to gather engineers' perspectives on their capstone projects that extend beyond the boundaries of the technical.Our findings strongly support the notion that engineering students perceive their capstone learning experiences as having had a direct and profound effect on their learning.Based on our findings, we believe that students completing the APA projects tend to appreciate user-centered design, exhibit greater motivation when able to regularly meet in-person with their client, discuss more altruistic factors regarding their capstone experience, and are able to develop strong multidisciplinary skills.Finally, clients played a pivotal role in students' design experiences.Students developed connections with their clients that helped them internalize their motivational goals with respect to their projects.While this internalization can happen with any type of client, we believe the personal and compassionate nature of service learning design projects provide an ideal environment in which this can and does occur.

TABLE I DESIGN
PROJECTS, YEAR OF FOCUS GROUP, NUMBERS OF FEMALE AND MALE STUDENTS ON TEAM, NUMBER OF STUDENTS ON TEAM FOR WHICH PROJECT WAS TOP CHOICE, AND AMOUNT OF CONTACT

TABLE II THEMES
, SUB-THEMES AND DESCRIPTORS DERIVED FROM THE 24 FOCUS GROUP TRANSCRIPTS International Journal for Service Learning in Engineering Vol. 9, No. 1, pp. 1-23, Spring 2014 ISSN 1555-9033planning, scheduling, and deadlines were discussed by the engineering teams as well as budget concerns.Daniel talks about planning, Ben values his client and not wasting money, and Greg discusses systems thinking and integration.
In design thinking, systematic questioning is integral to effective group project management as engineers generate, evaluate, and realize ideas to achieve client objectives while satisfying a specified set of constraints.However, realistic issues, problems and time constraints were frequently cited as capstone project concerns.Proper delegation of work assignments in terms of You can't just create a product and expect the customer to be like 'oh yeah that's awesome.'Youhave to go step by step to make sure they (client) are with you throughout the whole process so that there are no surprises at the end that cost a bunch of money down the drain."FoamWars II (2010) Greg: "The whole aspect of fitting… making different subsystems fit together into one system is… you work on one part by yourself, then you need to work with other subsystems do those systems interface.I learned a lot about that." Journal for Service Learning in Engineering Vol. 9, No. 1, pp. 1-23, Spring 2014 ISSN 1555-9033 13ny students chose the adapted physical activity projects because they felt they could help improve the lives of people with disabilities.Nearly all of the engineering teams reported a sense of altruistic beliefs that ultimately motivated them to complete their capstone project.Students acknowledge and appreciate the multidisciplinary collaboration between departments because they understand the need for such skills in the real world.However, focus group discussions clearly elucidated the need to establish clearer roles and expectations for Kinesiology students.Nearly all of the teams we spoke to agreed that this component was perhaps the weakest link in the capstone collaboration as evidenced by the following discussions, again in alignment with the findings in Wojahn et al.13Julian, David, and Aidro all report unclear knowledge about the abilities and roles of kinesiology students.Wally relates his role on the team to his expectations of "real-life engineering." student learning, motivation, perceptions of people with disabilities and interdisciplinary collaboration, while additionally trying to improve the capstone design experience.One of our major goals was to determine how participating in APA design projects affected design learning.Many student comments centered on scheduling, budgeting, and planning; topics that are International Journal for Service Learning in Engineering Vol. 9, No. 1, pp. 1-23, Spring 2014 ISSN 1555-9033