Effects of Service Projects on the Perceived Skills of Engineering Technology Students

Client-based service projects offer many advantages to instructors, students, and host communities. However, instructors must carefully choose the projects in order to achieve the benefits that come with this pedagogical tool. The purpose of this research was to investigate the perception of engineering technology students on how different service projects influenced their exam performance and perceived skills. A modified perceived skill assessment tool was used to measure the perceived effect of the team-based service projects on motivation to learn, contribution to research knowledge, skills and personal benefit, and project as a learning device. The projects were either client-based or non-client-based. In addition, they were categorized as either engineering, non-engineering, or some engineering projects. The non-engineering projects were those with no engineering content, whereas those with some form of engineering content were classified as some engineering. From the results, it appears that all the project types and categories were highly perceived by the students as contributing to their perceived skills. They were also highly perceived as great learning tools. Additionally, the projects with engineering content (client-based and nonclient-based) provided a significantly higher perceived motivation to learn. However, there was no significant difference in exam performance. Index Terms – Pedagogy, perceived skill, project-based learning, service learning.


INTRODUCTION
Engineering technology programs, compared to other engineering programs provide more handson experiences for students. These experiences can be in the form of lab work, project-based learning (PBL), service learning (SL), or other forms of experiential learning. Unlike PBL and SL, lab works are usually scripted and may not necessarily vary. However, PBL and SL are unique; for example, they can be client-based, or non-client-based. Client-based projects involve students working with a client different from the instructor for the entire semester or a fraction of it, while non-client-based projects usually have the instructor serve as a pseudo client. In addition, the scope of client-based projects can have a varying degree of relevance to the student's program of study. As a result, instructors must carefully choose projects that have the appropriate scope, relevance, and level of difficulty. Specifically, SL projects have challenges for engineering technology students when they do not provide the right engineering content for the students to practice their engineering skills. Hence, if not appropriately structured and executed, the selected projects may not be experiential for the engineering technology students who prefer hands-on activities. This paper reviews the perceived skills of engineering technology students who completed projects with varying degrees of engineering contents. Perceived skill is a self-assessment of one's competency in attaining a skill. Experiential learning has been widely accepted in higher education due to the numerous advantages it offers the learner 1 . Experiential learning enables students to acquire knowledge by doing. Unlike traditional education, experiential learning utilizes experience to impart knowledge. Research shows that traditional students 2 and non-traditional students 3 alike learn best not by listening but through an environment that provides the opportunity for experience 2,3 . This supports Dewey's "learning by doing" theory 4 . Dewey posited that education should not be a preparation for living, rather, a process of living 4 . Kolb 5 observed that experiential learning has multiple phases, and all must work together for the benefit of the learner. Beginning with concrete experiences, the students should be able to observe and then reflect on the observations. That should also lead to the conceptualization of the abstract, which will be then be experimented with and tested 6 .
There are various forms of experiential learning. However, not all pedagogies involving a form of experience is considered experiential. Chapman et al. 7 explain the factors that an activity must have to be considered experiential. The experiential activity can be in the form of an internship, class project, clinical experience, study abroad, service learning, or simulations. However, these activities may not necessarily be experiential if they are not carefully planned 7 . If well executed, experiential learning offers benefits that may not be achieved with the traditional pedagogy 8 .
Project-based learning (PBL) is an example of experiential learning if it demonstrates the traits explained by Chapman et al. 7 . Lamer and Mergendoller 9 explain that a project-based learning approach should demonstrate seven traits to appropriately be considered experiential. The instructor should be able to motivate the students with a project hook statement that captures their interests. Client-based and non-client-based projects can provide the benefits that students get from PBL 10 .
Client-based projects offer students the opportunity to be exposed to the realities outside of the classroom. However, instructors must carefully choose clients in order to be successful. An uncommitted client may erode students' interest and motivation in the project. Amy and Elzbieta explain that the instructor should ensure that the client understands the time commitment and be ready to work with students before assigning the projects. Nonprofit organizations usually serve as better clients for client-based projects 11,12 . These organizations work with a limited budget, hence, having students work on their needs as projects is a win-win.
Nonprofit organizations can be a source of client-based service projects for students. For example, some of the students who were surveyed for this research worked with a nonprofit organization to prepare training documents for the families of opioid addicts. This organization prepares the families to assist the addicts through the recovery journey. Hence, they provided a service learning or community engagement opportunity for the students to give back to their community. Service learning also provides the same benefits as the other experiential learning pedagogies if there are structured moments for reflection 10 . This can generate a sense of responsibility. However, some service learning projects may not necessarily be related to the major of the students, and that could be discouraging. Consequently, this research was conducted to study the perception of engineering technology students on project-based learning using service projects.
A section of the projects had engineering contents such as requiring the use of computer-aided design (CAD) software to design a product and the design serving as the main focus of the project. These projects were classified as engineering projects. The second set of projects had some engineering content such as a feasibility study of an existing design or product without any substantial redesigning. These projects were classified as some engineering project. The last set of projects were not directly related to traditional engineering and were also classified as nonengineering projects. The projects were completed in a project management class in a 4-year college in the Midwest. All of the projects were executed between 9 to 10 weeks. The students completed a survey (shown in the appendix) after the projects and the results are discussed in the latter sections of this paper. The objective was to measure the contributions of each of the project categories and type (client vs. non-client-based) to perceived skill. The next section reviews the literature on service, project-based and experiential learning.

LITERATURE REVIEW
Some researchers have identified inefficiencies in the traditional system of education, in which all that students do is to listen to the instructor and take notes 4, 13-15 . Trigwell et al. 16 explain that this pedagogy breeds students who may lack the skills of critical thinking. Education should allow students to apply their knowledge to real life activities. PBL offers this opportunity. PBL, whether serviced based or not, provides real-world problems or projects for students to critically think through them, apply acquired or new knowledge to generate a solution. Students learn by solving a problem or working on a project. The students do not have to complete the project independently as the instructor can facilitate and guide them through the process providing appropriate scaffolding. Luckmann defines experiential learning as "a process through which a learner constructs knowledge, skill, and values from direct experience" 17 , and this is the best way for a student to learn 5 . Even though knowledge comes from the experience, the level of knowledge depends on the "quality of the experience " 13 .
The full benefits of experiential learning may not be realized if the experience is devoid of moments for reflection. Reflective learning is defined as "the process of internally examining and exploring an issue of concern, triggered by an experience, which creates and clarifies meaning in terms of self, and which results in a changed conceptual perspective" 18 . Reflecting on the experience help students to learn 19,20 , bring abstract ideas to life 14 , leading to an appreciation of the concepts. Richardson observed that reflections in PBL lead to discernment 21 .
PBL, which is a form of experiential learning, can be client-based on non-client-based. With a question or problem, students work to understand and collaboratively work with peers (if team-Humanitarian Engineering and Social Entrepreneurship Vol. 14, No. based), instructor and client (if client-based) to generate a solution 22 . From the student's perspective, client-based projects, if executed in teams, provide real-life experience by providing the environment to improve interpersonal skills 23,24 . de los Santos and Jensen observed that students have a higher interest in learning in courses that have client-based projects because they are more real with the projects 25 . Client-based projects also help students to acquire and improve communication skills, problem-solving and teamwork skills. 26 People with some work experience after college agree that client-based projects helped them to acquire employable skills 27 .
McEachern observed that client-based projects have benefits that may not be replicable from case studies 28 . They also provide faculties with networking opportunities and sometimes the potential for research 25 , as well as the opportunity to learn about the industry and current trends in their field 27 .
Despite the benefits of client-based projects, they also have challenges which make them less attractive for some faculties to adopt. First of all, the instructor must ensure that: the project is appropriate for the class, the scope is reasonable for the period and the client is reliable and committed 8 . This is not a trivial issue and finding the optimum solution is always difficult 23,29 . They demand more instructor time than non-client-based projects or case studies. If an appropriate client-based project is not found, instructors may use non-client-based projects.
Service learning (SL) or community engagement learning is also a form of PBL which usually is client-based. Celio et al. 30 define SL as "teaching and learning strategy that attempts to integrate community service with an academic curriculum…" By providing an experiential learning opportunity for students, the community or organization providing the SL projects also benefits. The students benefit "personally, socially, and academically" 30 . In addition, SL that incorporates moments for reflection helps students to develop a positive attitude and enhances academic performance [30][31][32] Some researchers have observed that non-client-based projects can produce the same academic benefits as client-based projects. Amy and Elzbieta concluded that students generally perceive both client-based and non-client-based projects as motivating 8 . This means that non-client-based projects of similar scope as client-based projects can be carefully administered to achieve most of the benefits that come from the client-based projects. Even though some researchers indicate that client-based projects are more beneficial to students learning 23, 28 , it is not clear whether the observed benefits are as a result of the project type or any other factor such as teamwork, or faculty commitment and supervision. But there is a consensus that group projects (whether client or nonclient-based) are preferred to other project types 8,[33][34][35][36] .
Even though a lot of research has been done about students' perception about project types on their learning, there has not been a holistic view of the variances (such as the level of engineering design required) between the projects especially if there is a service component. For example, Amy and Elzbieta 8 looked at the perceived effects of client and non-client-based projects on students. The perception of students on long-term client-based projects and shorter-term case studies were studied by Abernethy and Lett 37 . These papers do not delve into the project variations. An attempt is made in this paper to break the projects down into 3 components, in terms of how closely related they are to the major of the engineering technology students. The next section explains the methodology. The non-client-based projects have the instructor serving as the client. The students are usually introduced to the concepts of projects management during the first four weeks before they are assigned to their team projects which they execute for the rest of the semester. At the end of the semester, they deliver a written report and oral presentation. The non-engineering projects have little or no traditional engineering component. For example, in one of the projects, the students were tasked to help a local nonprofit organization involved in helping opioid addicts to come up with a training manual for their volunteers. There existed a voluminous document which was very ineffective for the volunteers who do not have the required time to review them before training or working. So, a team worked with the representative of the organization to prepare a training manual from the documents. The projects with engineering components were divided into two clusters: the first cluster had the projects that had traditional engineering activities (such as designing with SolidWorks, AutoCAD, or any other engineering design software). If a greater proportion of the project required engineering design, the project was classified as engineering-based. However, if the engineering component was marginal but significant, the project was classified as some engineering. Design of a vertical aquaponic system and the optimization of a grocery shop layout were some of the engineering-based projects. The vertical aquaponic system was classified as some engineering because the design was the only section of the project which required engineering skill. The other sections of this project could be completed by a non-engineering major student, hence, the clarification. Compared to the aquaponic system, the optimization of the grocery store layout involved design and simulation. Therefore, it was classified as engineering-based because it required substantive (comparatively) engineering skills beyond the design stage. The students had to test their design through simulation and interview shoppers for feedback on the proposed layout. The scope of all projects was developed collaboratively with clients. The projects with no engineering component were all classified as non-engineering. These are referred to as the project category in the discussions. In addition, they were either client or non-client-based, which are represented as the project type in the discussions. The students who worked on the projects were sophomores, juniors, and seniors (academic level).
Even though the projects were different, they were all administered in a similar fashion since 2015. At the end of each semester, the students complete an online survey on the contribution of the project to their research knowledge, skills and personal benefits, as well as the project as a learning tool, and motivation to learn. The survey results were not accessed until the final semester grades were posted. This research was to determine whether there is a preference for any of the project types and categories. The following were the main questions that it attempts to answer: • Is there any interaction effect between the project types, categories, and academic level?
• Do the students (sophomores, juniors, and seniors) perceive the project types and categories differently? • Is any of the projects perceived to contribute more to research knowledge?
• Is any of the projects perceived as a better motivating tool?
• Is any of the projects perceived to contribute more to skills and personal benefit?
• Is any of the projects perceived as a better learning tool?
The Goodell and Kraft's perceived skill scale was modified to measure student responses to the set of questions on a 5 point Likert scale. From a sample of 205 student responses, the results are discussed below.

RESULTS AND DISCUSSIONS
Due to the multiple variables, a 2 (client vs. non-client) x 3 (sophomore, junior, senior) x 3 (engineering, non-engineering, some engineering) Multivariate Analysis of Variance (MANOVA) was used to analyze the survey data. The scale reliabilities of the contribution of the project to research knowledge, motivation to learn, the project as a learning tool and contribution to skills and personal benefits were above 0.7. Of the 205 students, 153 completed client-based projects while 52 completed non-client-based projects. Additionally, there were 43 sophomores, 80 juniors, and 82 seniors. The number of students who completed engineering related project, nonengineering, and some engineering projects was 77, 81, and 47 respectively. Basic MANOVA assumptions were checked. There was no outlier, and the data met univariate normality. Box's test for equality of covariance was insignificant (p>0.05). Test for Sphericity, in this case, was not applicable since it is not required by MANOVA 38 .
The MANOVA results shown in table I indicated that there were statistically significant differences in the levels of Project type (F = 22.822, p = 0.000, Wilk's lambda =0.625), Academic level (F = 2.724, p = 0.003, Wilk's lambda = 0.870) and Category (F = 4.159, p = 0.000, Wilk's lambda = 0.812) in the dependent variables considered as a group. The omnibus test (presented in the Multivariate section of table I) indicates each independent variable's effects on the dependent variables, considered together in one basket. In addition, there was an interaction effect between project type and academic level (F = 3.425, p = 0.005, Wilk's lambda = 0.917), indicating the effect of project type differed based on which academic level was being considered. However, there was no interaction effect between the other variables.
As seen in Table I, there was a significant difference in students' perceptions of how the project type contributes to motivation to learn, the project as a learning device, and skills and personal benefits. The Client-based projects (mean = 4.52) were perceived as impacting motivation to learn significantly more than the non-client-based projects (mean = 2.46). But the project types were equally rated for the remaining dependent variables. The means as shown in the parenthesis (client-  Table II. A Bonferroni post hoc test revealed that the sophomores perceived the projects as a contributor to skills and personal benefits at a statistically significantly higher rate than seniors (p = 0.035). However, there was no statistically significant difference between the sophomores vs. juniors, and juniors vs. seniors. In addition, there was no significant difference among the academic levels for motivation to learn, contribution to research knowledge and the project as a learning tool. The exams scores were also not significantly different among the student groups.
The Bonferroni test for the project categories (engineering, some engineering, nonengineering) revealed a statistically significant difference between the engineering projects and non-engineering projects (p = 0.000). The students who worked on engineering projects significantly perceived them as motivation to learn than those with non-engineering projects. However, there was no significant difference between engineering and some engineering projects' perceived motivation to learn. But the projects which had some engineering contents were significantly rated higher (p = 0.000) for motivation to learn than non-engineering projects. This did not translate to better exam performance as there was no significant difference in the exam scores for the project categories. Likewise, the Bonferroni test revealed no significant difference between the project categories in their contributions to skills and personal benefits, research knowledge, and project as a learning device. The means are provided in Table II.

CONCLUSIONS
This paper studied the perceived skill of undergraduate engineering technology students in a 4year college. With student teams working mostly on client-based and non-client-based service projects which had varying degrees of engineering contents, the perceived contribution of the projects to skills and personal benefits, research knowledge, motivation to learn, and project as a learning tool were measured with Goodell and Kraft's perceived skill scale.
The results indicate that the students generally perceive highly project-based learning as useful pedagogy. Client-based projects, as well as non-client-based projects provide motivation to undergraduate engineering technology students. However, the client-based projects appeared to provide a higher perceived motivation than non-client-based projects. Projects that offered the students some opportunity to practice their technical engineering skills were highly perceived as more motivational. This is not surprising since projects help students to connect theory and practice; the strength of engineering technology. However, all the project categories were highly perceived (3.1 and above) as a motivational tool by the student categories. It is noteworthy that even though client-based projects were perceived as a higher motivation to learn, it did not result in any significant performance in the exams. This suggests that the type of project does not necessarily influence student performance. Finally, all projects (client-based or non-client-based) and project categories (engineering, some engineering, non-engineering) were highly perceived (4.1 and above) as a contribution to research knowledge, skills, and personal benefit. Likewise, they were equally perceived highly (4.3 and above) as good learning tools whether client or non-client-based with engineering or no engineering content. Therefore, it suffices to say that service learning with non-engineering projects can produce the same outcome as engineering projects for engineering technology undergraduate students. Even though some of the students may feel that their skills are not been utilized as a result of the non-engineering nature of their projects, instructors may still be able to use them to enhance learning if the students understand what is expected of them.

Motivation to Learn
I prefer an all lecture course to the project. I preferred the project to analyzing case studies. The project increased my interest in the course. The case studies increased my interest in the course. The project made discussions in class more enjoyable. The case studies made discussions in class more enjoyable. The project was time consuming, but worth the time spent on it. I prefer to have the instructor as the client instead of an outside (real) customer. The client is committed to utilizing a portion, or all of my project report information.

Project as Learning Device
The project made the subject matter realistic. I integrated the material in the course into the project. The project illustrated concepts in the course. The project will help me remember the material better.

Contribution to Research Knowledge
The project illustrated practical problems with doing research. The project helped me understand client/customer needs. The project will help me to evaluate product/feasibility research done by outside professional engineers.

Skills and Personal Benefits
The project helped me develop my teamwork skills. The project helped my report preparation skills.