Cross-cutting skills: strategies for teaching & learning

ABSTRACT

This conceptual article focuses on teaching and learning strategies aimed at helping students in higher education develop cross-crossing skills such as communication, critical thinking, teamwork, problem-solving, and working with diverse others. Within the framework of high impact practices and their elements, the article shares practical ideas for developing and responding to assignments that emphasize cross-crossing skills and measuring associated learning outcomes. The article focuses on developing assignments with an emphasis on designing and structuring team ePortfolios, collaborative projects, peer review tasks, and reflection; responding to assignments with techniques such as rubrics, course communications, and self-regulation strategies; and measuring outcomes to determine the effectiveness of assigned tasks with the goal of informing curricular and pedagogical modifications.

KEYWORDS:

• Teaching and learning
• high impact practices
• cross-cutting skills
• self-regulated learning

Extensive research indicates that employers favor candidates who demonstrate proficiency in skills that cut across fields of study, specifically written and oral communication, teamwork, ethical decision-making, critical thinking, and the ability to apply knowledge in real-life situations (Association of American Colleges & Universities [AAC&U], 2002; Hart Research Associates, 2006a, 2006b, 2008, 2010, 2015, 2018). In fact, ‘most employers say that these cross-cutting skills are more important to an individual’s success at their company than his or her undergraduate major’ (Hart Research Associates, 2015, p. 1). Employers value discipline-specific knowledge but consider it insufficient for long term career success unless accompanied by a broad range of knowledge and skills. Recent college graduates rank their level of preparation much higher in these areas than do employers (Hart Research Associates, 2015). This suggests that higher education institutions need to be more effective in preparing learners with these proficiencies. Employers agree. Nearly twice as many (81%) feel that higher education institutions can do more to help graduates attain cross-cutting learning outcomes compared to attaining field-specific outcomes (48%) (Hart Research Associates, 2015).

These skills have traditionally been referred to as soft skills, and more recently as cross-cutting skills (American College Testing, 2019; Hart Research Associates, 2015), essential learning outcomes (AAC&U, 2008), boundary-crossing skills, or 21^st century skills (Glossary of Education Reform, 2016). Preferred terms may differ by context such as business and industry, primary/secondary education, or higher education. Generally, however, they describe skills that are not discipline or career specific, that can and should be developed within educational settings, and that prepare individuals for lifelong learning in a rapidly changing global world. The term boundary-crossing skills, in particular, refers to transformative pedagogies that connect formal and informal learning such as education and the workplace, or address the dichotomy between individual and collective learning (Engeström, 2004; Virkkunen, 2005). Similarly, essential learning outcomes are achieved through high impact practices, which include teamwork, collaborative projects, service- or community-based learning, and internships; thus, these, too, can be considered transformative pedagogies (Association of American Colleges & Universities, 2008; Kuh, 2008; Kuh, O’Donnell, & Schneider, 2017; Schneider, 2015; Watson, Kuh, Rhodes, Light, & Chen, 2016).

The term cross-cutting skills is used in this article due to its prevalence in higher education and its association with essential learning outcomes. Within the framework of high impact practices (HIPs) and their elements, the article shares practical ideas for developing and responding to assignments that emphasize cross-cutting skills and measuring associated learning outcomes. The ideas shared can be applied to a range of courses and contexts, and across delivery modes (e.g., face-to-face, blended, and online). The article demonstrates the application of the ideas in an introduction to organizational behavior course, required for all business majors the author’s university.

Guiding framework

While ideas for assignments can be inspired from a number of sources, the key elements of high impact practices (HIPs) are a helpful guiding framework (Kuh & O’Donnell, 2013). Although an official list of eleven HIPs exists, namely writing- and inquiry-intensive courses, collaborative assignments, ePortfolio, capstone courses, first-year seminars and experiences, common intellectual experiences, learning communities, undergraduate research, diversity/study away/global learning, service and community-based learning, and internships and field experiences (Kuh, 2008; Kuh et al., 2017), some of these (e.g., first-year seminars or learning communities) may be more specific to the U.S. higher education context than elsewhere. Academic staff, and even institutions, can and should develop their own HIPs to fit their contexts. As such, the foundational elements are particularly helpful (Kuh & O’Donnell, 2013). See Table 1.

Table 1. Key elements of high impact practices

Key Elements of High Impact Practices (Kuh & O’Donnell, 2013, p. 10)
Interactions with faculty and peers about substantive matters
Frequent, timely, and constructive feedback
Significant investment of time and effort by students over an extended period of time
Performance expectations set at appropriately high levels
Experiences with diversity, wherein students are exposed to and must content with people and circumstances that differ from those with which students are familiar
Periodic, structured opportunities to reflect and integrate learning
Opportunities to discover relevance or learning through real-world applications
Public demonstration of competence

The use of HIPs and their foundational elements as a guiding framework is well supported by data from the National Survey of Student Engagement (NSSE), a student self-report instrument. Results demonstrate relationships between enriching educational experiences (e.g., HIPs) and academic challenge, active and collaborative learning, student-faculty interaction, and supportive campus environment, as well as to the learning outcomes of critical thinking, writing competence, and quantitative reasoning (Kuh et al., 2017). Similar surveys (e.g., the Australasian Survey of Student Engagement) are used in other higher educational contexts, to provide ‘information on students’ involvement with the activities and conditions that empirical research has linked with high-quality learning and development’ (Coates, 2010, p. 2). Overall, both the practices and their elements demonstrate a valuable means of helping students develop cross-cutting skills in preparation for future careers.

Course design

Examples from an introduction to organizational behavior course are used to demonstrate how HIPs and their elements can inform course design. Course concepts focus on communication, conflict and negotiation, diversity, emotions and moods, groups and teams, leadership, organizational culture, motivation, perception, decision-making, attitudes and job satisfaction, and power and conflict. The course is essentially about understanding people and their behaviors in organizations in order to more effectively manage and motivate employees, and as a result, achieve organizational goals. Section size ranges from about 35–45 students although sections could be considerably larger as long as adequate support, such as the use of teaching assistants, is provided for marking assignments, and appropriate structure and technology is in place to guide teamwork and other course components across various delivery modalities.

Although this is a practice-based paper, not a research study, it includes responses from students pertaining to their learning in the course, required for all business majors at a large, regional, open admission university in the United States. All institutional research requirements were met, and permission granted for anonymous use of these quotations. The quotations are taken from various sections of the course as taught over several semesters. Information about how the comments were collected (e.g., assignment details) is included in the relevant sections.

The various elements of course design are next examined: 1) developing assignments focuses on designing and structuring team ePortfolios, collaborative projects, peer review tasks, and team and individual reflections; 2) responding to assignments emphasizes rubrics for instructor and peer review, course communications in learning management systems, and self-regulation strategies, and 3) measuring outcomes explores methods for determining the effectiveness of assigned tasks such as UNESCO’s pillars of learning and the POLC framework, which identifies managerial functions, and how these can be used to determine assessment loop closings and inform curricular and pedagogical modifications.

Developing assignments

The course incorporates a number of HIPs, namely collaborative assignments, experiences with diversity, community-based learning, intensive writing, and ePortfolios. Teamwork is a central part of the course in order to help students develop management skills and apply the organizational behavior concepts. Examples of key assignments are next provided to illustrate the integration of course content with HIPs and their elements with the aim of helping students develop cross-cutting skills.

Designing and structuring team ePortfolios

Students may resist teamwork due to negative past experiences in which they were given group assignments without any training or structure (Scott, Derrick, & Hoadley, 2012; Zarraga-Rodriguez, Jaca, & Viles, 2015). The result is often uneven participation, lack of communication, and conflict avoidance. Students must initially understand that groups and teams differ. In groups, members share information, are individually accountable, and have varied skills while in teams, members have a common purpose, focus on collective performance, are mutually accountable, and contribute complementary skills (Katzenbach & Smith 1993, 2015; Robbins & Judge, 2017).

To structure an effective team experience, students begin by creating a team charter; the charter outlines roles, communication methods, norms, behaviors, and problem-solving strategies. Team effectiveness is enhanced through clear processes and structures (Hoff, Jameson, Hannan, & Flink, 2004; McDonald, Waring, & Harrison, 2006; Simpson, 2007). Prior to creating the charter, students review materials on effective team performance (Katzenbach & Smith. 1993, 2015 and the stages of group development (Tuckman, 1965; Tuckman & Jensen, 1977). The charter serves as a guiding document throughout the course. When teams have problems, the instructor can refer them to their charters to identify what they agreed on and determine if they need to make adjustments. In essence, the instructor helps the students develop self-regulation, or take responsibility for the factors and conditions affecting their learning (Zimmerman, 1990).

The ePortfolio serves as a collection of student work throughout the semester. Students are provided with tutorials for setting up the ePortfolio as well as resources on effective design. The team charter serves as the home page. Topics for artifacts can vary, but may include analyses of course concepts, application of concepts, personal experiences or insights, team reflections on learning, management challenges where students are presented with a workplace scenario and resolve it with theory-based approaches, consulting project proposals and reports, and reflections on team work and goals for improvements. Inevitably, team members possess a range of skill sets that result in effective page design, written communication, critical analysis, and application of concepts. It typically takes an artifact or two with instructor and peer feedback for students to catch the vision but they produce incredibly impressive work. Overall, the ePortfolio assignment represents the HIP element of engaging students in a significant investment of time and effort over a period of time.

Collaborative projects

To get hands-on experience applying course concepts, students participate in a team service learning project. They identify a business in the community – one where a team member works or has formerly worked, a family-owned business, or a business where a team member has a connection. They coordinate with the owner or a manager to identify an organizational behavior problem, collect and analyze data, and make recommendations using course concepts as a lens. They learn to think critically, problem-solve, and identify evidence-based solutions as well as communicate effectively to coordinate with their contact person at the organization, with each other, and in their final report (in the form of an ePortfolio artifact).

The project has a number of milestones. This structure helps students stay on task, make steady progress, and meet deadlines. The first milestone is submitting a proposal, which identifies the organization and problem, steps needed to complete the project, responsibilities for the different aspects of the project, and a timeline with due dates. Students create multiple drafts of their artifact reports. These are reviewed by trained student university writing tutors as well as class peers with revisions made to drafts between reviews. As a result, when the final projects are submitted for a grade, they are generally polished and professional. The milestones help students plan, organize, and monitor their progress; they continue to develop desired learning outcomes related to teamwork and recognizing the diverse skill sets and perspectives that individuals contribute. They engage in the revision process to improve their writing and learn how to understand and apply feedback. Final project reports are included as an artifact in the team ePortfolio.

Peer review

For each ePortfolio artifact submitted, students provide a rubric-based review of another team’s work either individually or as a team (instructors can decide which of these options to implement). Students often comment on how much they appreciate this feedback. They set goals for what they want to improve based on peer and instructor comments. Students are insightful and capable of providing constructive criticism when guided by a rubric and provided with examples of effective reviews and instructor feedback on their reviews. The rubric asks students to comment on the ideas, writing, and design, and make suggestions for improvement (see Course Communications for more details). The excerpts below illustrate.

One thing we appreciated on the content was the statistics although sources were needed for these.

It was a great idea to add a summary and recommendations for each topic. This aided comprehension and made the ideas very clear.

The artifact started with a helpful summary, detailed the process, and then ended with a clear summary, making the layout easy to follow.

The artifact provided helpful explanations of two common decision-making errors. We reviewed this same information in our artifact but gained additional understanding from this artifact that would have provided us with a better analysis. The content was a little light overall, however.

In face-to-face courses, students can work in teams to do their reviews and then share their observations with the class (as well as post them in a discussion in the learning management system). It is not uncommon for comments to coincide with the instructor’s assessment of the work, but students also have their own perspectives. They serve as a real audience for the artifacts, which supports the HIP element of public demonstration of competence, and helps each team strengthen its communication skills, specifically writing, organization, and design. In one case, I had a team question some of my comments on their artifact. When the peer reviews were presented, the same elements that I had noted for improvement were also indicated by the review team. This helped the team see their work from an outside perspective rather than be defensive about it.

Team and individual reflection

Team and individual reflections are also critical components in the course. These can be organized in various ways. Having a formal team review part way through the course offers an opportunity to determine what is working, what needs improvements, and how to bring about needed improvements. This can also encourage teams to make changes in their team charters. When asked to reflect on these three questions, teams typically identify procrastination as an issue (waiting until just before the due date to complete an assignment), communication and response times to e-mails and messages, not coordinating across all parts of an artifact (which results in content overlap and other issues), the need for better editing and polishing, and so forth. Strengths include items such as feeling comfortable sharing ideas and perspectives, being respected and listened to, enjoying each other’s company, and forming friendships. The following excerpts illustrate.

We can improve in the following ways – preparing for assignments earlier in the week. Giving out individual assignments for each task earlier in the week. Making sure that everyone has done the training that is required for the given task. Being more open to new ideas from all team members on how to do the tasks.

We have learned the importance of trust. Each week we assign tasks to each individual member to complete before the assignment is due. These tasks are interdependent of one another, in that, if one is not done, the rest will fail. For example, one person creates the ePortfolio, another writes the summary, a third reviews and edits the summary, the fourth reviews the summary and provides relevant pictures for the ePortfolio; if one of these is missing, it has a detrimental effect on the rest. We anticipate that we will be learning more about behavioral theories and how they can apply to our projects and our life.

Our team has done really well in assigning tasks and setting expectations for each team member … We can coordinate earlier in the week by each team member being more cognizant about initiating a conversation earlier in the week … We also hold each other accountable for our responsibilities … While we have gotten to a point that we are comfortable now, it took us a while to understand what the best way to communicate is.

The challenge of having to collaborate in a diverse group, against a time limit, yet be rewarded with an individual grade based on the group’s results added to the satisfaction of the final product … We certainly lived all stages of Group Development as outlined in the Group trainings. Forming, Storming, Norming, and Performing resulted in a cohesive group … It was actually reaffirming to eventually read the Group trainings and realize that we were a great live action example of this model.

This semester, I thought it was interesting because every member was unique, which relates to the concept of personality and diversity. Each individual had skills and expertise to bring to the table, and some had more to offer than others. I quickly was able to determine who were leaders and who were followers and observe and understand behaviors.

Without the reflection component in the course, students would likely not take the initiative to talk though needed areas of improvement or to make changes. The reflections also encourage students to consider how they have applied components of the course and the value of what they are learning.

Responding to assignments

A helpful first day activity that serves as a foundation to the course, particularly the rationale for HIPs and their elements and the types of responses students will receive on their assignments, is to conduct a poll asking students what they hope to learn in the course. For example:

“Thinking of what you want to get out of your college education and this course, which of the following is most important to you?

1. Acquiring information (facts, principles, concepts)

2. Learning how to use information and knowledge in new situations

3. Developing lifelong learning skills” (Smith, 2008, p. 2).

Inevitably the majority of students choose the last two items. The instructor then has the opportunity to explain that all three areas are part of the course, but the ultimate goal is application of learning and lifelong learning. Instructors can refer back to this poll when reviewing the purpose of assignments and the types of feedback provided during the semester through rubrics and other communications.

Rubrics

Numerous benefits of rubrics have been identified such as increasing student awareness of strengths and weaknesses related to specific assignments and achievement of course objectives (Bolton, 2006; Gibson, 2011; Mora & Ochoa, 2010; Rau, 2009; Smith, 2008), improving instructor objectivity and illuminating both individual and class-wide areas of mastery or gaps in learning (Garfolo, Kelpsh, Phelps, & Kelpsh, 2016; Mora & Ochoa, 2010; Petropoulou, Vassilikopoulou, & Retails, 2011; Smith, 2008), and focusing conversations between instructors and students on learning as opposed to grades (Garfolo et al., 2016; Walvoord & Anderson, 1998).

Rubrics support the high impact practice element of frequent, timely, and constructive feedback. In the introduction to organization behavior course, I identify expectations for ePortfolio artifacts, peer review tasks, consulting projects, and other assignments and indicate these in the associated rubrics. The rubrics reflect specific course-level learning outcomes as well as broader cross-cutting skills. They also demonstrate the HIP element of setting performance expectations at appropriately high levels. An example of an ePortfolio artifact rubric follows. Table 2 indicates how the rubric descriptors correspond to HIP elements or cross-cutting skills. The artifacts referred to are focused on resolving a management challenge through evidence-based recommendations; these artifact assignments help prepare students for the consulting projects where they will use similar processes and skills.

Table 2. ePortfolio Artifact Rubric

Rubric Descriptor	HIP Element or Cross-Cutting Skill
All required content is present (issue, theory, recommendations, reflection).
Strong critical analysis and effective recommendations based on theory/concepts.	Interactions with faculty and peers about substantive matters; critical thinking, problem-solving, real-world application
Thoughtful reflection.	Periodic, structured opportunities to reflect and integrate learning
Effective writing, organization, and design.	Written communication, public demonstration of competence in the shared ePortfolios

I provide written comments within the rubrics to explain rationale for scores and elucidate rubric descriptors. For online courses, I sometimes use a screencast recording tool to provide feedback on students’ work and demonstrate how I am applying the rubric. When student teams complete their team performance evaluations, they often consider rubric-based assignment feedback as the basis for improvement goals. Rubrics can vary to meet course specifications as well as instructor preferences. I find that keeping rubrics simple with few categories aids grading efficiency, but approaches will vary.

Team performance reviews can also involve rubrics, such as the VALUE (Valid Assessment of Learning in Undergraduate Education) teamwork rubric (Rhodes, 2009). Developed by faculty teams across higher education institutions, the rubric (and others created as part of the Liberal Education and America’s Promise (LEAP) initiative; AAC&U, n.d.) is designed to assess undergraduate student learning outcomes. The Multistate Collaborative to Advance Student Learning project, involving 12 state higher education systems in the United States, approximately 100 institutions, and 300 trained faculty, entails the collection and rating of student work samples as part of a benchmarking process for these rubrics (AAC&U, 2017, (2018).

The VALUE teamwork rubric, or an instructor-created rubric, can be used at midpoint and at the end of a course. The former allows for formative self or peer evaluation (depending on instructor preference) and the opportunity for subsequent improvement while the latter, when used on its own, serves as summative assessment. It can be used in conjunction with team members awarding each other points that count toward the course grade and serve as an individual accountability measure since most team assignments are scored on a team basis and represent collective accountability.

A study using the VALUE teamwork rubric to compare students in face-to-face and online sections of the same course demonstrated that having students evaluate each other at midterm and the end of the course (with scores submitted to the instructor to compile and share with individual students while preserving rater anonymity) resulted in improved teamwork skills (as evidenced by peer ratings) for students in both online and face-to-face delivery modalities (Andrade, Miller, & Ogden, in press). As such, the practice fulfilled its purpose – using peer feedback to encourage reflection on performance, an element of HIPs.

Course communications

Course announcements in the learning management system are particularly relevant for online and blended courses where face-to-face meetings do not occur or occur on a limited basis, but are salient to all courses using learning management systems. Announcements can provide logistical information and whole-class feedback. For instance, students can receive tips for improving their peer reviews with reminders of rubric criteria and examples demonstrating various levels of adherence to the criteria. An excerpt from a course announcement follows with an instructor-annotated example of a student’s peer review of a team ePortfolio artifact. The instructor comments are aimed at focusing students on the rubric.

Make sure you do everything the task requires so that you get full points. See below for information contained in the rubric.

1. Comment on the ideas and how they helped you gain new insights. Make one suggestion for improvement.

2. Comment on the effectiveness of the design and writing quality and make one suggestion for improvement.

3. Respond to at least two posts from other students.

You need to do ALL of these in order to get full points. Note exactly how many things are being asked for (e.g., comment on ideas, make a suggestion; comment on the design; make a suggestion; respond to two posts). See below for examples with commentary in italics.

One thing that I realized is that it’s not just that each characteristic has a purpose to make a management team run smoothly, but each one has individual benefits and positive outcomes to any scenario! (Specific comment on the ideas/new insights; missing – make one suggestion for improvement on content/ideas) I really like your layout, and as I was reading each section, it helped me really get a good feel for what the management team’s goals and expectations were for the company. I think this is a great example of a well-functioning management team. (Specific comment on design; specific comment on new insights) One thing I noticed was after each section when you give an example, you always used the format “Example: ____” except for the leading section where you said “For example, ___”. It would be better to keep this consistent throughout the charter. (Specific comment on writing/organization; missing – overall comment on effectiveness of the design/writing)

Annotated examples such as this can help students focus on rubric criteria, evaluate their work, make changes, and monitor progress. This encourages metacognitive awareness through instructor structure and feedback. It should be noted that even in face-to-face courses, learning management systems are typically a central component of course delivery and enable instructors to manage course logistics through technology, using class time for hands-on learning activities.

Self-regulation

Direct feedback on individual and team assignments and whole-class feedback in learning management system announcements or by other means can encourage students to examine their methods, increase awareness of their approaches, set goals and implement changes, and improve their performance. In other words, instructors can guide students to self-assess, monitor, and evaluate their progress. These practices are related to the six dimensions of self-regulated learning, specifically motive (reasons or purpose for learning – why), methods (strategies for learning – how), time (prioritization, effective use of time – when), physical environment (a distractor-free study situation – where), social environment (help-seeking and collaboration – with whom), and performance (progress, reflection, and goals – what) (Dembo, Junge, & Lynch, 2006; Schunk & Zimmerman, 1994; Zimmerman, 1994; Zimmerman & Risemberg, 1997).

Consideration of these dimensions can help students increase their metacognitive awareness, or their ability to plan, monitor, and assess their learning and transfer effective practices to new contexts (Bransford, Brown, & Cocking, 2000; Pintrich, 2002; Weimer, 2012). Metacognition is an aspect of self-regulation; it is related to methods of learning or strategies (e.g., how one learns), and also to performance in the sense that students must monitor their progress and evaluate strategy effectiveness. Motive entails planning and goal setting which is informed by the performance dimension. The dimensions of time and physical and social environment direct students to specific factors that can impact their success. Self-regulated learning emphasizes responsibility for controlling the factors and conditions that impact learning (Dembo et al., 2006). Metacognition is purposively focused on awareness of one’s thinking and how one learns (Bransford et al., 2000; Weimer, 2012). The concepts of self-regulation and metacognition are complementary. The six dimensions of self-regulated learning, however, provide a specific guiding structure for both instructors and students, and as such are particularly useful in course design.

A number of pedagogical practices can be adopted to encourage self-regulation. Instructors can specify the purpose of assignments (motive/why), identify how assignments are connected to the development of cross-cutting skills (motive/why), provide guiding questions in reflection assignments (methods/metacognition/how), and offer other tips and strategies focused on various dimensions of self-regulated learning (methods, time use, physical/social environment; when). They can consistently refer to this information in various ways to individual students or to the whole class as they provide assignment feedback. This encourages students to focus on the tips and strategies provided, apply the suggestions, and increase their responsibility for learning. Reflection on performance as the result of this guidance can lead to changes in strategies and goals, and to improved learning (Tanner, 2012; Weimer, 2012).

I once had a student request a letter of recommendation for a prestigious internship. An excerpt from the instructions is included below. The letter was easy to write because the course had focused on all of the abilities requested in the application and I had first-hand knowledge of the student’s abilities in these areas. Because various dimensions of self-regulation were facilitated through the dialogue in the course, such as reflection, performance monitoring, and goal-setting, I was able to provide specific examples of the student’s learning based on team ePortfolio artifacts, which emphasized the desired cross-cutting skills. I like to share this experience with students so that they can see the relevance of the course components and the purpose of my feedback, which is to facilitate self-regulated learning as a means to the development of cross-cutting learning outcomes.

A letter of recommendation must be submitted from a professor at your college or university with whom you have completed an academic course. The professor should have a personal knowledge of your qualifications while addressing your abilities in one or more of the following areas: analytical ability; written or oral communication; teamwork and interpersonal skills; organization and time management; and/or leadership and personal initiative (Federal Deposit Insurance Corporation Internship Application).

In addition to self-regulated learning, the theory of transactional distance is also relevant. Providing structure and dialogue to encourage learner autonomy is the basic tenet of this theory (Moore, 2013). Structure occurs through course materials such as readings, assignments, and due dates, which are presented in a particular sequence to aid learning. Dialogue involves instructor facilitation of learning. Rubric-based feedback and course announcements are examples of dialogue and are aimed at helping learners develop autonomy in the sense of choice and self-direction (Andrade, 2017). Autonomy does not reflect learning in isolation but the ability to make appropriate choices, monitor learning, and apply effective learning strategies; in other words, autonomous learners have control over the learning process in a similar way to self-regulated learners. Dialogue facilitates this learner responsibility as the instructor interacts with students, guides them, and encourages analysis and reflection.

When structure and dialogue are extensive, autonomy decreases; however, instructors can guide learning through structure and dialogue to help students develop greater autonomy. Although the theory was developed to account for the transactional distance that occurs between students and instructors in distance education courses, the principles can be applied to course design and instruction more broadly, particularly as technology is a central part of many teaching contexts.

Measuring outcomes

Considering how to engage learners with course content is an important aspect of course design. HIPs and their elements were used as an overall guiding framework for the organizational behavior course highlighted in this article. The theoretical lenses of self-regulation, metacognition, and transactional distance guided instructor response with the goal of helping students develop lifelong learning skills. The next step in the process is to examine the effectiveness of the course design and pedagogical approaches, particularly in relation to the achievement of cross-cutting skills. The reflective assignments in the course provided opportunities to gain insights into the student experience. These included artifact reflections, teamwork reflections, and individual reflections. This section focuses on examinations of student reflections with the aim of identifying learning outcomes.

Review of the reflections over several different semesters of the course involved two frameworks: 1) UNESCO’s 21^st century skills, which are categorized as four pillars of learning – learning to know, do, live together, and be (Delors, 2013), and 2) the managerial functions known as POLC (planning, organizing, leading, and controlling) (Fayol, 1916). (For specifics regarding previous studies identifying the efficacy of the approaches outlined, see Andrade, 2019a, 2019b). The goal of measuring learning outcomes is to determine the effectiveness of the course design and pedagogies to inform curricular and pedagogical modifications. These instructor-led evaluations are also a form of action research.

UNESCO’s pillars of learning

UNESCO’s 21^st century skills are reflective of the cross-cutting skills valued by employers, as mentioned in the introduction. As such, they provide a means of understanding and categorizing students’ learning outcomes. The pillars also emphasize the skills and attributes needed for lifelong learning, which is a goal in many educational contexts and particularly, of higher education. Learning to know emphasizes knowledge acquisition; learning to do focuses on skills and methods; learning to live together reflects tolerance for diversity and connecting with and understanding others; and learning to be is about self-awareness, goals, and self-direction (Delors, 2013). These themes have been referred to in various ways throughout this article. ‘Skills and competencies; transformative potential; reflection; identity; active learning/student-centred engagement; employability; and assessment’ (Watty & McKay, 2015, p. 198) have all been recognized as benefits of ePortfolios, in which students document their learning in the course.

The analysis of student reflections provided evidence of all four UNESCO pillars (Andrade, 2019b). Aspects of learning to know included gaining deeper understanding of course topics through study and application. The latter involved providing examples of personal and workplace experiences where the concepts had been or could be practiced. The learning process was strengthened by interaction with peers as teammates discussed theories and concepts and as teams received feedback on their artifacts from classmates. More significantly, students applied the course concepts in their teams to work together more effectively. Students demonstrated achievement of learning to do as they described becoming better communicators, resolving conflict, planning and organizing, and being accountable. Students also gained abilities related to learning to live together as they came to appreciate the diversity on their teams and recognize what each person contributed. Students exhibited evidence of learning to be as they gained understanding of their personalities, expanded their skill sets, learned to be leaders and followers, increased their confidence, and overcame setbacks.

The POLC framework

To determine the extent to which students were achieving managerial skills through their course assignments and teamwork, reflections were also examined through the POLC framework to identify evidence of planning, organizing, leading, and controlling (Andrade, 2019a). These managerial functions are based on those identified by Henri Fayol (1916) and have long been accepted as useful categorizations of what managers do although some have objected that they do not reflect a typical day for a manager characterized by fragmentation and the need to prioritize (Lamond, 2004; Mintzberg, 1973). The analyses demonstrated that as students worked in teams, they had developed various managerial competencies. Planning and organizing were vital for completing artifact tasks and the community project. Students set goals, determined the actions needed to achieve those goals, and evaluated their success. Their team charters provided a structure that outlined roles and responsibilities. Leadership was largely shared although one person on each team was designated as the leader. Leadership was the weakest of the four areas with limited evidence of skills such as vision setting and motivating others. Controlling, or setting standards and reviewing performance, was evident as students learned from feedback and made changes accordingly.

Closing the assessment loop

The HIPs and elements of HIPs that were structured into the course encouraged students to develop the cross-cutting learning skills desired by employers. Students self-reported achievement of these learning outcomes in their reflections as well as in rubric-based assignments. This was particularly evident through the examination of reflections using the UNESCO pillars and POLC frameworks as lenses. This provided insights into how students experienced the course and informed curricular and pedagogical changes. For example, changing the content specifications for ePortfolio artifacts, adding milestones for the consulting project, alternating artifact submission with peer review assignments, developing various tutorials, revising assignment instructions and rubrics, including instruction on problem-finding and problem-solving tools to assist with projects, and requiring multiple drafts for the final project artifact are a few examples. These changes provided students with additional needed structure.

Implications and conclusions

The ideas shared in this article demonstrate how HIPs and their elements can be integrated into a course. Table 3 provides a review of the eight elements with examples of how they were operationalized.

Table 3. HIP Elements in the Course

Key Elements of High Impact Practices (Kuh & O’Donnell, 2013, p. 10)	Examples
Interactions with faculty and peers about substantive matters	Creation of team ePortfolio artifacts requires interaction about course concepts as do semester-long service learning projects.
Frequent, timely, and constructive feedback	Instructor and peer review occurs on a regular basis and is guided by rubrics, examples, and discussion.
Significant investment of time and effort by students over an extended period of time	ePortfolio artifacts help students develop new competencies and apply their learning; service learning project milestones due at various points during the semester guide students to more successful final projects.
Performance expectations set at appropriately high levels	Standards and expectations are evident through course rubrics and instructor and peer feedback.
Experiences with diversity, wherein students are exposed to and must contend with people and circumstances that differ from those with which students are familiar	Teamwork across multiple assignments over the course as well as interaction with company leaders and employees in the community project provide opportunities to experience many forms of diversity.
Periodic, structured opportunities to reflect and integrate learning	Peer reviews of artifacts, reflections on learning as part of the artifact assignment, team midterm reflections and individual final reflections increase students’ metacognition and self-regulation.
Opportunities to discover relevance or learning through real-world applications	ePortfolio artifacts on course concepts as well as the community service learning project provide extensive opportunities to apply learning.
Public demonstration of competence	ePortfolio creation and sharing of ePortfolio artifacts with peers as well as sharing project results with community organizations allow students to demonstrate various competencies gained in the course.

In addition to the HIPs and their elements, specific educational theories helped guide instructor response. These included self-regulation and the theory of transactional distance. These provide insights into how instructors can use approaches such as structure and dialog to encourage students to develop and refine their learning methods and increase their capacity for autonomy and self-direction.

Finally, the article provided examples of outcome measurement to inform improvements in teaching and learning. These measures can be enhanced through various lenses such as UNESCO’s pillars of learning and the management framework of POLC. While the UNESCO pillars are an effective way to measure the cross-cutting skills valued by employers, additional discipline-based principles of frameworks, such as the POLC framework for organizational behavior, might be used to analyze learning outcomes in different teaching and learning contexts.

The article demonstrates how the use of various frameworks and theories can be used to enhance teaching and learning, and particularly, students’ development of cross-crossing skills, to prepare them for successful employment and lifelong learning. It should be noted that these theories and approaches can be applied to any course and any delivery mode with appropriate adjustments for course size and subject matter as needed. For example, the HIPs and their elements can be implemented in any course. Practices such as teamwork, collaborative projects, community projects, writing-intensive assignments, and working with diverse others can be utilized in any discipline to help students develop cross-cutting skills. Similarly, rubrics, dialogue and facilitation of self-regulation through instruct0r communication, and peer review and reflection are practices broadly applicable across disciplines.

Such practices are also scalable. For example, teamwork is an effective strategy for highly enrolled course sections. Grading ten team ePortfolio artifacts is much more manageable than grading 60 individual artifacts, for instance. Team charters and video conferencing tools support effective teamwork in both face-to-face and virtual learning environments. The use of institutional resources such as writing centers can assist instructors in helping students increase their writing proficiency. Institutional teaching and learning offices typically provide assistance to instructors wanting to innovate, learn new technologies, or otherwise enhance their courses to engage students and improve their learning experiences. In essence, careful consideration of the list of HIPs and, in particular, the key elements of HIPs, can guide instructors in innovating within their own teaching contexts.

Disclosure statement

No potential conflict of interest was reported by the author.