Criterion 1
To what extent does your college's quality assurance process ensure that program-level learning outcomes for all programs of instruction are set, are consistent with the college mission and the programs' intended purpose, and are appropriate for the credential offered upon successful completion of the program?
1.1 Learning outcomes set for all programs of instruction are:
- consistent with the mission of the college;
- appropriate to the level at which the qualification is offered;
- consistent with the requirements of the Credentials Framework; and,
- appropriate to the occupational requirements of the program graduates
The Program Quality Policy (1.1.01) clearly mandates compliance (section 4, Principles 2 and 3) with published MTCU Program Standards and the Credentials Framework. Program Learning Outcomes are a pivotal component of Program Mapping, serving as a vital point of reference during various stages of the program life cycle (1.1.02): the new program proposal, curriculum development for a new program (1.1.03), program review and modification (1.1.04).
The fit with the College mission is a significant factor in securing internal College approvals for new programs. The New Program Proposal (1.1.06a 1.1.06b) requires the inclusion of Program Learning Outcomes as a key element describing the new program. These program standards serve as the reference for analysis of the program’s fit with the Strategic Plan and mission of the College, occupational requirements for graduates, and determination of the qualification to be awarded. In situations where the proposed program learning outcomes exceed or differ significantly from the provincial standards, it is usual College practice to seek guidance and approval from the Credentials Validation Service (1.1.07a 1.1.07b).
Program review is an integral strategy for ensuring the continued currency, relevance and responsiveness of the academic programs offered by the College. Program learning outcomes serve as an essential point of reference for program decisions during the comprehensive review completed for each program at least every five years (1.1.08a 1.1.08b). This year the College will also introduce an annual review for each program, as an enhancement of the current annual Program of Studies (POS) review. POS review integrates environmental scan data, feedback from program advisory committees, program support staff, faculty and students and recommendations from completed comprehensive program reviews. POS review will continue to be a component of the new Annual Review, but the new process will allow for a more direct and thorough analysis of Program Learning Outcomes annually and will accommodate more frequent changes and updates to the curriculum mapping matrix.
1.2 Clear statements of learning outcomes exist for all programs offered by the college, and are communicated effectively to students and faculty.
For the 2009 Audit, Mohawk was not able to demonstrate full compliance with the requirement for effective communication of program learning outcomes to students. That issue has since been addressed. Program Learning Outcomes for each post-secondary program offered by Mohawk College are available on the Mohawk College website (1.2.01a 1.2.01b), as an easily identifiable link within the program description (1.2.02a 1.2.02b). A brief explanation is provided for applicants to enhance understanding of these standards and emphasize their importance in their college study. In addition, program learning outcomes are included in many of the Program Handbooks (1.2.03a 1.2.03b 1.2.03c) developed by the individual academic departments. Students can also access information about Mohawk College programs in the full-time course calendar presented in hardcopy and digital format (1.2.04). For each program, students can easily go to the URL that links to the central program description where the program learning outcomes reside.
Access to the Program and Course Learning Outcomes is provided to faculty through CORE, Mohawk’s online application for Curriculum Overview, Research and Evaluation (1.2.05a 1.2.05b 1.2.05c 1.2.05d)
As part of curriculum mapping activities undertaken for new program development and program review, college practice requires faculty to access the Program Learning Outcomes and determine and document links between the program standards and learning outcomes developed for each program course. As part of curriculum mapping activities undertaken for new program development and program review, college practice requires faculty to access the Program Learning Outcomes and determine and document links between the program standards and learning outcomes developed for each program course (1.2.06a 1.2.06b 1.2.06c). Further details about these processes are provided in other sections of this report.
1.3 Program learning outcomes are operationally meaningful in that they:
- provide a sound basis for curriculum development and the design of student learning assessment; and,
- are internalized and used in the day-to-day work of program faculty.
Program learning outcomes serve as a pivotal curriculum component for faculty in the design of new programs. Under the guidance of college curriculum design specialists (1.3.01), and using approved support documentation, academic staff use the provincial curriculum standard – the program learning outcomes, as the starting point for the curriculum development process (1.3.02). As part of the approval process, the New Program Proposal template (1.3.03a 1.3.03b) requires proponents to submit a proposed Program of Studies, Program Description, and Program Maps, all of which are derived from the existing provincial standards. Consistent with accepted college practice, if no provincial standards exist for the proposed program, new program learning outcomes are vetted by the Credentials Validation Service, before college-specific approval processes are initiated.
The Centre for Teaching and Learning has continued to implement and refine the comprehensive Curriculum Mapping first introduced as the Learning Excellence project in 2005. Integral to the curriculum mapping process, “Clinking” is a strategy for identifying the link between the course learning outcomes and the Program Vocational Standards and Essential Employability Skills and assessing the extent to which these vocational standards are addressed in a specific course. This information is used to develop the Curriculum Mapping Matrix (1.3.04a 1.3.04b 1.3.04c), which then becomes the basis of program mapping and informs the processes of quality enhancement and program review.
Program Learning Outcomes (PLOs) are also vital points of reference in the comprehensive Program Review, completed at least once every five years. As part of Program Review, PLOs are carefully reviewed by faculty as an important standard against which proposed curriculum changes are assessed and validated. As one of the key steps in the process, faculty assess the quality of the curriculum, focusing on the individual course outlines, and the curriculum map, and engage advisory committee members and other relevant stakeholders in a review of the curriculum (1.3.05a 1.3.05b 1.3.05c 1.3.05d - in progress). For all of these steps, the Program Learning Outcomes serve as the essential standard that determines program relevance and guides decisions for change.
The Centre for Teaching and Learning has developed a multi-faceted strategy to assist faculty throughout various stages of the program life cycle. Supports include specific workshops addressing a variety of curriculum design issues, mapping strategies, and assessment practices; specific guidance and consulting sessions to assist in the development of new programs and program review. A comprehensive program review toolkit presented in eLearn@Mohawk guides administrators and faculty through the detailed analyses and assessments required for the Program Review Final Report. In addition to these formal structured processes for regular review and enhancement, faculty are encouraged to use the provincial program standards as everyday tools for sequencing course components, planning student activities, and determining appropriate assessment strategies.
1.4 Student requirements and obligations stated for each program are derived from and flow coherently from, the program's stated learning outcomes.
The curriculum mapping process referenced above originates with the Program Learning Outcomes and establishes a sound foundation upon which to base decisions regarding program admission requirements, course prerequisites and co-requisites. The information that is derived from “Clinking” clarifies relationships between courses and helps to make reliable decisions about the knowledge and skills that students require on entry to the program and to succeed in specific program courses.
The consistent and structured course outline template (1.4.01a 1.4.01b 1.4.01c 1.4.01d) accessed through CORE requires faculty to translate the results of the curriculum mapping process into decisions about how the course will be structured, and the nature of student assessment introduced for the course. The Course Outline requires participants to identify links between the individual course learning outcomes and elements and the Vocational Standards, Essential Employability Skills and General Education identified for the program.
An area that has been flagged for enhancement is alignment of teaching strategies and assessment approaches with course learning outcomes. Release of the new Program Information Policy (1.4.02), currently under development, will place renewed emphasis on Learning Plans as a required complement to course outlines, and provide a consistent structure which will accommodate more detail on course assessment strategies. This strategy will also eliminate potential variation across delivery sections.
In addition, the College is investigating technology solutions to either replace or enhance the current software application – CORE where course outlines reside. The current system lacks the responsiveness to support continued enhancement of the curriculum mapping process.
1.5 Program learning outcomes are consistent with the credential granted, the provincial program standards (where they apply), and the minimum essential expectations of the workplace. They are:
- reflected in the course outlines; and,
- used in prior learning assessments.
This requirement is clearly addressed in college policy and in practice. The Program Quality Policy (sections 4.2 and 4.3) (1.5.01) mandates that all Ontario college certificate, diploma, advanced diploma and graduate certificate programs offered by Mohawk College comply with the Minister’s Binding Policy Directive Framework for Programs of Instruction, the Credentials Framework, college system standards and programming principles, as well as external accrediting bodies where applicable. As a result, compliance with these standards is an assessment criterion that is built into development processes for new programs, as well as the program review cycle. As one of the steps in preparing the Program Review Final Report, reviewers are expected to analyze compliance with the Credentials Framework following completion of the Program Map and populate a chart giving evidence of the Depth, Breadth and Scope of that compliance (1.5.02).
The Program Information Policy is being developed and will replace the existing course outline policy and program of studies procedure (1.5.03a 1.5.03b 1.5.03c). The purpose of this new policy is to clearly outline requirements for the creation of curriculum, communication to students and approval of curriculum changes. The Program Information Policy will mandate clear and transparent linkages with provincial standards where they exist and other vocational standards as applicable. The Self-Evaluation Course Outline Checklist (1.5.04) developed by the Centre for Teaching and Learning as one of its general assessment criteria references the need to ensure that course outcomes are linked and characterized to vocational standards, essential employability skills and external standards where appropriate, and that the course contributes to attainment of the broader program outcomes.
Relevance with workplace standards and expectations is an essential focus at both the program development phase and in the regular review cycle. As appropriate, national standards are integrated into some accredited programs to ensure that graduates meet employer expectations. Annual market research is used to determine trends and guide the deployment of college resources. The structured new program approval process requires proponents to submit evidence of research of market trends and consultation with industry representatives (1.5.05).
In addition, as part of the comprehensive program review, the Review Team is required to provide an assessment of the contributions of Program Advisory Committee members to program development and renewal (1.5.06 1.5.07a 1.5.07b 1.5.07c 1.5.07d). Key Performance Indicator satisfaction results from graduates and employers also provide important validation of program relevance. At regular intervals throughout the program life cycle, feedback is sought from Program Advisory Committee members and focus groups of industry stakeholders. (see focus group docs and minutes) . Program faculty are also important sources of information regarding workplace requirements. Using information gained through professional networking, attendance at conferences, and other professional development initiatives, they are able to base recommendations for change on first-hand information on industry trends.
The Program Information Policy also positions the master course outline as the official document for reference in prior learning assessments. Most PLAR challenges addressed by the college are course-based, so the Course Outline becomes the significant reference document. The Course Outline is used as an essential tool in assisting potential PLAR candidates to determine their eligibility to challenge a specific course. Appendix A of the PLAR Policy (1.5.08a 1.5.08b) specifies that the learning to be credited must be consistent with current Mohawk College course offerings and satisfy the current course/program outcomes. The PLAR Policy was recently revised to provided added clarity and improve interpretation around the process. As referenced earlier, the curriculum mapping process ensures appropriate and relevant linkages between individual program courses and the program standards.
1.6 Program learning outcomes are consistent with MTCU Provincial Program Standards where they exist.
Throughout the program life cycle, provincial program standards remain an ongoing focus. The need to maintain consistency with provincial standards is formalized in several College policies, including the Program Quality Policy (1.6.01), the General Education Policy (1.6.02), and the Program Information Policy (1.6.03).
As outlined in section 1.3 above, program standards where they exist are a key point of reference in the development of new programs, and in the ongoing review and refinement of existing programs. In new program development, the provincial program standards inform and validate the development of the program title, description, program of studies and individual course descriptions. As the next curriculum design step, the curriculum mapping process provides valid and reliable assessment of the manner and the degree to which each course in the program addresses vocational standards, essential employability skills and general education. This curriculum mapping matrix is reviewed and updated as necessary in the comprehensive program reviews (1.6.04a 1.6.05b). The curriculum map may also be updated in response to changes in the vocational standards as the result of program renewal undertaken by MTCU.
Since the last Audit, Mohawk has worked to address compliance issues associated with General Education requirements. All programs now meet minimum standards, but work is ongoing to enhance the general education offerings. For fall 2014, as part of a two-phased plan, all 2-hour general education courses will be increased to 3 hours. In Phase 2, the College will review general education offerings to ensure that all courses fully meet the requirement as discrete and separate from vocational courses.
1.7 Program learning outcomes are regularly reviewed to ensure they remain relevant, and changes are made when necessary, including consequential changes to programs, courses, and assessments.
Again, the need for regular review of program learning outcomes is reinforced in a suite of academic policies and procedures developed by the College (Program Quality Policy, Program Information Policy, Program of Studies Procedure, Program Promotion and Graduation Requirements). In practice, the Program Quality Assurance Process in place at Mohawk mandates two approaches to program review to ensure continued program alignment with the mission, vision, and values of the College; the College’s Strategic and Academic Plans; provincial program standards and directives; and industry accreditation standards where applicable. Review of program learning outcomes is a significant feature of both review phases.
Effective this year, Mohawk has introduced a new requirement for an annual review for every Mohawk program. The annual review, facilitated through established processes and criteria applied college-wide, will involve review and revision of curriculum in response to College strategic directions, feedback from faculty, Program Advisory Committee members, students and program data. This new review process builds on and integrates the annual Program of Studies Review (1.7.02) which has been an ongoing process at the College. Depending on review findings, this annual process may result in recommendations for changes to admission standards and/or the program of studies, including updates to the curriculum mapping matrix. The new annual review will add more structure and depth to program decision-making in the short-term.
In addition, every program offered by the College undergoes comprehensive program review every 5 years, using established processes and criteria. The comprehensive review, which typically spans a period of 12 months and culminates in the preparation of a final report (1.7.03a) incorporates data collection and analysis related to program quality , market demand, student satisfaction and success, and labour market and employment demand. Recommendations for change are articulated in an Action Plan (1.7.04a 1.7.04b - in progress), implementation of which is monitored through the Annual Review Process. The Comprehensive Program Review is also aligned with the program closure and consolidation process (1.7.05) that addresses changes to the college’s overall programming mix.
1.8 The capabilities of program graduates, including knowledge, understanding, skills, and attitudes are consistent with the intended program learning outcomes.
Assessment strategies built into each course are an internal mechanism that demonstrates how well students individually have addressed learning requirements articulated in the course learning outcomes. As demonstrated elsewhere in this report, through the mapping matrix, course learning outcomes in aligned with the overall program standards.
At the program level, the accomplishments of program graduates are an indication that the learning captured in program learning outcomes has actually been achieved. The College relies on external data gathered through Key Performance Indicator surveys, and feedback from Program Advisory Committee members and other external stakeholders to assess graduate performance. Key Performance Indicators related to Graduate Employment and Employer Satisfaction are generally regarded as reliable evidence of graduate skills : (1.8.01). For 2012-2013, Mohawk College scored highest in the province in Graduate Satisfaction and Employer Satisfaction with scores of 80.8 and 94.3% respectively (1.8.02).
Ongoing feedback from Program Advisory Committees, which include program graduates and employers in their membership roster, is also a useful measure of graduate performance (1.8.03a 1.8.03b 1.8.03c 1.8.03d). As part of the Program Review Final Report, the review team is required to describe the ways in which the Advisory Committee has contributed to program development and renewal (1.8.04). Feedback on graduate skills is identified as one of the components of the Committee’s contributions.
In addition, as another component of the Program Review Final Report, program staff are required to collect, analyze and report on data gathered during strategic curriculum discussions (1.8.05). Various stakeholders for the program, including Program Advisory Committee members, employers, and graduates, are invited to these discussions to provide feedback on various program issues, including how well program graduates perform in the workplace. Graduates participating in these sessions are able to provide valuable first-hand experience on how well their program of studies has prepared them for employment responsibilities. The information gathered during these sessions provides a useful indication of the ongoing relevance of program outcomes as well as areas where changes would be desirable (1.8.06a 1.8.06b - in progress).
Assessment of Criterion #1
Does the evidence provided for each of the 8 requirements indicate the criterion is Met, Partially Met, or Not Met using the definitions provided on Page 20 of the PQAPA Orientation and Training Manual?
In the event the Criterion is rated as Partially Met or Not Met, what plans are being identified to improve on this?
Mohawk College has worked diligently over the years to ensure alignment with MTCU standards and the Credentials Framework. Provincial learning outcomes are firmly established as the initial and essential point of reference for decisions relating to the program of studies, program content, and assessment strategies. The title and duration of each program are set within the operating framework established by MTCU. In the period since the last audit, additional process refinements have been introduced to further strengthen the essentiality of program standards in program development and review. The most significant of these is the introduction of the annual program review which expands the focus of the program of studies review in place for the previous audit and introduce more structure to ensure regular and relevant updates to the curriculum mapping matrix prior to the comprehensive program review. The introduction of the toolkit for the comprehensive review also ensures that all essential points of reference are addressed consistently in every review. Consequently, with confidence, we assert that all eight requirements of Criterion # 1 have been met.
The chart below identifies initiatives currently in progress or planned to enhance quality assurance at Mohawk.
Initiative |
Responsibility |
Timeline |
Current Status |
|---|---|---|---|
Implement Annual Program Review Process |
Program Quality |
Pilot in Spring 2014; |
Draft process developed |
Test new Course outline systems / opportunities including Algonquin and D2L |
Program Quality |
Pilot in 2014/15 |
Preparing for pilot |
Implement program information policy |
Program Quality |
Full implementation 2014/15 |
Policy under development |