Standard Two: Planning and Evaluation
Description
The planning and evaluation process at this institution is a systematic and ongoing approach to the assessment of how well and to what extent the institution performs its mission and achieves it goals within its changing internal and external environments. Measurements of
performance indicators for each goal and short and long range objectives are used in the continuous improvement process to ensure mission and goal accomplishment and to provide feedback in support of budgetary and operational planning process.
The MTCS Strategic Plan serves as a guide for decision making and focuses the energies of the system on shared priorities. The plan
directs the system in achieving its mission and establishes an environment that is conducive to effective, collaborative planning and evaluation. It defines the benchmarks by which overall system effectiveness and progress are measured. The guiding principles in the
strategic plan specifically address the MTCS commitment to creating and sustaining an environment that encourages teaching, learning, and scholarship (ref. 3.1). Through delineated emphasis on service to the
citizens of Maine, to students, to employees, and to the college environment in general, the guiding principles outline the MTCS vision of excellence in education.
The NMTC planning process utilized in developing the five-year plan is both a top down and a bottom up process. It is top down in the sense
that the Maine Technical College System goals and assumptions define the direction of the college. The system goals are supported by the college
goals, which are framed by planning assumptions resulting from an environmental scanning process and guide. These provide the planning
parameters for college personnel involved in the planning process. The process is a bottom-up process in the sense that area, division, and department personnel develop action plans which are framed by the campus plan. Each area develops action plans containing measurable objectives, timelines for accomplishment, including accountability, and budget impact statements. Area administrators assemble the composite plan by
integrating information from action plans within their respective areas under the appropriate objective for each area. The area objectives are
listed under the appropriate college goal.
NMTC has adopted a mission-based assessment model. The mission and
goals of each major area flow directly from the mission and goals of the college and support the MTCS goals and assumptions. Program outcomes
are evaluated using multiple measures, determined by the needs and resources of the individual departments. Student learning is measured
using both direct and indirect measures. Assessments of students'
cognitive, behavioral, and affective learning are all part of the data collection process.
Faculty and staff provide a summary review of assessment efforts in their evaluative annual action plans. These serve two purposes:
* Provide faculty, through the continuous improvement process, indications of successes and areas for improvement of student learning, and
* Provide the strategic planning committee with information to use in its evaluation of the effectiveness of institutional mission and goal attainment. Assessment is also tied to the budget acquisition process. Cost-center managers provide assessment results to area administrators who base their requests for capital and personnel on the information provided. Each area specifies its own timetable for collection of data and process improvement. All have data collection within the academic year cycle, but some have ongoing assessments requiring longer time periods, such as alumni surveys done every year, external agency testing, program specific accreditation self-studies, and program completion information. Such evaluations and assessments are included in each area supervisor's annual report.
Each organizational division/unit of the college follows an assessment process, which includes:
* A mission statement which supports the mission of the college, * A set of goals, operational objectives, and action plans which support the goals of the college, and * Measures of unit effectiveness. NMTC has defined the assessment of student learning as the systematic collection, examination, and interpretation of qualitative and quantitative data about individual student learning and the use of that information to both document and improve student learning. The student academic achievement plan involves the following steps: * Setting mission-based goals; * Explicitly stating the goals for student learning (course syllabi); * Systematically gathering data to determine how well goals are being met utilizing a variety of methods to look at behavioral, cognitive and affective learning; * Summarizing and evaluating the data and communicating those results to students, faculty, staff, and the college community; and * Utilizing a continuous improvement process to improve performance in pursuit of excellence in teaching and learning. The assessment plan for measuring student academic achievement has three requirements:
* That it be well thought-out and relevant, * That it be beneficial and easy to understand, and * That the programs have the resources and commitment to implement it. A major goal of NMTC's academic area is to offer programs that meet student and community needs for diversified and innovative programs. Defining a single set of outcomes to meet these diversified needs is difficult. As a guide, the MTCS board of trustees has adopted a set of core abilities derived from wide ranging input provided by people representing business and industry, labor, the technical colleges and other institutions. These core abilities are imbedded in some specific
outcomes, which are measurable across the curriculum.
Measurement of student achievement begins with the initial testing of students prior to matriculating in a program. Several options for
student placement are available ranging from developmental education, direct program placement, and advanced program placement standing through qualifying examinations, articulation agreements, transfer credit, and/or work experience. Once matriculated, academic achievement
begins with course specific outcomes. Each course in each discipline
has desired outcomes that are determined by faculty and published in each course syllabi. Course outcomes are measured by any combination of
the following, specific to the program:
* Direct learning assessment, (classroom, clinical, occupational lab, co-op experience, etc.) * Pre and post-testing, * Standardized tests with national norms, * Final examinations, * Licensure examinations, and * Other course-imbedded measurements defined by faculty. Each department determines specific educational outcomes at the program level. Program level outcomes encompass a much broader range of competencies than those at the course level. Program outcomes are
linked to the individual academic department's goals and objectives, which are linked to the institution's goals and objectives. Specific measurements of academic achievement, along with who is responsible for collecting the data, what kinds of data are to be collected, and how often are under the domain of the vice-president. Measurements include,
but are not limited to:
Direct measures
Common finals
Capstone courses/projects
Performance on licensure examinations
Classroom assessment
Indirect measures Alumni and employer surveys
Retention and transfer studies
Job placement data
Student Satisfaction Survey
The current program review process provides for evaluation of student academic achievement and institutional effectiveness. This process consists of surveying current students, alumni, program advisory committee members, and employers, all of whom provide much needed input into program effectiveness.
The continuous ongoing loop between assessment and improvement is enhanced by widespread communication of the results, followed by implementation of changes. At the course level, there are classroom
assessment techniques which give immediate, formative feedback to students. Competency evaluation, standardized and locally developed
exams also measure student learning at the course level.
At the program level, assessment is documented in the annual action plan report and program review. The vice president and department chairs
evaluate the assessment results by program/department for
recommendations. The vice president summarizes the assessment results
and develops a plan to correct deficiencies that may exist. Based on these results the vice president submits a report to the strategic planning committee and the curriculum committee (if needed) as part of the continuous improvement process. The strategic planning committee
utilizes the data to evaluate institutional effectiveness in mission and goal attainment.
Lastly, department and/or program action plans are evaluated to assess whether or not the institution accomplished the goals set to improve student learning and did so with excellence. Following this review, new
goals are set and the process begins anew. In general this assessment
process follows an annual cycle. See Appendix E for a description of
NMTC's institutional effectiveness process and associated process flow chart.
The strategic planning process requires annual review of specific goals, objectives and activities at the college level as well as at the system level. Each college completes an annual assessment and reports its findings to the MTCS. The findings guide future direction and decision
making for the colleges as well as for the system.
The board of trustees is apprised annually of the results of this self-assessment process and incorporates the reviews into its continuous improvement efforts. This appraisal occurs via a written report and/or
a presentation by one or more faculty to the MTCS educational policy committee. Typically there is an opportunity for questions and dialog relative to the program and an opportunity to view the laboratory/shop facilities associated with the program. Through this process, the board
seeks assurances that the colleges are attaining the highest quality of education and service; that graduates are developing appropriate scholarship; and that the colleges are inspiring lifelong learning. Appraisal
The strategic planning process is designed to be a "community" effort with faculty, staff, managers, student representatives and the college's general advisory council providing input. It is intended for this
broad-based involvement of the entire campus community to generate ownership and yield strong employee support for the process. The process is accomplished through regular meetings of the strategic planning committee. All major functional areas are represented and
involved in the process. This process is essential if NMTC is to
effectively utilize all resources to provide a broad mix of services to its service area.
Even though a thorough framework for planning and assessment has been developed, the degree of implementation and communication of assessment results varies considerably. The process for documentation and
reporting needs refinement in order that it more fully communicate assessment findings to all constituencies. Assessment and documentation
of occupational skill attainment by students is a strong part of each department's assessment process. Related "lifelong learning skills"
outlined in the MTCS Skills for the 21st Century report have been identified for most programs and presented to the respective program advisory committees for review and endorsement. However, continued work
needs to occur on assessing student achievement of these "lifelong learning skills.?
As the planning and assessment process has become more complex, it has also become a very labor-intensive process which demands much input from faculty and staff if it is to provide adequate support for institutional change and growth. A clearer understanding of the responsibilities of each department and individual staff members for the process is needed. Also needed is a more organized manner by which assessment data is reviewed and reported to facilitate adjustments at the department level and to integrate results in the planning process at all levels of the institution. A clearer delineation of time lines for completion of various activities is also needed to ensure all areas of the institution are involved in the process in a timely manner.
Projection
The college recognizes the need to enhance its ability to gather data and interpret data for the purposes of assessment and planning. In the summer of 2002, the college conducted a study to assess its present capabilities in this area and to look at models to enhance institutional research capabilities. The results of the study have been presented to
the college?s strategic planning committee and the management team. The study focused on four key areas:
1. Summarize historical and existing institutional research strategies 2. Secure representative job descriptions for institutional research positions
3. Create a comprehensive inventory of current information assets (data) on the campus
4. Formulate recommendation for integrating institutional research processes into NMTC?s organizational structure and procedures The recommendations in this study will provide the foundation for further development of the college?s institutional research processes and capability. It is envisioned that by the end of the academic year 2002-03, the college will have implemented an institutional research process that will better serve the college and will provide for a solid foundation from which to make decisions.
The institutional effectiveness process will continue to be refined and become a regular part of the operational routine for all departments and personnel. Appropriate timelines for assessing outcomes, reporting results, and revising action plans will be identified and clearly communicated. Department and area managers will review the process with staff members annually to help ensure understanding and awareness of the process and its importance to the growth and development of NMTC. It is envisioned that an enhanced, more timely reporting process would also facilitate increased awareness of faculty and staff and serve to create a stronger "buy in" to the process. The planning and assessment process at NMTC will continue to grow and be refined and play an ever increasingly important role in institutional growth and change. |