NMCC - Planning and Evaluation
  NMCC  
 
Standard Two: Planning and Evaluation    
 
Description
 

The planning and evaluation process at this institution is a systematic

and ongoing approach to the assessment of how well and to what extent

the institution performs its mission and achieves it goals within its

changing internal and external environments. Measurements of

performance indicators for each goal and short and long range objectives

are used in the continuous improvement process to ensure mission and

goal accomplishment and to provide feedback in support of budgetary and

operational planning process.
 

The MTCS Strategic Plan serves as a guide for decision making and

focuses the energies of the system on shared priorities. The plan

directs the system in achieving its mission and establishes an

environment that is conducive to effective, collaborative planning and

evaluation. It defines the benchmarks by which overall system

effectiveness and progress are measured. The guiding principles in the

strategic plan specifically address the MTCS commitment to creating and

sustaining an environment that encourages teaching, learning, and

scholarship (ref. 3.1). Through delineated emphasis on service to the

citizens of Maine, to students, to employees, and to the college

environment in general, the guiding principles outline the MTCS vision

of excellence in education.
 

The NMTC planning process utilized in developing the five-year plan is

both a top down and a bottom up process. It is top down in the sense

that the Maine Technical College System goals and assumptions define the

direction of the college. The system goals are supported by the college

goals, which are framed by planning assumptions resulting from an

environmental scanning process and guide. These provide the planning

parameters for college personnel involved in the planning process.

 

The process is a bottom-up process in the sense that area, division, and

department personnel develop action plans which are framed by the campus

plan. Each area develops action plans containing measurable objectives,

timelines for accomplishment, including accountability, and budget

impact statements. Area administrators assemble the composite plan by

integrating information from action plans within their respective areas

under the appropriate objective for each area. The area objectives are
listed under the appropriate college goal.
 
NMTC has adopted a mission-based assessment model. The mission and

goals of each major area flow directly from the mission and goals of the

college and support the MTCS goals and assumptions. Program outcomes

are evaluated using multiple measures, determined by the needs and

resources of the individual departments. Student learning is measured
using both direct and indirect measures. Assessments of students'

cognitive, behavioral, and affective learning are all part of the data

collection process. 
 

Faculty and staff provide a summary review of assessment efforts in

their evaluative annual action plans. These serve two purposes:
 

    * Provide faculty, through the continuous improvement process,

      indications of successes and areas for improvement of student

      learning, and

    * Provide the strategic planning committee with information to use

      in its evaluation of the effectiveness of institutional mission

      and goal attainment.

 
 

Assessment is also tied to the budget acquisition process. Cost-center

managers provide assessment results to area administrators who base

their requests for capital and personnel on the information provided.

Each area specifies its own timetable for collection of data and process

improvement. All have data collection within the academic year cycle,

but some have ongoing assessments requiring longer time periods, such as

alumni surveys done every year, external agency testing, program

specific accreditation self-studies, and program completion

information. Such evaluations and assessments are included in each area

supervisor's annual report.
 

Each organizational division/unit of the college follows an assessment

process, which includes:
 

    * A mission statement which supports the mission of the college,

    * A set of goals, operational objectives, and action plans which

      support the goals of the college, and

    * Measures of unit effectiveness.

 
 

NMTC has defined the assessment of student learning as the systematic

collection, examination, and interpretation of qualitative and

quantitative data about individual student learning and the use of that

information to both document and improve student learning.

 

The student academic achievement plan involves the following steps:

 

    * Setting mission-based goals;

    * Explicitly stating the goals for student learning (course syllabi);

    * Systematically gathering data to determine how well goals are

      being met utilizing a variety of methods to look at behavioral,

      cognitive and affective learning;

    * Summarizing and evaluating the data and communicating those

      results to students, faculty, staff, and the college community; and

    * Utilizing a continuous improvement process to improve performance

      in pursuit of excellence in teaching and learning.

 
 

The assessment plan for measuring student academic achievement has three

requirements:
 

    * That it be well thought-out and relevant,

    * That it be beneficial and easy to understand, and

    * That the programs have the resources and commitment to implement it.

 
 

A major goal of NMTC's academic area is to offer programs that meet

student and community needs for diversified and innovative programs.

Defining a single set of outcomes to meet these diversified needs is

difficult. As a guide, the MTCS board of trustees has adopted a set of

core abilities derived from wide ranging input provided by people

representing business and industry, labor, the technical colleges and

other institutions. These core abilities are imbedded in some specific
outcomes, which are measurable across the curriculum.
 

Measurement of student achievement begins with the initial testing of

students prior to matriculating in a program. Several options for

student placement are available ranging from developmental education,

direct program placement, and advanced program placement standing

through qualifying examinations, articulation agreements, transfer

credit, and/or work experience. Once matriculated, academic achievement
begins with course specific outcomes. Each course in each discipline

has desired outcomes that are determined by faculty and published in

each course syllabi. Course outcomes are measured by any combination of
the following, specific to the program:
 

    * Direct learning assessment, (classroom, clinical, occupational

      lab, co-op experience, etc.)

    * Pre and post-testing,

    * Standardized tests with national norms,

    * Final examinations,

    * Licensure examinations, and

    * Other course-imbedded measurements defined by faculty.

 
 

Each department determines specific educational outcomes at the program

level. Program level outcomes encompass a much broader range of

competencies than those at the course level. Program outcomes are

linked to the individual academic department's goals and objectives,

which are linked to the institution's goals and objectives. Specific

measurements of academic achievement, along with who is responsible for

collecting the data, what kinds of data are to be collected, and how

often are under the domain of the vice-president. Measurements include,
but are not limited to:
Direct measures
Common finals
Capstone courses/projects
Performance on licensure examinations
Classroom assessment

      Indirect measures

Alumni and employer surveys
Retention and transfer studies
Job placement data
Student Satisfaction Survey
 
 

The current program review process provides for evaluation of student

academic achievement and institutional effectiveness. This process

consists of surveying current students, alumni, program advisory

committee members, and employers, all of whom provide much needed input

into program effectiveness. 
 

The continuous ongoing loop between assessment and improvement is

enhanced by widespread communication of the results, followed by

implementation of changes. At the course level, there are classroom

assessment techniques which give immediate, formative feedback to

students. Competency evaluation, standardized and locally developed
exams also measure student learning at the course level.
 

At the program level, assessment is documented in the annual action plan

report and program review. The vice president and department chairs
evaluate the assessment results by program/department for
recommendations. The vice president summarizes the assessment results

and develops a plan to correct deficiencies that may exist. Based on

these results the vice president submits a report to the strategic

planning committee and the curriculum committee (if needed) as part of

the continuous improvement process. The strategic planning committee

utilizes the data to evaluate institutional effectiveness in mission and

goal attainment.
 

Lastly, department and/or program action plans are evaluated to assess

whether or not the institution accomplished the goals set to improve

student learning and did so with excellence. Following this review, new
goals are set and the process begins anew. In general this assessment
process follows an annual cycle. See Appendix E for a description of

NMTC's institutional effectiveness process and associated process flow

chart.
 

The strategic planning process requires annual review of specific goals,

objectives and activities at the college level as well as at the system

level. Each college completes an annual assessment and reports its

findings to the MTCS. The findings guide future direction and decision
making for the colleges as well as for the system.
 

The board of trustees is apprised annually of the results of this

self-assessment process and incorporates the reviews into its continuous

improvement efforts. This appraisal occurs via a written report and/or

a presentation by one or more faculty to the MTCS educational policy

committee. Typically there is an opportunity for questions and dialog

relative to the program and an opportunity to view the laboratory/shop

facilities associated with the program. Through this process, the board

seeks assurances that the colleges are attaining the highest quality of

education and service; that graduates are developing appropriate

scholarship; and that the colleges are inspiring lifelong learning.

 
Appraisal
 

The strategic planning process is designed to be a "community" effort

with faculty, staff, managers, student representatives and the college's

general advisory council providing input.   It is intended for this

broad-based involvement of the entire campus community to generate

ownership and yield strong employee support for the process.  The

process is accomplished through regular meetings of the strategic

planning committee. All major functional areas are represented and
involved in the process. This process is essential if NMTC is to

effectively utilize all resources to provide a broad mix of services to

its service area.
 

Even though a thorough framework for planning and assessment has been

developed, the degree of implementation and communication of assessment

results varies considerably.   The process for documentation and

reporting needs refinement in order that it more fully communicate

assessment findings to all constituencies. Assessment and documentation

of occupational skill attainment by students is a strong part of each

department's assessment process.   Related "lifelong learning skills"

outlined in the MTCS Skills for the 21st Century report have been

identified for most programs and presented to the respective program

advisory committees for review and endorsement. However, continued work

needs to occur on assessing student achievement of these "lifelong

learning skills.?
 

As the planning and assessment process has become more complex, it has

also become a very labor-intensive process which demands much input from

faculty and staff if it is to provide adequate support for institutional

change and growth.   A clearer understanding of the responsibilities of

each department and individual staff members for the process is needed.

Also needed is a more organized manner by which assessment data is

reviewed and reported to facilitate adjustments at the department level

and to integrate results in the planning process at all levels of the

institution. A clearer delineation of time lines for completion of

various activities is also needed to ensure all areas of the institution

are involved in the process in a timely manner.
 
Projection
 

The college recognizes the need to enhance its ability to gather data

and interpret data for the purposes of assessment and planning. In the

summer of 2002, the college conducted a study to assess its present

capabilities in this area and to look at models to enhance institutional

research capabilities. The results of the study have been presented to

the college?s strategic planning committee and the management team. 

 
The study focused on four key areas:

1.    Summarize historical and existing institutional research strategies

2.    Secure representative job descriptions for institutional research

positions

3.    Create a comprehensive inventory of current information assets

(data) on the campus

4.    Formulate recommendation for integrating institutional research

processes into NMTC?s organizational structure and procedures

 

The recommendations in this study will provide the foundation for

further development of the college?s institutional research processes

and capability. It is envisioned that by the end of the academic year

2002-03, the college will have implemented an institutional research

process that will better serve the college and will provide for a solid

foundation from which to make decisions.
 

The institutional effectiveness process will continue to be refined and

become a regular part of the operational routine for all departments and

personnel. Appropriate timelines for assessing outcomes, reporting

results, and revising action plans will be identified and clearly

communicated.   Department and area managers will review the process

with staff members annually to help ensure understanding and awareness

of the process and its importance to the growth and development of

NMTC. It is envisioned that an enhanced, more timely reporting process

would also facilitate increased awareness of faculty and staff and serve

to create a stronger "buy in" to the process. The planning and

assessment process at NMTC will continue to grow and be refined and play

an ever increasingly important role in institutional growth and change.



 
Top! Top!