The Academic Relations Committee has updated the AAR's Guide
for Reviewing Programs in Religion and Theology. This document provides
those involved with program review considerations for conducting a program
review and choosing external reviewers. We have also included guidance for those
who undertake the task of being an external reviewer.
This document is written primarily to provide faculty and
department or program leadership with suggested steps and strategies for
conducting a self-study and external review. These guidelines do not aim to
prescribe elements of an effective program; rather, they provide practical
suggestions for conducting the most constructive internal and external program
evaluations. These guidelines suggest strategies around similarities across
institutions and thus, of necessity, overlook specific institutional contexts.
Users of this document should always follow institutional guidelines first.
This document is modeled on a document approved by the Academic Relations Task Force in 1999. That earlier guide,
which was modeled on guides produced for other fields, was meant to be a work
in progress, and this document aims to be the same. We ask for your comments and suggestions, especially your assessment of its helpfulness for your own
review process. In the future, we hope
to work on different iterations of the Guide more closely tailored to specific
institutional segments, e.g., private liberal arts college, public university,
church-related college, etc.
Department- or Program-Level Review
All institutions engage in some form of program review,
either routinely on campus or as part of accreditation processes (or both).
Many campuses follow a predictable calendar of review (e.g. every 5 years,
every 7 years, once each decade, or in conjunction with accreditation cycles);
others demand reviews when necessary (e.g. in the face of tightening resources,
as the result of a request for a change in status, or amid administrative
concerns about a program’s productivity). Program reviews are always designed
around specific departmental/program and institutional realities and needs, so
there is no one-size-fits-all approach. All reviews should keep institutional
mission and strategic plans in mind and all should bear in mind the point of
view of the administration, which must consider competition for resources on
campus and competition for students more broadly.
Although they are labor intensive and sometimes contentious,
program reviews can provide opportunities to raise questions and discuss issues
with the shared goal of program improvement. They can be a tool for
reconfiguring a program, for lobbying for new resources, or for arguing to
preserve existing resources. A program review can allow any department or program
to advocate for itself and improve on deficiencies or build on strengths
commensurate with the institutional mission.
That said, most review documents should be designed to speak
to various stakeholders, including administration, faculty in other departments
or programs, faculty within the department or program, and any reviewers
external to campus (a committee or an individual). These multiple audiences can
be particularly tricky for departments of religious studies, whose work is
often misunderstood by their colleagues. Some would argue that the most
important audience for the entire process is those administrators who have
power over a program, from controlling resources to making decisions about
program viability. Others would privilege the department or program faculty as
the key audience for any review. Still others write a self-study primarily for
external reviewers. In every case, the stakes of program review can be high.
Awareness of and focus on the most critical stakes and stakeholders are essential
to productive program review.
Steps in the review process
Most reviews follow this sort of timetable and generally
take a full academic year to complete:
for the review (collection of data)
the self-study and writing a narrative
review (by faculty and/or administration)
review (by a visiting committee of faculty from other institutions)
to external review accompanied by plan going forward
Preparation for the review:
The department and programs should plan one or more meetings
to discuss the review process and the stakes. Affiliated faculty and students
should also be invited to participate, if appropriate. The department or
program should take advantage of the wisdom provided by departments and
programs on their own campus who have recently gone through the process.
The best program reviews allow a department or program to
highlight their contributions to the institutional mission, the success of
their own (and, when appropriate, the general education) curriculum, the
scholarship conducted by their faculty and students, and their engagement in
the life of the institution and/or its community. Gathering evidence of those
contributions is a key part of the early stages of any review process.
Reviews are usually driven by administrators who set
parameters (e.g. specific foci for the study,
who will serve as external reviewers [both on- and off-campus], and
opportunities for departmental or program response). If there is input
from the department or program as to the design of the review, the department
or program should think in terms of the results they want and the kinds of
questions and approaches that would best lead to those results.
In an ideal world, department or program faculty should talk
together regularly about program goals and strategies to achieve those goals.
Ongoing conversation makes for an easier self-study and external review
process. In the absence of ongoing conversations, for whatever reason, a program
review can provide an opportunity to discuss and assess issues together. A
program review can be a formative experience for all involved, resulting in
changes as far ranging as updates to the department or program website to a
fully revised curriculum.
Also in an ideal world, departments and programs will track
their productivity at least annually. In general, departments and programs
should regularly track, for instance, faculty and student awards, publications
of all types, internal and external funding awards, alumni achievements,
faculty service on campus and beyond, and student testimonies of teaching and
mentoring. Those documents will be collated and used to assemble the
Departments and programs should be sensitive to the kinds of
information administrators might expect. In many cases, numbers count (e.g. SCH
or major numbers, enrollments, job placement of students, and external
funding). Departments and programs may be asked to analyze enrollment trends
over time; to examine the balance of introductory courses vs. advanced courses;
to track patterns of over-enrollment and under-enrollment; and to assess
whether current course offerings meet department or program learning goals by
examining longitudinal assessment data. Some numbers will need significant
context, which can mean doing institutional research beyond the department or
Conducting the self-study and writing a narrative
The self-study document sets the tone for the entire review.
This document is written for multiple audiences and should serve as a candid
assessment of department or program strengths and weaknesses. The focus of the
self-study should almost always include an in-depth critical examination of the
curriculum and student learning, as those metrics most directly affect workload
and resource decisions. The department or program should thus be committed to
examining the current curriculum, the balance of courses taught at various
levels, and the department or program and course-level goals for student learning.
Consideration of the curriculum should include a review and analysis of new directions or innovations within the discipline more
Preparation for writing the self-study involves amassing
data from various sources, as described above. Most institutions have some
institutional research outlet that will provide data over time. Some
departments or programs will want to develop an alumni survey to find out about
career placement, for a retrospective evaluation of learning from former students,
and to find out about outstanding alumni. Some data is more difficult to
gather, such as the role of the department or program in extracurricular
activities or community engagement. Annual collation of this information, as
noted above, helps enormously in this effort.
Evidence (much of which will be presented in raw form in
appendices to the self study) of department or program productivity (or lack
thereof) must be embedded in a coherent argument about the state of the
program. Some institutions will provide a template for this document. Most self
studies describe the history of the department or program as well as its
current goals and dynamics. Honest disclosure of pressures and responses help
to contextualize decisions and approaches. The self study should aim to make a
persuasive case for current or future strategies.
It is essential for all self-study documents to be focused
and concise. The narrative itself should be no longer than 15-20 pages
(excluding appendices). In those 15-20 pages, the self study should
information and data pertinent to the department or program and its operation,
including but limited to enrollment data, results from an alumni survey, or
institutional data like senior surveys or graduate school admission results.
those data to assess the department’s or program’s effectiveness in furthering
its goals and objectives and the mission of the college, to identify strengths
and weaknesses, and to highlight key issues it wishes to raise during the
review. This document is a chance to identify and define problems or challenges
so they aren’t defined or identified for you. It is also a chance to propose
evidence of faculty achievement and engagement.
a preliminary plan for the subsequent five to ten years.
The following elements are often included in descriptive and
- Brief history of the program
- Defining characteristics of the program
(subject areas; missions; in context of campus; beyond campus)
- Overview of curriculum, including
fieldwork and internship and other “high impact” opportunities.
- Discussions of data about enrollments,
degrees awarded, and such
- Assessment system and results
over time, including exit experiences and capstone
- Faculty (number, expertise,
- Relations with other departments or
- Funding and fundraising
- Resources (space, equipment, library
- Graduate program, if any, description
and outline of requirements and support
Appendices often contain the following elements:
- Overview of campus (size,
organizational structure, denominational affiliation, if any)
- Catalog program description
- Program mission statement
- Advising handbooks, if any
- Degree requirements for any degrees
- Descriptions of core courses, including
syllabi when possible (generally over the last 3 years)
A summary of course enrollment patterns as well as numbers of
majors and minors over the past decade
A summary of learning and career outcomes over the last five to
- List of faculty, with ranks and titles
and year joining faculty
CVs of all faculty members who regularly teach in the department
- Official description of governance
- Data over 5- or 10-year period (e.g.
majors, degrees awarded, student/faculty ratio, particularly in light of
college or institutional data)
- Program budget (not always required by
- Other documents that provide context
for program goals and achievements
This is an on-campus review that precedes the external
review. Most institutions constitute an internal review committee comprised of
faculty with some familiarity (or none, depending) with the program being
reviewed. This committee can provide invaluable insight into resources and
priorities on a particular campus. Bear in mind that internal reviewers might
need more orientation to a particular department or program than one might
The external review is often a pivotal part of the program
review process. The nomination process for constituting a committee is specific
to institutions, with some involving the department or program more closely in
the process and with others working with more “top down” approach. When the
department or program has a say, it is best to suggest reviewers in similar
departments or programs who are active in the guild and who, ideally, have some
administrative experience. Departments and programs are often permitted to
suggest reviewers from which administrators will choose. The Executive Office
of the AAR can help match reviewers with programs (see Request an External Reviewer on the AAR Academic Relations Committee website), and inquiries of peer departments or programs
on comparable campuses can yield good results.
Most external review committees consist of two or three
faculty from other institutions who possess pertinent experience and who, in
the view of the department or program under review, can offer objective, useful
advice from a disciplinary perspective. External reviewers usually visit campus
to meet with administrators, undergraduate majors, graduate students (if any),
faculty, staff, and any other groups that the department, program, or upper
administration chooses. This visit should take place over a day or two, with
ample time for the review committee to work together while on campus to draft
A typical schedule for an external review might look
something like this:
External Review Committee usually meets with the department or program members
over dinner on the evening before the first full day. Although this is usually
a more informal social gathering, this dinner should help orient the External
Committee and review the agenda for the following day(s).
campuses will arrange for the External Review Committee to meet with
upper-level administrators (president or chancellor, dean(s), and chairs) as
the initial on-campus meeting.
External Review Committee needs considerable time (one hour per faculty member
if possible) to meet individually with members of the department or program and
with faculty and staff in other departments or programs, as appropriate.
Members of the External Review Committee should also meet with students
(usually in a group).
should be made available for the committee to meet by itself to discuss the
review as it progresses. Several hours on the last half day of the visit should
be set aside for the committee to discuss and draft its report.
External Committee sometimes meets with the entire faculty in the department or
program under review to discuss its findings and outline the general
conclusions it will present in its written report.
committee often meets with upper-level administrators for an exit interview.
There are a few best practices in preparing and treating
external reviewers. Reviewers should have at least two weeks to read the self
study and any other relevant documents (e.g. institutional strategic plan,
department or program strategic plan). They should always be given time to work
together while on campus to process issues. This is particularly important
prior to any exit interview with administrators. It is imperative that faculty
in the department being reviewed be honest about any problems and proposed
solutions. It is also important to focus on the most critical issues and to
remember to highlight strengths without being too modest. If, for instance, the
department or program contributes significantly to the general education
program, give outside reviewers a chance to meet with people who can speak to
Above all, remember that outside reviewers are often the
source of diplomatic and effective solutions that will get the attention of
administration. Be open to constructive suggestions, both during the visit and
in the final report.
The external review committee will be expected to provide a
final report on a timeline and often following a template provided by upper
administration. This report, which is often around 7-10 pages, provides an
outside perspective on the self study documents as well as the campus
experience. The department or program should use this document to help advocate
for any needed change.
Response to external review accompanied by plan going forward
Much of the work of the program review is done after the
external reviewers have sent their reports, but that doesn’t mean that
departments or programs need to wait for the official report to start
responding to it. Think in terms of making changes regarding issues raised by
the review and of producing a tactical document that shows the department or
program sees itself clearly and has a plan for fixing any issues, either
internally or by asking for resources to do so. If the department or program
disagrees with anything in the external review report, it should be able to
explain its reasoning. The response document should include strategies and
timetables for improvements and should make an argument to administration about
why the program is worth funding.
Institutions differ widely on
how much or how formal any administrative response might be. This document
cannot begin to anticipate what administrative officials might do with any
self-study or what any department or program’s response should be.