It is as easy as 1-2-3. 

1. Choose rubrics to study – this may be chosen by the department head, a focus group, or a survey of the faculty.  It is recommended that everyone’s voice be heard during this process.
2. Provide links or copies of PDF files of the chosen rubric(s) – the files are located on the TfP website. It is important for the faculty to fill out (pen/paper or electronic pen) the rating and justification section for each criterion in preparation of the department discussion for consensus.
3. If the department wants to have the individual data readily available for analysis, copy the Google Form for the appropriate rubric to a common location for the department.  Each faculty member should fill in their responses on the Google Form. 

The rubrics are intended to stimulate good discussions on various topics. Departments may choose areas they know need improvement, areas they really do not have a sense where they stand, or perhaps areas that an accreditation agency may have recommended the department examine deeper.

This will vary from institution to institution, but here are a few things to consider:

• To get the department engaged and provide a start to the conversation about each criterion chosen, a paper/poster could be posted for each section (A,B,C,D) of a rubric.  The poster would have each criterion listed with the scale.  Faculty would provide their ratings by placing sticky notes (which allows space for comments) on the poster.  This could be placed in a staff lunch room along with a stack of sticky notes. By putting this in a public place, it will remind the faculty of the importance of this endeavor and also provide a physical place (similar to ‘water cooler’ discussions) where faculty may start having discussions about the criteria.  As an alternative, Google Jamboard could be used in a digital version of this process.
• It is imperative to communicate that all faculty are respected as professional instructors. Hopefully that will be enough for the faculty to want to be involved. However, especially when the meeting/session is scheduled to have the discussion about the consensus scores, pay for time may be necessary.  For some departments, having food available can be the added incentive needed. It may be valuable to do a virtual meeting to accommodate varying schedules but also to provide a way to record the discussions so that an accurate documentation of the conversations is available.

Some faculty may feel that they are not knowledgeable enough about a given criterion to provide a rating. It is important for them to score their best ‘guess’. The fact that this may happen is in itself an indication of something that the department should discuss.

Once the individual scores have been established, the department should have a meeting (or meetings) dedicated to making a consensus decision on each criterion.  The process will vary according to a department’s culture, but here are some things to consider:

• If the department has collected the individual data electronically, the ‘average’ rating could be presented and then individuals can volunteer explaining why their score was either above or below that rating. 
• If the department used the poster method described above,
• In the case where scores are very similar, it may be useful to use the average rating as a starting point for the consensus rating;  
• In the case of varying scores, ask for justifications from those with low scores and those with high scores and use the justifications to begin a discussion that leads to consensus
• For some departments it may be necessary to establish a formal method for running the meeting such as Robert’s Rules of Order.  
• In the case of larger departments it may be judicious to have smaller groups work together on specific criteria and then the groups report to the department; the department then discusses each recommendation and accepts the consensus or works together to arrive at a consensus.
• When providing the justification it may be best to have someone take notes, write up the justification, and then provide it to the department for final approval at a later date.

Why does the department need a consensus rating and not just an average rating?

Average ratings can be skewed, especially in small departments. In addition, the rubrics are intended to initiate discussions which may result in individuals changing their interpretation and rating of a criterion.

The department needs to determine a consensus (not an average) rating for each criterion as well as a justification for each rating. The importance of this being a consensus of the WHOLE department cannot be emphasized enough. All faculty need to be involved in the process and not just a sample to make this a productive effort. Much of the benefit of using these rubrics will come simply from getting the department together to have open and engaging discussions about how the department ‘operates’ and what they like and what they would like to change. This process does not work if it is not the entire department involved in these discussions.

Because we are currently in the Beta testing stage, the TfP project would appreciate information about the department’s methods for getting participation, how the consensus scores were determined, and how the department plans to use the information to move forward.

The consensus scores and justifications should be entered using the Teaching for PROWESS Vision and Transformation Catalyst Tool Google form for the appropriate rubric.

1) Student Learning and the Learning Environment

2) Instruction

3) Curriculum and Program Development

4) Assessment of Student Learning

5) Diversity, Equity, and Inclusion

6) Professionalism

7) Climate for Change

8) Snapshot

Additional feedback (as mentioned in the previous question) should be sent to the leadership team at tfpleadership@amatyc.org.

In the near future the TfP project, in conjunction with AMATYC, plans to create a Vision and Transformation Certification process that includes providing a Vision & Transformation Consultant to assist a department as it works through the change process. The consultant will be well trained in research-based methods for transformational change efforts.

Until the Certification process is put in place, the department will define their priorities with respect to 1) areas they want to change, 2) methods for improvement, 3) deadlines for various stages, and 4) an assessment plan to provide feedback as part of an iterative process for improvement. Since the Vision and Transformation Certification process is a work in progress as a part of the TfP project, formal consultants are currently not available to help in the process. The project, however, does have several resources that can be shared with the department in a less formal manner, perhaps in conjunction with a meeting with one or more of the personnel involved in the project.