Which observation protocol should I use to observe teaching?

posted April 10, 2021 and revised July 30, 2023
by Cassandra A. Paul, Adrian Madsen, Sarah B. McKagan, and Eleanor C. Sayre

This recommendation initially appeared as an article in the American Journal of Physics: A. Madsen, S. B. McKagan, E. C. Sayre, and C. A. Paul, Resource Letter RBAI-2: Research-based assessment instruments: Beyond physics topics. Am. J. Phys, 87, 350 (2019). 

Faculty in physics departments often observe each other’s teaching and give each other feedback to improve teaching. Using an observation protocol for these informal observations can help faculty articulate the goals of these observations and focus on particular aspects of the classroom. Observation protocols can provide data that illustrate what happened in the class, which can be useful for self-reflection and professional development. You can use observation protocols once as a stand-alone activity or to track your own improvement.

Classroom observations using observational protocols can be conducted using segmented, continuous, and holistic procedures (Velasco et al. 2016). Segmented protocols are those in which the class period is broken up into shorter periods of time, 2-min intervals for example, and then observers note whether they see certain behaviors during that interval or not. At the end of the observation, observers note the number of intervals in which each of the different behaviors happened. Continuous protocols allow observers to indicate what is happening at any given moment in a class, and an observation results in a timeline indicating what happened when. This also allows the different classroom activities to be considered as a certain percentage of overall class-time. Holistic protocols are protocols in which the entire course is considered at once. This is done using a series of questions that the observer responds to at the end of an observation.

There are seven observation protocols that we will discuss here. Four of these protocols focus on recording what is happening in the classroom. These are the Classroom Observation Protocol Undergraduate STEM (COPUS) (Smith et al. 2013), the Teaching Dimensions Observation Protocol (TDOP) (Hora 2015, Hora et al. 2013) the Real-time Instructor Observation Protocol (RIOT) (Paul and West 2018, West et al. 2013), and the Student Participation Observation Tool (SPOT) (Paul et al. 2018). One protocol, the Laboratory Observation Protocol for Undergraduate STEM (LOPUS) (Velasco et al. 2016), focuses on recording what is happening in laboratory courses. One protocol, the Reformed Teaching Observation Protocol (RTOP) (MacIsaac and Falconer 2002, Sawada et al. 2002) focuses specifically on assessing the degree of reformed teaching. Finally, one protocol, the Behavioral Engagement Related to Instruction (BERI) (Lane and Harris 2015), looks at the level of student engagement in a class session. All of these observation protocols can be used in high school or college-level courses.

Title Focus Intended Population Format Research Validation Purpose 
Reformed Teaching Observation Protocol (RTOP)

Degree of reformed teaching

All levels

Holistic

 

Gold

To assess the degree to which reformed teaching is occurring in classrooms.

Teaching Dimensions Observation Protocol (TDOP)

Instructor and student classroom behaviors

All levels

Segmented, available online

Gold

To classify instructor and student behaviors in STEM classrooms.

Classroom Observation Protocol Undergraduate STEM (COPUS)

Instructor and student classroom behaviors

All levels

Segmented, available online

Gold

To classify instructor and student behaviors in STEM classrooms.

Real-time Instructor Observing Tool (RIOT)

Instructor-student classroom interactions

All levels

Continuous, available online

Silver

To classify instructor and student behaviors in STEM classrooms.

Student Participation Observation Tool (SPOT)

Student behavior and participation

All levels

Continuous, available online

Bronze

To allow and observer to classify instructor and student actions in the classroom, with an emphasis on types of student participation.

Laboratory Observation Protocol for Undergraduate STEM (LOPUS)

Instructor and students’ lab behavior

All levels

Segmented, available online

Silver

To classify instructor and student behaviors in STEM labs.

Behavioral Engagement Related to Instruction (BERI)

Student engagement

All levels

Segmented

Bronze

To quantitatively measure student engagement in large university classes.

Reformed Teaching Observation Protocol (RTOP)

Perhaps the most well-known observation protocol is the Reformed Teaching Observation Protocol (RTOP) (MacIsaac and Falconer 2002, Sawada et al. 2002), a holistic paper and pencil observation protocol developed to evaluate the extent to which a classroom uses reform-based teaching techniques, meaning “constructivist, inquiry-based methods” (MacIsaac and Falconer 2002). The RTOP developers operationally define “reformed teaching” as “classroom practices that result in a high RTOP score” (MacIsaac and Falconer 2002). The RTOP consists of 25 Likert-scale items from three different categories including, “lesson design and technique,” “content,” and “classroom culture.” Observers watch a class session and respond to each item with a maximum of 4 meaning that the item is “very descriptive” of the class to a minimum of zero indicating that the item “never occurred.” The RTOP data can be reduced to a single score by adding up the scores for each item. A higher RTOP score means that a class is more reformed, meaning that the course is more active and student-centered. The single RTOP score makes it particularly useful as quantitative evidence of instructor change in practice over time. The RTOP has been used very widely, and RTOP scores have been shown to correlate with conceptual learning gains in college physics courses (Falconer et al. 2001). There are several questions on the RTOP that evaluate the instructor on the content or design of the lesson, and so, it is more appropriate to use the RTOP with instructors that designed the lesson themselves (and not a teaching assistant who did not have autonomy in deciding what happens in the classroom). The RTOP developers emphasize that RTOP results are not valid unless the observers have gone through several days of training on how to use the instrument. While in-person training is best, there is online training available. The items on the RTOP were developed based on previous research and existing instruments (Sawada et al. 2002).

Teaching Dimensions Observation Protocol (TDOP)

The Teaching Dimensions Observation Protocol (TDOP) (Hora 2015, Hora et al. 2013) is a segmented observation protocol that aims to record what is happening in the classroom, unlike the RTOP, which is designed to evaluate the degree of reformed teaching. The TDOP looks at three basic dimensions of the classroom including “instructional practices,” “student-teacher dialogue,” and “instructional technology,” and three optional dimensions, including, “potential student cognitive engagement,” “pedagogical strategies,” and “students’ time on task.” Each of these dimensions has codes associated with them, and observers memorize the meaning of these codes (28 basic and 11 optional) and circle that code when it happens during each 2-min interval of an observation. Observers can collect data with pencil and paper or with a computerized interface available on the TDOP website. Once data are collected, observers can examine the percentage of intervals that each code (or code category) appears. The TDOP website also automatically creates some charts and graphs for review. TDOP creators recommend that users establish inter-rater reliability and stress that training may take several days depending on how many dimensions are used. Both a TDOP users guide and TDOP scoring sheet are available for download. The codes and categories on the TDOP were developed based on an instrument designed to study inquiry-based middle school sciences courses (Osthoff et al. 2009).

Classroom Observation Protocol for Undergraduate STEM (COPUS)

The Classroom Observation Protocol Undergraduate STEM (COPUS) (Smith et al. 2013) was developed based on iterative modifications of an early version of the TDOP, so it is also a segmented protocol and is similar in many ways. The COPUS developers aimed to create a more user-friendly version of the TDOP (though the version of the TDOP they were working with had more mandatory categories, and the newer version of the TDOP discussed in this paper has been simplified). COPUS codes are separated into two broad categories, “what teachers are doing” and “what students are doing,” with a total of 25 codes using a simplified language. This allows individuals to learn to use COPUS much more quickly than the TDOP or RTOP, in as few as 1.5 hours. The COPUS developers also added some categories that were aligned with best practices in large-enrollment college-level STEM courses, such as discussions motivated by clicker questions. Like the TDOP, observers indicate whether a certain behavior happened or not in each 2-min period using a specialized scoring sheet. The COPUS developers have also recently developed the COPUS profiles online tool that allows a user to upload COPUS data in a spreadsheet in order to create several different visual representations of these data that can be helpful for reflection.

Real-time Instructor Observation Protocol (RIOT)

The Real-time Instructor Observation Protocol (RIOT) (Paul and West 2018, West et al. 2013) was developed independently from COPUS at the same time, and therefore, the two were developed to fill similar needs but with slightly different focuses. RIOT, which is similar to COPUS and TDOP, allows an observer to categorize what is happening during a classroom observation. Unlike the COPUS and TDOP, the RIOT is a continuous web-based protocol that only follows the instructor and records what they are doing (including if they are interacting with students) but does not record what students are doing independently of the instructor. The categories for RIOT are organized by the types of interactions that are possible with students in the classroom, “talking at students,” “talking with students,” “observing students,” and “not interacting with students.” An observer clicks on icons representing these categories in order to indicate that a certain interaction is occurring as they are observing a classroom and continuously clicks new observations as they are observed. The web program records timestamps for each observation. The RIOT was originally developed as a part of a Teaching Assistant (TA) pedagogy course to help new graduate student TAs understand how to interact with students in an active learning environment, so it is useful for helping faculty as well as teaching assistants understand and improve their teaching. Like COPUS, RIOT requires little training to use. The RIOT categories were developed based on observations of classrooms using the Collaborative Learning through Active Sense-making in Physics (CLASP) curriculum (Potter et al. 2012) at University of California at Davis and emergent behaviors seen there.

Student Participation Observation Protocol (SPOT) 

The Student Participation Observation Tool (SPOT) (Paul et al. 2018) is an observation protocol very similar to the RIOT in that it is web-based and continuous and has the same developers, but there are a few key differences in the content and layout. SPOT had a more rigorous development process than RIOT, as categories are backed by research on student-centered learning in science classrooms. SPOT categories represent the observable features of seventeen of the best practices in active learning (Paul et al. 2018). Different from RIOT, but similar to the COPUS, the SPOT records what both the instructor and students are doing (whereas RIOT focuses on the instructor) and is organized by class “mode” referring to how the instructor and students are interacting with each other at any given time. The class can be in “small-group mode,” where students are working in small groups, “whole class mode,” where students are watching a lecture, movie, or demo, and “independent mode” where students are working silently and independently (such as when they are taking an exam). In each mode, different codes are available to describe different behaviors of instructors and students. SPOT is optimized for courses that include some traditional lecture elements in order to better classify how participation happens, and who is participating. For example, during a lecture where the instructor may interact by asking or answering questions, SPOT allows an observer to classify student responses as either shouted-out, asking a question, answering a question, contributing an idea, or via whole-class choral response. SPOT also allows the observer to keep track of individual students using a map interface based on where they are sitting in the room. This can help instructors determine if many students are participating, or if it is the same five or six each time. Since SPOT is web-based like RIOT, it also generates colorful figures useful for self-reflection. To see examples of these figures, see the PhysPort assessment pages for the RIOT and SPOT.

Laboratory Observation Protocol for Undergraduate STEM (LOPUS)

The Laboratory Observation Protocol for Undergraduate STEM (LOPUS) (Velasco et al. 2016) was developed to categorize student and teacher actions in laboratory settings. The LOPUS creators started their development with a draft of the COPUS, then reviewed the literature and watched video of laboratory classes, to determine new behaviors that should be added to the LOPUS, which were not included in the COPUS. Like the COPUS, the LOPUS is a segmented protocol and organized into two broad categories of instructor behaviors and student behaviors, but LOPUS also has a third category that captures the content of student and teacher verbal interactions in laboratory classes, and who (teacher or student) initiated the interaction. For example, someone viewing an instructor lecturing about data analysis would use the pair of codes: “Lec” (indicating that the instructor is lecturing) and a qualifying code from this third category, “Ana” (indicating that the conversation is about data analysis and calculations). The LOPUS team also cut some of the codes from the COPUS that they found were highly correlated with each other, in order to cut back on the number of codes an observer needed to memorize. The LOPUS is available in a web-based format through the General Observation and Reflection Platform (GORP). The platform auto-creates charts and plots that are useful for reflection.

Behavioral Engagement Related to Instruction (BERI)

The Behavioral Engagement Related to Instruction (BERI) protocol (Lane and Harris 2015) is a segmented observation protocol to measure student behavioral engagement, defined as on-task behavior, in large university classes. The BERI can help an instructor figure out which parts of their class resulted in higher student engagement. The BERI protocol outlines six engaged behaviors, for example, listening, writing, and engaged instructor interactions, and six unengaged behaviors, for example, settling in/packing up, being off-task, or disengaged computer use. The observer chooses a group of ten students and sits near them. During the class, the observer cycles through each of the 10 students and records, on a printout of instructor notes, if each student was engaged or disengaged during part of the class. The BERI observation protocol categories were developed based on observations of large classes to determine which student behaviors could be defined as engaged and disengaged.

The BERI protocol focuses particularly on student engagement, whereas the other protocols discussed above have a more general focus. The COPUS, LOPUS, TDOP, and SPOT all record student behaviors during the class, but they do not label these behaviors as engaged or disengaged.

Several of the observation protocols mentioned here have been incorporated into web-based tools to make them easier to use. See our expert recommendation "What observation protocols are available online?" for more information on accessing these online protocol tools.

Recommendations for choosing an observation protocol

While all the observation protocols discussed here are potentially useful for self-reflection and professional development, we particularly recommend the COPUS and RIOT for these purposes based on their short training times, and resources for self-training. Use the COPUS for your professional development and self-reflection if you are particularly interested in what specific pedagogical tools are used in the classroom (e.g., students making a prediction, instructor showing a demonstration). Use RIOT if you are more concerned with what the instructor is doing more generally, and their interactions with students (e.g., the instructor is explaining content, the instructor is listening to a question). If you are interested in evaluating how reformed a course is, especially if you want to apply a numeric score to this evaluation to compare to national results, and you can attend a training, use the RTOP. If you want a detailed account of what pedagogical actions take place in a classroom and have time for training, use the TDOP. Use SPOT if you have questions about the frequency, type, and diversity of student participation in the classroom. Use the LOPUS if you are interested in lab environments and use the BERI if you are particularly interested in how your students’ level of engagement in class depends on what you are doing in class.