ATLAST’s approach to measuring student opportunity to learn was situated in a theory of teaching for understanding. Consistent with this theory, we adopted a variation of the learning cycle as the basis for instrument development. The cycle includes the following phases of instruction:
- Situating the learning;
- Students expressing their initial ideas;
- Students examining relevant phenomena;
- Students making sense of phenomena; and
- Students making sense of the targeted idea(s).
To construct a framework for performing analyses of enacted curricula, we identified phenomena that support students’ development of the targeted content for each sub-idea in the three content areas. The content areas differ in the extent to which associated phenomena are directly observable. To accommodate these differences, we distinguished between evidentiary phenomena and primary phenomena.
Evidentiary phenomena are observable, naturally occurring events that provide evidence for the primary phenomena (e.g., bubbles appear on the leaves of a plant submerged in water (observable), providing evidence that plants produce oxygen during photosynthesis (not observable)).
Initial efforts to construct a tool for rating opportunity to learn using instructional logs resulted in a multi-faceted matrix. This approach was intended to take into account both what content was included in instruction and how students were engaged with the content. Although the matrix captured several important aspects of instruction, reliability among raters was unacceptably low. In addition, rating all of the instructional logs for a unit required a great deal of rater time.
Consequently, we developed an alternative approach that requires less inference (and less time) by raters. A portion of a resulting enacted curriculum rating sheet shows the relative simplicity of this approach. Although we forfeited detail about opportunity to learn that we thought would be predictive of student learning, inter-rater reliability with the new approach was quite good.