Recommendations for Successful Involvement of Educators in Practice

We hope that this chapter has provided a number of useful examples of how including classroom educators at a range of key steps in the assessment development process can improve the quality of a K-12 ELP assessment program. We now provide some general and practical principles for the successful involvement of educators, based on our experience on ELPAC and other K-12 ELP assessment programs.

The involvement of educators requires significant planning from the onset of the assessment design effort, considering factors such as the following:

• • The inclusion of educators must be intentionally planned from the early stages of a new assessment or new assessment cycle (and must be budgeted for). If the program is to be developed based on an RFP, the roles of educators should be specified to allow the testing vendor to budget appropriately and plan appropriate timelines for integration of their input.
• • A recruiting effort must be planned and implemented considering thoughtful criteria for selecting educators to achieve appropriate representation of educators with appropriate qualifications and representing the diversity of the testing population (e.g., from a range of geographical areas, backgrounds, and teaching contexts).
• • The testing vendor must devote appropriate resources to develop suitable training materials, to facilitate meetings with educators effectively, and to update test materials to reflect educators’ input. Perhaps most prominent are the assignment of assessment developers with broad skills in the principles of test design and development and the ability to work effectively to train educators and facilitate their involvement through panel reviews and other means, ensuring that the input from educators is implemented in a thoughtful and consistent manner.
• • Individuals representing each party involved in the assessment development process (the DOE, the testing vendor, and the educators) must enter into the work with a conscious attitude of professional collaboration. That is, they must all realize that they have valuable experience and opinions to impart; that the others have different but similarly valuable experiences and opinions; and that the test will be more valid if it reflects a considered and balanced blending of input from all who are participating.
• • Appropriate steps must be taken to ensure security of test materials at all stages of development.

Finally, while educators’ involvement in ongoing annual activities such as item-writing workshops and item-review panels are of significant value, it is worth noting that involving educators in test-design activities allows them to have a somewhat deeper and more lasting impact on the assessment program. Thoughtful advance planning to support the involvement of educators at key stages of assessment design and development can help to ensure that these efforts result in a more robust validity argument for the assessment and meaningful professional development for the educators themselves.

Note

1 See the ELPAC Practice Tests www.elpac.org/resources/practicetests/for representative samples of the ELPAC task types as they appeared when the test was launched operationally.

References

Baron, P., Linquanti, R., & Huang, M. (2020), Establishing an English-proficient performance standard for English language proficiency assessment uses. In M. K. Wolf (Ed.), Assessing English language proficiency in U.S. K—12 schools. New York. NY: Routledge.

California Department of Education. (2014). California English language development standards: Kindergarten through grade 12. Sacramento, CA: Author. Retrieved from www. cde.ca.gov/sp/el/er/documents/ eldstndspublicationl 4.pdf

Educational Testing Service. (2016). Item writing guidelines for the ELPAC. Unpublished document.

Educational Testing Service. (2017). Summative assessment test blueprints for the English language proficiency assessments for California. Sacramento, CA: California Department of Education. Retrieved from www.cdc.ca.gov/ta/tg/ep/documents/elpacadoptedblue prints.pdf

Fulcher, G. (2012). Assessment literacy for the language classroom. Language Assessment Quarterly, 9(2), 113-132.

Hauck, M. C., Wolf, M. K., & Mislevy, R. (2016). Creating a next-generation system ofK-12 English learner (EL) language proficiency assessments (ETS research report no. RR-16—06). Princeton, NJ: Educational Testing Service.

Herrera, L., & Macias, D. (2015). A call for language assessment literacy in the education and development of teachers of English as a foreign language. Colombian Applied Linguistics Journal, 17(2), 302-312.

Inbar-Lourie, O. (2013). Guest editorial to the special issue on language assessment literacy. Language Testing, 30(3), 301-307.

Malone, M. E. (2013). The essentials of assessment literacy: Contrasts between testers and users. Language Testing, 30(3), 329-344.

Popham, W. J. (2009). Assessment literacy for teachers: Faddish or fundamental? Theory into Practice, 48(1), 4-11.

Stiggins, R. J. (1995). Assessment literacy for the 21st century. Phi Delta Kappan, 77(3), 238-246.

Stiggins, R. J. (1999). Evaluating classroom assessment training in teacher education programs. Educational Measurement: Issues and Practices, 18(1), 23-27.

Taylor. L. (2013). Communicating the theory, practice and principles of language testing to test stakeholders: Some reflections. Language Testing, 30(3), 402-412.