Include a Multidisciplinary Design Team

From the creation of assessment specifications through item development, assessment assembly, administration, and scoring, a multidisciplinary panel of experts should be engaged to evaluate the design and materials (inclusive of supporting documents, such as assessment blueprints and score reports) and advise on accessibility for the target population of test takers. This advisory panel should include accessibility, EL, and special education researchers and practitioners as well as additional experts (e.g., content-area specialists, educators of young learners) based on the construct(s) measured and the grade level(s) of the assessment. Working closely and collaboratively with item writers (i.e., educators, contractors, or other qualified professionals who author items) and assessment developers (i.e., qualified professionals who create assessments), the expert panel will help to ensure that the construct of the assessment, task types, items, and response options are designed to be maximally accessible to all test takers to the extent possible. The expert panel will recommend revisions to the design and items to enhance accessibility, and, where item adaptations are required, will help to ensure the adapted items parallel the original items to preserve construct and rigor.

Consider Appropriateness, Efficacy, and Opportunity to Learn in Accommodation Assignment

In some cases, the process of assigning accessibility resources to individual students is a challenging practice given the reality that accommodations are not uniformly appropriate and thus may pose a threat to the validity of the interpretation of test scores. In the context of next generation computer-based assessments, there may be some interactions with students’ opportunity to learn assessment devices or digital versions of their accommodations (Guzman-Orth, Laitusis et al., 2016; Guzman-Orth, Supalo, & King, 2019). Additionally, changing standards and policies can have a lag-time for implementation in practice, evidenced in the recent shift in standard braille codes used in the U.S. Traditionally, English Braille, American Edition (EBAE) was the norm and used in instruction and assessment. Since 2015, the standard braille code changed from EBAE to UEB. This shift resulted in some students experiencing new difficulties taking assessments due to the change in braille standards and subsequent testing regulations (Guzman-Orth, Supalo, & King, 2019). As a result, the act of assigning a braille accommodation to a student who reads braille may have face validity, but the accommodation assignment lacks efficacy because the implementation may not be in the manner familiar to the student.

Determine Whether Domain Exemptions Will Be Needed

Bom-accessible assessments are the goal, especially since accommodations may not provide enough support for ELSWDs. When the assessment is not sufficiently accessible and accommodations do not provide enough support, domain exemptions, or se/ecfii’e participation in ELP assessments (i.e., where a student may take some assessment sections but not others; see Christensen et al., 2014) may also be needed for some ELSWDs. For example, states who receive federal funding for ELP assessments may grant domain-level exemption^) for which appropriate accommodations are not available to the student following ESSA requirements. One example of this is when EL students who are deaf/hard of hearing are exempted from the listening domain. This exemption is common when the construct definition is more traditional by constraining listening to an auditory act, in contrast with a more universally designed, inclusive definition that recognizes the practical act of students who are deaf/hard of hearing listening with their eyes (i.e., with the assistance of signed communication or printed text). In some cases (e.g., where federal or state regulations do not apply), assessment developers, along with the multidisciplinary advisory panel and other stakeholders, must determine whether selective participation should be allowed or those domains should be provided as a special version and/or following an alternative administration approach. Further questions to consider include Should the listening domain of an ELP assessment be administered to students who are deaf or hard of hearing? How would listening be defined for this subgroup of test takers? What task type, item level, and /or administration adaptations might be necessary?

Develop Guidance for Which Students May Need an Alternate Assessment

In some instances, designing an accessible assessment and pairing numerous accessibility features with that assessment will not be sufficient. Some of these students are ELSCDs, who "... may participate in a state’s Alternate English language proficiency assessment if they cannot participate in the regular ELP assessment with accommodations” (U.S. Department ofEducation, 2018, p. 6).

Alternate ELP assessments are new to the field. Although one is currently in operational use (WIDA’s Alternate ACCESS), limited empirical evidence exists about alternate ELP assessments or even background characteristics for those ELSCDs who qualify to participate in their state’s alternate content assessments (Guzman-Orth, Cavalie, et al., 2016). In light of the federal mandates to ensure that ELSCDs are included in standards-based accountability assessments, the field is now coming together in response to these requirements. Advances in defining who these students are (Christensen, Gholson, & Shyyan, 2018; Thurlow, Christensen, & Shyyan, 2016), measuring background characteristics (Shyyan, Gholson, Mitchell, & Cevian, 2018), designing standards (Michaels, Gholson, & Scheinker, 2018), and conceptualizing theories of action for alternate ELP systems (Gholson & Guzman-Orth, 2019) are emerging as states strive to design these alternate ELP assessments based on the latest information addressing the needs of ELSCDs.

Attend to Accessibility at Every Stage of the Assessment Design and Development Process

Accessibility should be considered during each stage of development, from the initial design work to the administration of the items. Assessments designed for maximum accessibility from the onset can “reduce the need for individualized accommodations, thereby reducing potential threats to validity across the testtaker population” (Elliott, Beddow, Kurz, & Kettler, 2011, p. 10). Key considerations for an iterative process of accessibility reviews during the design phase could include the following:

• • Involve disability experts at the onset of the design work to articulate an inclusive target construct and comprehensive accessibility solutions, minimizing likelihood for ELSWDs to be unintentionally excluded and assessments needing retrofitting to allow access.
• • Intentionally design tasks in concert with administrative considerations and allowable accommodations to promote maximum accessibility for the target population without compromising the construct, technical quality, or test taker engagement.
• • Invite groups of disability experts and stakeholders for review and feedback of the assessment materials (e.g., teachers of the visually impaired; teachers of the deaf; teachers of students with multiple disabilities such as visually impaired and deaf/hard of hearing).
• • Conduct systematic reviews for accessibility and usability with students, such as piloting and field-testing activities targeting all disability groups, accommodations users, and test examiners who administer these accommodated assessments.
• • Continue collecting updated accessibility and usability evidence on the iterative changes made and revise assessment specifications as needed; repeat these steps for all content, and expand findings to accessible ancillary materials (directions for administration, practice tests, training tests, etc.) to ensure that students with disabilities have opportunities to access accommodated content and ancillary materials that are equitable to those of their non-disabled peers.

These key considerations for an iterative approach to enhancing the validity and efficacy of an accessible ELP assessment are imperative. Further, inclusion of diverse groups of stakeholders during the duration of the assessment development process can ultimately minimize the likelihood that content will introduce accessibility challenges for students during the operational assessment administration. Examples of publicly available instruments that enable the systematic review of items for accessibility are the Test Accessibility and Modification Inventory (TAMI; Beddow, Kettler, & Elliott, 2008) and the TAMI Accessibility Rating Matrix (ARM; Beddow, Elliott, &

Kettler, 2009). Documenting the attention and effort toward improving accessibility supports claims about technical quality and appropriateness for ELSWDs, an area under much scrutiny and evaluation by federal requirements.

While universal design is the gold standard, it is possible that even when adhering to these considerations, content may not be fully accessible to each test taker. Task type and item adaptations or special versions may still be needed. Our previous example of EL students with visual impairments taking graphics-heavy ELP assessments is one such instance where, despite all the attention to accessibility in the initial design phases, additional equitable versions may be needed to ensure meaningful presentation of the content for each student group (Guzman-Orth, Supalo, Pooler, & Sova, 2019).