The two standard criteria for assessing the quality of an evaluation are reliability and validity. Reliability refers to the consistency of a measurement process. Would an instrument yield similar results if it were administered by different assessors, or on different occasions? Is the measure internally consistent? Are results from different subsets of a test’s items correlated? In a study involving over 1,000 Kindergarten children the EYE-TA 2.0 has proven to be highly reliable, with reliability coefficients for internal consistency as follows:
Awareness of Self and Environment: 0.92
Social Skills and Approaches to Learning: 0.92
Cognitive Skills: 0.92
Language and Communication: 0.90
Physical Development: Fine Motor: 0.82
Physical Development: Gross Motor: 0.84
The second criterion – that the measure is valid – requires one to gather evidence on the fit between the items of the instrument and the underlying theoretical constructs, such as cognitive development or language development. This is called construct validity. In practice researchers assess whether scores on the instrument are correlated with other well-established instruments. For example, in a validity study conducted with francophone children in New Brunswick, the EYE-TA cognitive development subtest correlated 0.55 with the French version of the Peabody Picture Vocabulary Test (PPVT) and 0.65 with the reading subtest of the Wechsler Individual Achievement Test (WIAT), while the language subtest of the EYE-TA correlated 0.67 with the PPVT and 0.45 with the WIAT (Laurie, 2010).
Recent thinking about validity also requires one to consider the kinds of decisions made from the assessment; a test is valid only if inferences made from the results are appropriate, meaningful, and useful. This calls for consideration of how results are interpreted and used in the school context. Has the use of EYE-TA increased opportunities for children to learn? Have school districts expanded services to parents and children? How have assessment results been used by parents? How are results used within schools? The EYE-TA is being used in some school districts to identify children that may require further assessment or are potential candidates for extra support at school. The data are also used by speech and language therapists and other consultants to offer strategies to classroom teachers that support all children in the classroom. These strategies go beyond emergent literacy skills; they include school and home strategies for fine motor, gross motor, behavioral, communication, and social development. David Hursin, formerly the coordinator of Kindergarten Inclusive Developmental Services, noted, “The EYE gives you the whole picture of the makeup of the kids in your class. Knowing their strengths and weaknesses early in the year helps you modify programming for the entire class so everyone can be successful.”
The EYE can also be used as a population measure to track changes in children’s skill development from year to year or to assess the efficacy of large-scale interventions. For example, the EYE-TA has been used to discern whether children improve their skills at a faster pace if they are in a full-day Kindergarten rather than a half-day Kindergarten.