Written by: Kristin Barbour
Springtime is my favorite season: new growth, warmer east coast weather, and more daylight. But I also enjoy springtime because it marks the season when we measure and reflect on our students’ progress in educational therapy! Now is the time of year to begin annual progress monitoring by conducting annual testing. Kim LeFevre, Wendy Bokmiller, and several NILD instructors often field annual testing questions that come to NILD headquarters. We thought it would be beneficial to share others’ questions and our responses.
1. When should annual testing be administered and what should be included?
Springtime, between late March and early May, is the typical time NILD educational therapists administer annual testing with students who received NILD educational therapy for the entire school year. However, school-based and private practice educational therapists accept students throughout the year. The rolling admission makes the determination for when to conduct annual testing a little less clear. In general, if the student can complete 44-60+ sessions by the end of the school year, administer annual testing after the student has had at least 44 sessions. For students unable to complete the minimum of 44-60+ sessions, please wait until the following spring to complete testing.
Annual testing includes formal and informal measures. The Woodcock-Johnson-IV Tests of Achievement (WJ-IV ACH) is an important formal measure we use to monitor students’ progress. Administer subtests 1-11, any Extended and Oral Language battery subtests given at initial testing. A reading inventory such as the Ekwall/Shanker Reading Inventory is also administered. Informal measures include Bender, Dictation and Copy Draw a Person, Writing Sample, and Draw A Clock. Have students write Name/Alphabet/Numbers/Days/Months only if there are challenges in these areas. Also, only check Rhythmic Writing problem areas using the Rhythmic Writing Progress Monitoring Form (See Appendix in the Level I manual for the form).Annual testing is only done once a year at the end of a school year.
As a reminder, initial testing is not done annually, only once when the student begins NILD Educational Therapy. We rely on the annual testing data for comparison each year. So, stated another way, do not test students at the beginning and end of each school year.
2. Does the WJ-IV have to be given yearly? What if I have not been trained to administer it?
The WJ-IV ACH is administered annually to measure students’ progress in basic academic skills, fluency, and application in reading, writing, and mathematics. The WJ-IV ACH should only be administered by an NILD educational Students annually take the WJ-IV ACH. If you have yet to be trained to administer the test, work with your program coordinator or another PCET to arrange testing. Don’t hesitate to contact the NILD office if you need assistance finding an educational therapist to administer the WJ-IV ACH. We have PCETs available to assist you with gathering the important WJ-IV ACH data.
3. Why does NILD require that I submit my annual testing results?
Helping students become competent and confident independent learners is essential and should be one of our primary goals as educational therapists. We want to make sure that NILD educational therapy is a means to bring transformational changes in students’ thinking and learning. Building and maintaining NILD program credibility is essential as we share the program’s distinctives with administrators, educators, and parents.
NILD program credibility is built on student success stories reflected in both qualitative and quantitative data. Each year, you personally collect qualitative data reflected in the success stories about how your students overcome learning challenges and successfully manage learning opportunities. These student success stories shared with other educational therapists, classroom teachers, students, and parents encourage everyone and testify to NILD educational therapy’s effectiveness! Another vital way to measure student progress and response to intervention is through formal and informal assessments that provide quantitative data (e.g., numbers, percentages, and change over time).
The quantitative data we capture through initial and annual testing focuses on how well we are improving four targeted areas:
- Cognitive and Perceptual Processes
- Language Processes
- Literacy/Writing/Mathematic Skills
- Non-Cognitive Factors
Based on our theory of change, we have expected outcomes for each of the targeted areas. For example, we anticipate that program completion after three to five years will demonstrate increased cognitive, perceptual, and language efficiency as measured by the WISC-V full-scale IQ and index scores. Additionally, we expect students will demonstrate annual increases in literacy and math performance as evidenced by reading, written expression, and math achievement outcomes measured by the WJ-IV ACH. Submitting your annual student testing scores enables NILD to measure student success outcomes, highlight program efficacy to internal and external stakeholders, and evaluate any necessary changes to PCET certification training. The annual test reporting link will be sent in an email to you in the next couple of weeks.
4. Where can I find the guidelines for annual testing?
Your most comprehensive resource for annual testing guidelines, which we refer to as “Annual Progress Monitoring,” is in your NILD Level I 2022 manual beginning on page II-68. In addition, the Level I and Level II Manuals also contain various testing forms that allow you to present student results visually.
5. What if my student made minimal gains?
The timing and amount of improvement in students’ thinking and learning are different as students respond uniquely to NILD educational therapy. Don’t be concerned if you see some academic areas respond quicker than others to the intervention. It is common for students to have an uneven pattern of improvement across academic skills. Over twenty years ago, I asked my mentor, Ken Scott, how to interpret student scores that were relatively unchanged within a year. Ken helped me to understand that typically, students with learning disabilities who do not receive effective interventions demonstrate a regression in standard scores over the years. Students demonstrating maintenance of the same standard scores within a year actually indicate progress as age and grade norms increase annually. However, we must take the opportunity to carefully analyze our student’s annual test scores and evaluate if we are being as prescriptive as possible in applying the NILD techniques to our student’s learning needs. We should review the NILD technique goals and objectives to ensure that we emphasize aspects of each technique that most specifically address our student’s areas of weakness.
6. Can I readminister a portion of testing if I could tell that my student was having an “off” day?
Both formal and informal assessments have specific guidelines for administration. Test administration follows a standard format specific to each test. We invalidate the test results if we reassess a student with the same portion of the testing on another day because it seems the student is having a better day. However, it is acceptable to remind the student to apply the strategies they have been using in NILD educational therapy before administering testing.
When we see our students struggling with testing, it is helpful to remember that formal tests have been tested for reliability. A reliable assessment is replicable, meaning it will produce consistent student performance scores over time. A useful test metric to use is the confidence interval. The confidence interval is a hypothetical range of scores predicted if the student were given the WJ-IV ACH 100 times. It gives us a range of scores around the student’s obtained score that is sure to include the “true score” with a 68, 90, or 95% likelihood (e.g., the image below uses a confidence interval of 68% to indicate the range of scores). So, if you believe the student was not having a good day when you administered a portion of the WJ-IV ACH, the confidence interval can help you interpret the scores as it provides the range of scores you expect your student to fall between if you redo your test.
7. How often should the WISC V be administered?
The design of the WISC V is to capture students’ current cognitive abilities. Therefore, the test guidelines indicate that students may only take the WISC V once in 12 months. Additionally, WISC results older than two years are no longer considered valid. The student will need to retake the WISC to have a valid representation of their current cognitive abilities. On average, students receive NILD Educational Therapy for three years. Please communicate with your families that WISC V exit testing provides essential information to make a data-informed recommendation about the student’s readiness to complete NILD educational therapy. Additionally, without exit WISC testing and reporting to NILD, we miss critical quantitative data that informs program efficacy and credibility.
We know that sometimes students are not ready to graduate from NILD educational therapy in three years. For example, one of my students received educational therapy for five years. In these instances, use your best judgment in terms of WISC re-testing. In the example above, the student took the WISC after three years of educational therapy and was reassessed again two years later. Multiple WISC scores provide excellent insight into the longitudinal changes students experience through NILD educational therapy over time.
8. What should a parent meeting look like?
Progress monitoring meetings with parents reviewing annual testing outcomes are opportunities to strengthen the collaborative partnership between you, the student, and their parents. Please use the Progress Monitoring forms found in the NILD Level I manual to assist you in reporting test results and observations about students’ performance in therapy sessions:
- Formal test summary sheet - records WJ-IV ACH scores
- Progress Monitoring Graph - emphasizes areas of academic improvement
- Informal Testing Record Sheet - highlights observations of informal tests
Please don’t get stuck experiencing writer’s block; check the sample annual progress reports included in the NILD Level I manual! When you present the annual report, rather than reading the entire progress report during the meeting, highlight observations from educational therapy of progress and continued recommendations. Remember to use anecdotal records as a reference for observations and examples. Invite parents to share their perceptions about student progress and continued challenges. Finally, determine mutually agreed-upon goals and recommendations for the following school year.
Thank you for being so diligent with this critical component of NILD educational therapy. Let’s have our strongest year yet with progress monitoring report submission.
Kristin Barbour, Ed.D., NILD PCET