Ideas on Documentation and Accountability in Assistive Technology

This document shares ideas on documentation and accountability in AT.   AT teams need to focus more on documenting the work of the team. Information about what your team does is data that your team can use to document AT benefits for students and staff.   Accountability is what teams do to show their level of responsibility for the work they do. Both provide essential information that you can share with administrators.

Multiple individuals have contributed to this list of ideas about documentation and accountability to enhance AT services and supports. If you have an idea that you would like to share, please contact the NATE Network at info@natenetwork.org .

 

Documenting AT Services

Data Focusing on the Individual Student vs. the AT Team  

Portfolio methods can provide great information and data about individual students. But I also want to emphasize that we need data about what the TEAM does (or what the AT specialist does if there is only one AT service provider in the district). One of the problems with collecting data about the use of AT is that it is harder to demonstrate a direct relationship between AT and function. If you change a child’s AAC system and suddenly they are communicating in more complex ways, was it because of the system, because the child was developmentally ready, or because the student’s medication got changed? But information about what your team does is data that administrators and your team can use to document  AT benefits for students and staff, as well as document the needs of the AT team. Is there data that you can report about AT services ( e.g., number of devices loaned, number of classroom trainings, number of school-wide trainings, number of parent meetings)? I really think both kinds of data are needed. So as you look at student portfolios, think too about your AT services “portfolio”. Documenting student success is the role of the school team and the purpose of the IEP. AT specialists can contribute to this body of information. AT teams, however, need to focus more on documenting AT Team effectiveness. (Contributed by Gayl Bowser)

Building Data Collection into Routine AT Team Procedures

There are multiple ways that AT teams can build data collection into routine team procedures.  Here are four examples:

• • Request for Support

HIAT’s Request for Support Survey Questions:  Because some AT services got their start in medical centers, they have often continued to work from a “medical model” which begin with a referral. “Referrals” for AT services often use a questionnaire that focuses on a detailed description of the student.  This information is sometimes a case of information overreach and does not necessarily provide clarity on what the school team needs to know in order to consider AT for a student in the context of the classroom. The HIAT team of Montgomery County Public Schools, Maryland, as part of an initiative to build capacity shifted from a medical referral to a survey form that requests a minimum of information on the student and focuses more on the needs of the school team.  (Note: detailed information on the student is typically available in the district’s online student database.) This “request for support” communicates the important role of the school team in collaboration with the AT team or specialist. The survey taps into the school team’s ability to consider AT and its knowledge of AT tools available in the district. This survey generates important data on the reasons for AT service requests and tool usage which adds to an end of year report.  It helps the AT team differentiate the appropriate starting point for AT support. It also provides essential information such as the student identification number, school contact information, grade, etc. It can be automatically linked to a spreadsheet or database. For example, Google forms can link to a Google spreadsheet, likewise using Microsoft Office 365. Columns can be added to the spreadsheet to track AT services for students. Larger districts may want to invest in a relational database such as Filemaker Pro which is especially useful if there is an extensive AT equipment loan system.

1. 2. Tracking AT Services

Tracking AT services begins with good note taking using mobile note taking tools, (e.g., One Note or Evernote.)  But one way to measure AT services is to identify the nature of the services provided across the school year (e.g., Consideration support, Trial period support, AT tools consultation, assessment, AT training, equipment support, implementation plan support.)  An alternative to this is to track the intensity of AT services by measuring the number of hours of service provided (e.g., 1-5 hours, 6-10 hours, etc.) This data is entered into a spreadsheet or database and is readily available for an end of year report.

This spreadsheet documents the amount of time allocated by total hours in a school year.  (https://docs.google.com/spreadsheets/d/1J202)ZBO0RUnNpEk7NGU67d2RxtIewfknD/edit#gid=2021520540)

This spreadsheet documents the type of services provided by the AT team (https://docs.google.com/spreadsheets/d/1sB8QR3-bkCc72MCRbft93Ov4R6yX4qBv/edit#gid=537406842)

1. 3. Trial Period Data

The data collection described in the previous two examples is still only surface level information.  Collecting data on the effectiveness of AT strategies and AT Team support is a more powerful reflection of the value of AT services.  Getting school teams to collect AT usage data is typically hard to do and data that clearly isolates the effectiveness of AT on student performance is even harder to obtain. At minimum, AT teams should collect the results of AT trials that typically involve four to six weeks of AT use.  When AT teams provide trial period support, there should be an upfront, communicated expectation that a description of effectiveness must be sent back to the AT team. Periodic email reminders can reinforce this agreement. Trial period feedback leads to data on the outcomes of the use of AT tools and strategies.  A simple rubric can be applied to the description of the trial period results: a) the student is developing the use of a new tool or strategy; b) there was no improvement in student performance; or c) the student did not demonstrate a need for the targeted AT. This information is added to the database or spreadsheet.

https://docs.google.com/document/d/115J3C9uxHpStnxWCUMNOughry7o-3MRO6kXMj5a03V4/edit

1. 4. AT Services Exit Survey

Information on the effectiveness of AT tools and AT team support can be obtained through the use of a short end-of-year survey that can be completed in less than 5 minutes.  The survey is sent to the contact person associated with each request for AT support. One question should address perceptions about student outcomes; the second question should address perceptions on staff outcomes.  This is a critical question that reflects capacity building. The third question asks what more the AT team could have done to support the implementation of AT. To keep the survey brief, logic branching is used to expand responses.  The exit survey form can automatically generate charts that can be included in an end-of-year report for questions one and two.

Sample Exit Survey Link:  https://forms.gle/9GUV25E9ogkUDFQPA

These four examples of data collection can be built into routine AT team procedures and can be included in an end-of-year report in very little time.  An end-of-year report not only documents the work of AT service providers, but also serves as a history of the work of the team. AT teams, no matter how small, should demonstrate accountability.  In addition to outcome data, this report can include information on professional development, resource and website development, and outreach efforts. It should also include goals and objectives for the following year and a review of current goals and objectives.  The end-of-the-year report serves as a planning tool and an accountability report. There’s always room for improvement, but you need data to help you see the patterns. You need to take the time to look at these patterns.

Click here for an example of the table of contents for an End-of- Year Report Table of Contents.  

(Contributed by Denise DeCoste, MA)

 

Using Surveys to Collect AT Data

When you ask people to provide data, there needs to be a clear reason for taking that data.

This is particularly true for surveys.  For the most part, staff hate long surveys and don’t feel they have the time to complete them. For this reason, you don’t always get a representative sample. I also feel we should not expect staff to fill in all the data that we’ve neglected to collect for lack of an ongoing process.  If we ask staff to complete a survey, it should be brief and the reason/benefit behind the survey should be clear to the person asked to complete the survey.

Sometimes, instead of collecting historical data, consider finding ways to incorporate data collection into routine procedures going forward. For example, incorporate it into your online referral/request for support. Or add a question or two to the end of a professional development training evaluation.

When you use a tool such as Survey Monkey or Google Forms, it is important to word questions in a way that will automatically tally data. Use radio buttons, multiple choice, likert scales instead of narrative responses whenever possible. Make sure the question truly reflects the information you need. (Contributed by Denise DeCoste)

 

Storytelling vs. Data Collection  

Graphs and charts can lead to great discussions about the AT Team’s allocations of time and resources.  However, videos, pictures, and student success stories also make an impression. The graphs and charts are important in that they keep your team accountable, provide a way for you to reflect on trends, and set objectives for your team.  But I agree that success stories have a lasting impact. Storytelling is important. Student success stories in the context of the classroom reflect the importance of building AT capacity within schools. Nonetheless, while pictures capture hearts (and that’s a good thing), data speaks to the important role of your team.  It justifies the district’s ongoing expenditure to support AT, and the data demonstrates that AT service delivery is serious work. Success stories and data work well together. (Contributed by Denise DeCoste)

 

A Data Story  

I’m always impressed at how even a little bit of quantitative data can influence a program. Here’s a story about it.

I was working with an AT team in a medium sized district a couple years ago and they kept saying that they needed more FTE positions on their team. They told me that their administrator was not willing to entertain the idea and that he just didn’t understand what they do. This particular team had 4 people but they were all .2 FTE in their AT work and AT kept getting buried due to other work demands. The team did keep records of all the students they served, the supports that they provided to those students, the AT that the students were using, and a lot of other things they were doing. But when they talked with the program supervisor, they more often spoke about the impact that AT had for individual learners e.g.,  “Amanda is doing so much better in written composition”, “Juan is using 2- word phrases.” The AT team realized that they were only talking about individual student stories and were not describing the multifaceted work of the team.

To remedy this, they identified all the kinds of data about their own work that they could compile and collect and even built in one additional data point by developing an electronic form about the impact of trial periods on student performance.  In their first-ever end-of-year report, they were able to show all the ways that they were spending their AT allocated time, and the trial period data showed that they had actually saved the district over $17,000 in one year because trial periods identified places where equipment that the district owned could be used, eliminating the need to purchase new devices.  Their administrator looked at the report data with a new respect for the team. Instead of a flat “No” in response to the request for additional FTE, he said , “I’ll keep it in mind that you need a more consistently available person. Ten months later, he came back to the team and invited the SLP to work full time as the AT specialist. Data not only helped the administrator understand what the team was doing, but also the reason that a full time person would be valuable.  (Contributed by Gayl Bowser)

 

Data Initiatives

 Moving forward, I am going to use an end of the year report with data reflecting on all the services I provided, how many students require AT, the hours spent with each student and the types of services provided. This is in my 3 year plan and I have already worked with a tech guy to help build a database system for me free through google forms and sheets! It is going to be so nice to have this data to show the hard work I have done throughout the year. I also believe talking to the administration about adding an AT related question during an interview can be a great asset to the district! I love the idea of evaluating a program to help implement AT in the classrooms. It would be nice it the evaluation process was updated and added questions on how teacher utilize AT within their classrooms.(Contributed by Riley Pec, IL)

Accountability

End of Year Reporting 

When I started doing end of year reports, I thought I was merely documenting the history of the team. I really didn’t think anyone but our direct supervisor would care. But what I found was that the “process” of preparing and pulling together an end-of-year report clarified and honed our vision. Revisiting what you do really helps you make plans going forward– what worked and what didn’t work. We even had a more humorous “lessons learned” page where we admitted what we got wrong and what we learned from that! Again, all this crystallizes your intent as a team. Are you true to your mission, are you making progress toward your goals?

What I also learned is that by sharing our data, as well as our yearly objectives, training and outreach outcomes, was that it gave our supervisor more insight into what we did, and she in turn, began to look at ways she could support our goals. She started asking for copies of the report to share with her next in command. I doubt admin higher up the chain even read them, but they sure knew we took our AT roles seriously. (Contributed by Denise DeCoste)

 

Data and Accountability

Most professionals understand the importance of data collection. One solid reason for data collection is to give the team the ability to reflect on their allocation of services.  Are you spending so much time in 1 to 1 training that you need to develop webinars, or more on-demand resources, or should you be creating a professional learning community? Or are you finding that you are consulting on the same things often, such that a brief video quick guide would be useful.  It’s also an opportunity to reconsider equity of services. Are you spending a lot of time with the most needy students and not able to get to staff serving students with milder, but valid disabilities. Should you be spending a large proportion of time adapting materials (at your rate of pay) or should you be seeking to hire an aide to help you with this?  Accountability data is not just about making school staff accountable for the use of AT in classrooms. It’s also about making AT team members accountable for how they provide services. There’s always room for improvement, and you need data to help you see the patterns. You need to take the time to look at these patterns. (Contributed by Denise DeCoste)

 

Why End of Year Reports are Good for Small or Large AT  Teams 

Even if you are an AT team of one or a new AT team, it is important to  establish some data collection strategies. As a new team, you don’t have to sift through the past! If you are a team of one, the end of the year report often serves as rationale for more staff. That is not its primary purpose, but it leads to discussions with your supervisors about future needs. The end-of-the year report will grow with you. My first report was less than 20 pages with large fonts and pictures of very basic line graphs and pie charts. Many years later, as we got better at collecting ongoing data and documenting our work, it ended up being more than 80 pages.  No one questioned our commitment to AT based on our End of the Year Report! (Contributed by Denise DeCoste)