Reading and Understanding AT Research

Assistive technology research runs the gamut from small action research projects in classrooms to formal experimental research conducted by skilled researchers. To understand research it is important to understand the terminology. Following are definitions of some of the key words that are used.

Definitions of research terms

ABABA design

When there is only one subject, researchers try to establish a causative relationship by collecting data on a student’s performance both with and without the use of the tool being studied. The data collected before the AT is provided is called baseline data or condition A (e.g. the child is successful in their attempts to communicate 3 out of every 10 attempts with no AT). Then they introduce the use of the tool and take data about any changes in performance that occur. This is condition B (e.g. given a spelling board and pointing to the first letter in the word he is saying, results in successful communication 7 out of 10 tries.)  Taking away the board for a short time or collecting data in a situation where it is not available (return to condition A) helps to show that it is the use of the board that is making the difference.  Repeating conditions A and B help to rule out other things that might have caused the improvement such as time of day, more interesting activity, or accidently taking data at a time of day when the student’s articulation is always more understandable, etc.

Case Control Studies

Case control studies in education are similar to case control methodology often used in cancer research. Think about a huge database full of information. Start with all the records and match them on areas of relevant background factors, dropping the records where no match occurs. Match on things like age, disability, behaviors, technology, achievement, etc. Match these records on every variable possible, dropping more and more records in the process, until the only variables in the database left unmatched are the ones you are interested in researching. For this example, let’s say the use of graphing vs. non-graphing calculators in Algebra. Perhaps, after all this matching and dropping of records is done, there are as many as 20 cases left. These cases are matched on all the variables in the database, with some number (let’s say 9) using graphing calculators and the others (the remaining 11) taking Algebra with regular calculators. The difference in performance on some criterion measure—like a state assessment test score—should be a direct indicator of the “graphing calculator effect” for these 20 cases.

The attraction of such research is clear. First, it involves no expensive additional testing, data collection, treatment or experimental designs. Second, it can be done quickly and efficiently because the data already exists in an electronic database and is easily accessible. Third, it can be done over and over again with different matches and different variables at essentially no additional costs and can be done year after year such that true longitudinal research can be conducted. In the past, databases may not have been in place to support such intense data manipulation. With improvements in technology, it is possible to see integrated data systems becoming more and more sophisticated and much larger.

Case Series with no Control group

A case series with no control group is similar to a case control study in that it is carried out by reviewing existing data.  However there is no attempt to identify a control group. It is simply a compilation of numbers of cases where two things occurred together. An example is the 1998 study in which a link was suggested between mumps-measles-rubella (MMR) vaccination and autism. This was based on an uncontrolled case series of 12 children referred to a pediatric gastroenterology unit with a history of normal development followed by loss of acquired skills, including language, together with diarrhea and abdominal pain. It was suggested that the gastrointestinal and developmental symptoms were a syndrome that could have been triggered by MMR vaccination. The conclusion was based simply on finding these cases where children both had the two things occur. Subsequent reviews have concluded that they could not prove or refute the suggested associations between MMR vaccine and autism. The problem with using a case series with no control group is that a causative relationship can be the conclusion that is reached, when that may not be true at all. It could be coincidence or it could be the result of some other factor that you have failed to identify.

Control Group

The group of participants in an experiment who do not receive the treatment (e.g. use of an AT tool) that is being studied.

Controlled Trial

The strategy used in scientific research to regulate the effects of variables in a study that are not intended to influence the results or conclusions. For example if a researcher is studying the impact of using a portable word processor versus a computer, the students participating in the study would get to use the tools for a set of amount of time for the same number of days, at the same time of day and for the same assignments to ensure that amount of time, time of day, day of the week or type of assignment were not the cause of any differences found in the completed assignments produced by the students.

Experimental Research

A type of research that has the goal of determining whether something causes an effect.

Experimental (true) Research Design

A research design in which (1) an independent variable (e.g. use of a spell checker) is directly manipulated to measure its effect on a dependent variable (e.g. number of words spelled correctly), and (2) participants are randomly assigned to different groups that receive different amounts of the independent variable. This is sometimes referred to as randomized field trials or randomized controlled trials. For example if a researcher wanted to study the effect of using software that reads text on the comprehension of text book passages, he might randomly assign 30 students with reading difficulties to participate in one of two programs: (1) standard teaching of reading or (2) eight weeks of using scan and read software (such as Kurzweil 3000, Read and Write Gold, WYNN, etc.) in addition to the standard reading teaching. After the end of the eight weeks, the researcher compares the student’s scores on a standard reading assessment. The addition of the scan and read software is the independent variable, and performance on the test is the dependent variable. Group 1 is the control group. Group 2 is the treatment group because they participate in the special treatment (independent variable).

Multiple Baseline

Introducing the intervention in specific activities or settings at different times is a way to help prove that it is the use of the new AT tool that is causing the change in performance. For example a student may begin to use a portable spell checker in Language Arts class in October.  His spelling improves on Language Arts assignments. It is then also introduced in Social Studies in December.  At that point his spelling improves in Social Studies and continues at the improved level in Language Arts. In January the staff arranges for him to use it in all classes and similar improvement results.  Because of the staggered introduction of the use of the AT, the change in his spelling ability is shown to be caused by the use of the portable spell checker rather than maturation, change in difficulty of assignments, use of a new teaching strategy or some other undocumented event.

Random Assignment

Students are randomly assigned to participate in one of two different situations. For example a researcher conducts a study of two different programs for teaching math skills to students with learning disabilities. The researcher controls for differences among students by randomly assigning them to one of the two groups (one group will get program A and one group will get program B). The researcher controls for differences among teachers by having a single person teach both classes.

When students are not randomly assigned, sometimes a whole class in one school gets one treatment (e.g. text to speech software to support reading) while a whole class in another school gets a different treatment (e.g. standard reading instruction with no text to speech). To compensate for lack of random assignment, researchers often try to match students on such factors as age, disability, reading level, etc., but it is not as effective in controlling for undocumented events (e.g. differences in the teacher’s skills, differences in motivation at one school, etc.) as random assignment.