Cooper, Heron & Heward (2007, p. 149) say if you do visual analysis, you must look at three things.
Trends of data
The level of data
The extent in type of variability and data
And here is a very nice quote from Lane & Gast (2014, p. 445): "You need to look at variability or stability, you need to look at level, and you need to look at trend."
Do I have to convince anybody that if you are doing visual analysis, that variability and level and trend are important? You probably agree with me. This is just how we do our business.
Trend Take a look at the first condition in the graph below. We have an ascending trend, then we have another ascending trend. How many would be convinced that the intervention was good? That the progress happened as a result of the intervention itself? Would you be suspicious?
You should be! The problem is the behavior is already going up. So if something's going up and you say, "I did this thing and now it's going up," who are you convincing? This is done completely with visual analysis. You don't need statistics for this. You can look at the data and see. This is our science.
Now look at the second condition in the graph. If it's flat and then it goes up, that's convincing. And the third graph? If we have something that's going down and our intervention makes it go up, we can convince people of our effectiveness. This is part of trend analysis.
Level Let's just say we have the average in baseline, which is a 2. Then you do something, now the average is a 10. Do we have an effect? Yes! You went from 2 to 10. You're doing this through visual analysis.
Variability Imagine that you have data points that are bouncing around and those two lines below are showing you what's called the variability envelope.
On the left, although it went up, the variability envelope is the same. Has variability changed? No. On the right, it has. So here's my question to you:
Where does visual analysis occur?
Laboratory settings. Maybe you do work in labs and do visual analysis there. Maybe you are doing applied work where, again, visual analysis is frequent. You did a thesis, you did a dissertation? You probably did visual analysis.
Now, here is a critical question:
What does visual analysis rely upon?
Think about this for one minute. All that stuff about trend, level, variability; if you have a compromised visual graphic, how good is your analysis? It will likely also be negatively affected. I can't tell you how important it is that we have a very good graph when we're making decisions. All of those things that I just shared with you; how we move our science forward, how we make applied decisions, how we affect lives. Everything is reliant upon a graph. So would you believe no one has ever done a study, not once, ever, to see how well we actually make graphs? No one's ever done it. I don't know why. But I decided that's something I needed to do. More on that later...
If you're ahead of the curve and want to get started, Chartlytics enables you to create the most effective graphs for monitoring and changing behavior. Find out how it works here.
Want more on graphing & standardization?
Watch a FABA 2017 panel discussion with Julie Vargas, Eb Blakely, Patrick McGreevy, and Rick Kubina.
We believe the "learner knows best," and it is our privileged responsibility to adapt educational materials to our learners' needs. We do so by precisely measuring behavior, judging the outcomes of the performance intervention on a standard chart, and make changes if the data tell us an adaptation is in order.