In other words, a teacher may work with a performer who generates a number of data points with multiple practice trials. Take the example of see-write answers basic multiplication facts products 0-81 (practice sheet shown below).
Notice the Time column. The data show efficiency. The performer does a 30 second trial, receives feedback, records the trial on Chartlytics, and then starts the next trial. All in all, the practice for the day took only 6 minutes. Sidebar: While 4 minutes might seem like a lot of time the performer made 135 correct response with only 2 incorrects in one day! Imagine doing that for a week - an extrapolation shows the performer made 675 correct responses in only 20 minutes! Compare that to a student who does no systematic practice and perhaps homework - that performer might have 100 or correct responses per week in 30 or more minutes if he or she is lucky.
OK, back to worksheet interface. Notice the icon for the the counting time, in that column you see 00:00:30, 30 seconds for each practice trial. Then the next two columns display the counts of correct and incorrect answers respectively.
We can see our performer has 6 data points from 6 different practice trials. Which trial should we place on the chart?
The difference between practice and assessment
The answer to the question we previously asked lies in the difference between practice and assessment.
Practice, or as we call it in Precision Teaching frequency building, refers to performing (repeating) a behavior in time and then providing feedback after the trial ends. The goal of practice directly involves ameliorating performance. And if we chart those daily performances across time we have a celeration line which let’s evaluate learning. Teachers use practice to help improve daily performance and more importantly learning.
Assessment deals with “the process of using tests and other measures of student performance and behavior to make education decisions” (Venn, 2000). With assessment we have a goal of understanding the effects of a method, procedure, or variable on performance and learning.
With our performer above, we have a problem if we treat all 6 trials as practice. Namely, we allow the performer to do each trial and then provide performance feedback afterwards. What clearly did we miss? An assessment trial.
Without an assessment trial a dilemma now exists; out of the 6 practice trials which one represents the performance?
Problem solved - add an assessment trial
To solve the problem, we must add an assessment trial. For our six trials, if we have 1 assessment trial (no feedback, just a timed 30 second trial) and then 5 practice trials, we clearly have an dependent and independent variable. The independent variable (the variable manipulated by the teacher) and the dependent variable (the measured behavior) now exist.
Independent variable or IV (practice trials) - 5 timed trials of practice or frequency building (includes feedback; goal = improving performance).
Dependent variable or DV (assessment trial) - 1 timed trial (does not include feedback; goal = assessing performance).
Now the question becomes, which assessment trial would best represent the effects of the independent variable? And which trial you select for the DV also alters the practice trial intervention (IV).
Chartlytics gives you other options to understand your data set. See the figure below.
Option 1 - First. We could chart the first trial (we call that the assessment trial, it would also serve as a our DV). The first trial of the day has the advantage of showing how the performer does without the benefit of previous practice (the performer hasn’t practiced since the last day).
Option 2 - Median. Displaying the median data point would capture the middlemost performance. Charting the median would tell the chart reader how does the performer change most typically. Median performance has advantages over the arithmetical mean (the average of all performances - see below).
Choosing option 2 means we no longer have a true assessment trial (DV). In other words, a teacher may have done 6 practice trials and uses one of those practices to judge the effects of the intervention. In essence, a practice trial represent all 6 practice trials. Definitely an option but the median has the disadvantage of not possessing a true assessment trial.
Option 3 - Geometric Mean. The geometric mean would show the average of the 6 performances. The typical value of the practice would lead the chart reader to judge the effects of all trials. But like the median, the geometric mean lacks a designated assessment trial.
Sidebar: While the arithmetical mean and the geometric mean both display the average, the geometric mean better represents data for a few reasons (I will spend a future blogpost explaining the beauty of the geometric mean because you want to know!).
The arithmetical mean works well when scores occur independent of one another. However, the geometric mean functions best with values that effect one another (not independent of each other). One timed trial and feedback will effect the next one. Therefore, the geometric mean represents the average performance better than the arithmetical mean.
Option 4 - Minimum. The minimum refers to the lowest score among the data set. For the values in figure 2, 21 correct and 1 incorrect would appear on the chart. The minimum value would conservatively represent progress. Using the minimum value means not having a predetermined assessment trial.
Option 5 -Maximum. The maximum refers to the highest score among the data set. For the values in figure 2, 24 correct and 0 incorrect would appear on the chart. The maximum value would liberally represent progress. Maximum data points charted also do not have a pre-established assessment trial.
Option 6 - User Selected. The user chooses any practice trial as representative of the others.
Option 7 - Stacked. Stacked data points results in displaying all corrects and incorrects on one line. Some charters like seeing all data visually dispersed. The advantage of stacked data points comes in having access to each trial the performer did. A disadvantage lies in determining the celeration line.
Which chart option should you use?
We chart data to observe, analyze, interpret, and communicate changes in behavior. Depending on your situation, different options will serve you and the performer well. In a research setting, clearly an assessment trial (dependent variable) and set number of practice trials (independent variable) must exist prior to beginning to chart. Order and tight experimental protocols must take place in research to uncover secrets of nature.
In clinical practice or teaching settings, however, different options may appeal to the educational team. The teacher should weigh the advantages and disadvantages to determine which data point best achieves the objective of understanding what the data have to say.
Venn J. (2000). Assessing Students with Special Needs. London: Prentice Hall International.