Validity Testing

Analyses and report by:

Jordan Vossen, MS (Psychology PhD student); Ekaterina Burduli, PhD; Celestina Barbosa-Leiker, PhD; Washington State University, Spokane, WA. September 12, 2017


Data was used from a large project where clinicians were assessed using the MICA tool over a two-year period. Because the original dataset included multiple sessions for each clinician we anticipated that the correlation between sessions within the same clinician to be highly related, thus leading to inflated correlations and reliability. Therefore, we created two data sets. The first one consisted of the original data that includes all sessions (including repeated sessions with the same clinician; N = 2274) and second one included a sample dataset that reflected only one session for each clinician (N = 1178). Overall, results show only minor differences in the correlations between the two datasets. The results below are for the analyses on the full original dataset.

Results Summary:

Correlations between the 7 global scores (5 MI Intentions items and 2 Strategic Response items) revealed that these items are highly related. Correlations between these scores are statistically significant, range from 0.66 to 0.86, and are all in the positive direction. This suggests that, for example, as a clinician becomes more proficient in responding to Change Talk, they also demonstrate more proficiency in intentions to express empathy. Overall, these findings suggest that high scores on one of the 7 global items is related to high scores on each of the other 7 global items.

The correlations between the 7 global scores and the number of questions the clinician asks are small, ranging from -0.11 to -0.004, only 4 out of the 7 items are statistically significant, and none of them are substantial. Overall, this suggests that the number of questions a clinician asks likely unrelated to the 7 global measures. The correlations between the 7 global scores and the number of reflections a clinician makes range from 0.30 to 0.57 and are all statistically significant. This suggests that higher numbers of reflections are related to greater proficiency in the 5 Intention items and the 2 Strategic Responding items. For example, increases in the number of reflections is related to increases in proficiency in responding to Sustain Talk or intentions to support autonomy and activation.

Reliability analyses between the items revealed that the items have good internal consistency. The alpha for the 7 items altogether, the 5 intention items, and the 2 Strategic Response items ranged from .85 to .96, suggesting good internal consistency for the scales.