1、,Measurement Issues,Xue Yanan,Measurement Issues,Another set of potential problems centers on the measures that are used: tested, questionnaires, observations, interview procedures, or any other means of quantifying language behavior. Since the result of a study are only as good as the data on which
2、 they are based, it is essential to make sure that the measures themselves are not introducing extraneous variables into the study. Three problems that the researcher should particularly watch for are the practice effect, the reactivity effect, and the instability of the measures or the results.,Pra
3、ctice effect,Practice effect ：essentially involves the potential influence of the measures on each other. Its strongest form occurs when the same test is given repeatedly in a study to determine if there are changes in performance.,For instance, a study might be concluded to determine whether studen
4、ts are making gains in grammar during three weeks of intensive Spanish study. By giving the same grammar test before and after such learning, the researcher might attempt to determine how much the students improved during this period.,The problem is that students often learn from their mistakes. So,
5、 formally or in formally, they may have cleared up any problems they had with the first test at the time of the second test, especially if subject expectancy was involved. Thus, any apparent gains in the scores on the second test may reflect the students actual growth in general proficiency in gramm
6、ar or their learning the items on the particular test in question.,If the practice effect is not controlled, it will obviously be impressible to separate these two variable. Thus, the results will be ambiguous and difficult to interpret.,Reactivity effect,In some language studies, a reactivity effec
7、t may occur when the measures themselves are causing a change in the subjects.,One example of this effect might be an attitude questionnaire, in which the subjects actually form or solidity attitudes that they did not have before filling out the questionnaire.,In this instance, the questionnaire bec
8、omes the catalyst for the very attitudes that are being studied. This effect would obviously influence, to an unpredictable degree, the results and the interpretation of the results.,Instability of measures and results,The instability of a measure of any kind refers to the degree to which the result
9、s on the measure are consistent. Would those result be the same if the measure was administered repeatedly while the effect of testing was controlled? Likewise, the instability of the result of a study refers to the degree to which the results would be likely to recur if the study was replicated abs
10、olutely.,Although it is not possible to control both types of instability absolutely, statistical procedures provide the researcher with tools to determine the degree to which measures are stable, or consistent, and to estimate the probability that the results of a study are stable, or will recur in replication.,Thank you！,
