Equipe Raisonnement Induction Statistique
|
1. Behaviour of statistical inference users |
2. Probabilistic representations in purely random situations |
3. Representations of randomness |
1. Behaviour of statistical inference usersEmpirical study of cognitive models used in inductive inference situations, by experimental researchers and statisticians. This study involves:
throughing from particular (for instance a data sample) to general (here reference population);
probabilistic judgments essentially based on statistical data.
It was possible to discern some general attitudes vis-à-vis the statistical analysis of experimental data which were completely independent of the researchers' specialization
A need for objectivity"In problem of scientific inference we would usually, were it possible, like the data to 'speak by themselves'." (G.E.P. Box & G.C. Tiao)
The majority attitude appears to consist of expecting the statistical analysis to express, in an objective way, "what the data have to say" independently of any outside information (notably a priori hypotheses, references to theories, etc.).
The dictatorship of a significant test result"Researchers and journal editors as a whole tend to (over) rely on 'significant differences' as the definition of meaningful research." (J.R. Craig, C.L. Eison & L.P. Metze)
Nowadays researchers' strategies appear to be "dictated" by a significant test result:
it is often the only criterion used to draw a conclusion about a study.
The arguments advanced in this case are generally circumstantial: the significance test is undeniably one
of the social criteria which have to be used for the results to be accepted by the scientific community;
the entire academic world supports the application of significance tests.
Misinterpretations of significance tests and adaptative biases"It is very bad practice to summarise an important investigation solely by a value of P." (D.R. Cox)
More than real "errors", the various misinterpretations of significance tests have to be seen as adaptative biases
of the normative references, designed to make them fit one's true needs
In this perspective, a systematic analysis of these biases enlightens the nature of the questions naturally
asked by researchers, and the ingredients which they privileged (descriptive results, sample sizes,
significance test's outcomes, social criteria, etc.).
This analysis allow to reveals some internal coherence in statistical users' judgments.
Even statisticians...
|
Consider the results of a study designed to test the efficacy of a drug by comparing two groups
(treatment vs placebo) of 15 patients each.
|
|
What conclusion would you draw for the efficacy of the drug? |
| Answer spontaneously (without computation) |
|
From a normative viewpoint, the task involves the following simple and general result: d ± 2(d/t) hence here [-2.93,+5.97] This very simple approximation is generally sufficient (the exact interval is [-3.04,+6.08]). This straightforward and easily interpretable result should theoretically prevent the abusive interpretation of a nonsignificant result as "proof of the null hypothesis". Clearly here the data cannot lead to conclude to the inefficacy of the drug (because of the great variability observed. |
|
However, in front of this situation 84% of the professional applied statisticians and 85% of
the psychologists (all with experience in processing and analyzing experimental data) concluded inefficacy.
|
Comportement des chercheurs dans des situations conflictuelles d'analyse des données expérimentales
La démarche du chercheur en psychologie dans des situations d'analyse statistique de données expérimentales
Méthodologie de l'analyse des données expérimentales - Étude de la pratique des tests statistiques chez les chercheurs en psychologie, approches normative, prescriptive et descriptive
Pratiques des tests statistiques en psychologie cognitive: L'exemple d'une année d'un journal
And... what about the researcher's point of view?
The interpretation of significance levels by psychological researchers: The .05-cliff effect may be overstated
Even statisticians are not immune to misinterpretations of Null Hypothesis Significance Tests
Predictive expression of inferential statistical results"The essence of science is replication: a scientist should always be concerned about what would happen if he or another scientist were to repeat his experiment." (Guttman).
Other experimental investigated particularly important questions concerning the replicability of experimental results:
given the results of a first experiment, what is the probability
of finding again similar results (for some criterion) in a replication of the experiment?
Recently, the Association for Psychological Science introduced in the
"author guidelines"
of Psychological Science, a new norm of publication asking
the authors to report a "probability of replication"
("Killeen's prep").
What is the probability of finding again...
|
In a study that compares an experimental condition to a control condition, a difference +1.82 between the two moyennes has been observed. The difference is significant at two tailed level 0.05: t=+2.09, 19 degrees of freedom, p=0.05. |
|
(1) What, for you, is the probability that, in a replication of the experiment,
the observed difference will be positive? |
| Answer spontaneously (without computation) |
|
From a normative viewpoint, since there is no a priori information external to the experiment,
it seems reasonable to base the prediction on the data only. |
|
The majority of investigated psychological researchers underestimated the first probability
and overestimated the second probability.
|
Predictive judgments in situations of statistical analysis
And... what about the researcher's point of view?
Even statisticians are not immune to misinterpretations of Null Hypothesis Significance Tests
|
1. Behaviour of statistical inference users |
2. Probabilistic representations in purely random situations |
3. Representations of randomness |
2. Probabilistic representations in purely random situations"Les questions les plus importantes de la vie ne sont en fait, pour la plupart, que des problèmes de probabilité." (P.-S. Laplace)
Study of purely random situations (games of chance, drawings from a jar, etc.), etc, with adult subjects and children. This study involves:
probabilistic Judgments essentially based on considerations of symmetry.
Many outstanding results have been obtained.
Existence of implicit models and in particular the equiprobability biasThere exits an intra-subject vicariance of different cognitive models in various structurally isomorphic situations. The specific activation of a particular model is mainly linked to the "surface features" of the situations. The chance context of a "purely random" situation evokes to most subjects an implicit model called "chance model" which is not adequate: random events are thought to be equiprobable "by nature" (equiprobability bias).
A problem with socks
|
A pair of socks is (blindly) draw from a drawer in which there are a pair of red socks and a pair
of green socks.
|
|
Do you think there is:
|
| Answer spontaneously (without computation) |
|
The correct response is:
2) more chance of obtaining result 2
|
|
If you answered "an equal chance of obtaining the two results" (equiprobability bias),
you are part of the majority.
|
Activation of an appropriate model by masking the random aspect of the situation The chance model is highly resistant. Nevertheless, appropriate combinatorial or logical models are available to most subjects. It is possible to induce the activation of appropriate models from experimental tricks consisting in masking the random aspect of the situation.
The role of the subject's cognitive activity
Nevertheless such an activation remains superficial: the transfer of an appropriate model to
an isomorphic random situation is not as frequent as one might expect. The little transfer could be
explained by the fact the subjects did not succeed in constructing an abstract representation of
the situation.
The main purpose of the more recently carried out experiments is to demonstrate
that when the subjects succeeded in constructing an adequate representation by themselves
(with situations of "cognitive conflict", learning situations with feed-back, etc.), then the
inadequate "chance model" would fail, and the result of such a cognitive activity would be a more
frequent and a more stable transfer to isomorphic situations.
A lot of questions remain asked. In particular, what is the origin of the cognitive models which are
spontaneously available to most subjects, and appear so highly resistant? Is it various everyday-life
experiences, erroneous interpretations of what is taught? Furthermore, when experimental tricks are used
to trigger the activation of appropriate models and transfer to isomorphic situations occurs,
is such an acquisition stable?
A study of two biases in probabilistic judgments: representativeness and equiprobability
Cognitive models and problem spaces in "purely" random situations
Étude de l'évolution de biais probabilistes avec l'âge à partir de résultats obtenus en France et en Israël
Learning and transfer in isomorphic uncertainty situations: The role of the subject's cognitive activity
Failure to construct and transfer correct representations across probability problems
|
1. Behaviour of statistical inference users |
2. Probabilistic representations in purely random situations |
3. Representations of randomness |
3. Representations of randomness"The successes of quantum theory no more prove the randomness of nature than the success of statistical description of coin flipping proves that coin flipping is intrinsically random, or the fact that a random number algorithm passes statistical tests proves that the numbers it produces (in a purely deterministic fashion!) are random." (T.J. Loredo)
The representations of randomness are studied in various random situations. In particular, categorization situations are considered. Two types of items are distinguished:
There is random and random
|
Consider the two following events:
"The fact that a pair of socks that match is obtained from a blindly draw of two socks from a drawer in which
there are two pairs of different socks"
|
|
Do you think that randomness is involved or not in each of these two events? |
| Answer spontaneously |
|
Of course, there is no "good response"! |
|
Three groups of subjects have been questioned: college undergraduates students,
researchers in psychology, and researchers in mathematics and statistics.
|
Somes results
Our studies have confirmed the wide range of meanings that individuals attach to the notion of randomness
Nevertheless, it was possible to distinguish some general conceptions of randomness.
Another important finding was the little effect of the background in the theory of probability on the views on randomness.
However psychologists and mathematicians exhibited some distinctive features.
The arguments for judging that an event is random and for judging that an event is not random were found
to be different by nature.
Application de la méthode des arbres de similarité additifs de Sattath et Tversky dans une tâche de catégorisation de situations d'incertitude
Interprétations intuitives du hasard et degré d'expertise en probabilités
People's intuitions about randomness and probability: An empirical study
Catégorisation de situations d’incertitude et variabilité des points de vue sur le hasard
|
1. Behaviour of statistical inference users |
2. Probabilistic representations in purely random situations |
3. Representations of randomness |
