Hostname: page-component-7b9c58cd5d-wdhn8 Total loading time: 0 Render date: 2025-03-16T23:17:41.908Z Has data issue: false hasContentIssue false
  • English
  • Français

A Monte Carlo Analysis of the Fisher Randomization Technique: Reviving Randomization for Experimental Economists

Published online by Cambridge University Press:  14 March 2025

Robert Moir
Robert Moir*
Affiliation:
Department of Economics, University of New Brunswick, P.O. Box 5050, Saint John, NB, E2L 4L5, Canada
Get access Rights & Permissions [Opens in a new window]

Abstract

Data created in a controlled laboratory setting are a relatively new phenomenon to economists. Traditional data analysis methods using either parametric or nonparametric tests are not necessarily the best option available to economists analyzing laboratory data. In 1935, Fisher proposed the randomization technique as an alternative data analysis method when examining treatment effects. The observed data are used to create a test statistic. Then treatment labels are shuffled across the data and the test statistic is recalculated. The original statistic can be ranked against all possible test statistics that can be generated by these data, and a p-value can be obtained. A Monte Carlo analysis of t-test, the Mann-Whitney U-test, and the exact randomization t-test is conducted. The exact randomization t-test compares favorably to the other two tests both in terms of size and power. Given the limited distributional assumptions necessary for implementation of the exact randomization test, these results suggest that experimental economists should consider using the exact randomization test more often.

Keywords

Randomization testMonte Carlo simulation

JEL classification

C15: Statistical Simulation Methods: General C90: General
Type
Research Article
Information
Experimental Economics , Volume 1 , Issue 1 , June 1998 , pp. 87 - 100
Copyright
Copyright © 1998 Economic Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

References

Alf, E.F. and Abrahams, N.M. (1972). “Comment on Component-Randomization Tests.” Psychological Bulletin. 77(3), 223–224.CrossRefGoogle Scholar
Bradley, J.V. (1968). Distribution-Free Statistics. Engelwood Cliffs, NJ: Prentice-Hall.Google Scholar
Chan, K., Mestelman, S., Moir, R., and Muller, R.A. (1996). “The Voluntary Provision of Public Goods Under Varying Income Distributions.” Canadian Journal of Economics. 29(1), 54–69.CrossRefGoogle Scholar
Edgington, E.S. (1987). Randomization Tests. New York: Marcel Dekker.Google Scholar
Edgington, E.S. and Ezinga, G. (1978). “Randomization Tests and Outlier Scores.” Journal of Psychology. 99(2), 259–262.CrossRefGoogle Scholar
Edgington, E.S. and Strain, A.R. (1973). “Randomization Tests and Computer Time Requirements.” Journal of Psychology. 85(1), 89–95.CrossRefGoogle Scholar
Efron, B. and Tibshirani, R.J. (1993). An Introduction to the Bootstrap. Toronto: Chapman & Hall.CrossRefGoogle Scholar
Fisher, R.A. (1935). The Design of Experiments. New York: Hafner.Google Scholar
Good, P. (1994). Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses. New York: Springer-Verlag.CrossRefGoogle Scholar
Kennedy, P.E. (1995). “Randomization Tests in Econometrics.” Journal of Business and Economic Statistics. 13(1), 85–94.CrossRefGoogle Scholar
Manly, B.F.J. (1991). Randomization and Monte Carlo Methods in Biology. New York: Chapman and Hall.CrossRefGoogle Scholar
Mestelman, S. and Welland, D. (1988). “Advance Production in Experimental Markets.” Review of Economic Studies. 55, 641–654.CrossRefGoogle Scholar
Moir, R. (1996). “The Analysis of Cooperation Collective Action Games: Experimental Evidence.” Ph.D. thesis, McMaster University.Google Scholar
Noreen, E.W. (1989). Computer Intensive Methods for Testing Hypotheses. Toronto: Wiley.Google Scholar
Pfaffenberger, R.C. and Patterson, J.H. (1987). Statistical Methods. Homewood, IL: Irwin.Google Scholar
Rasmussen, J.L. (1986). “An Evaluation of Parametric and Nonparametric Tests on Modified and Non-modified Data.” British Journal of Mathematical and Statistical Psychology. 39(2), 213–220.CrossRefGoogle Scholar
Siegel, S. (1956). Nonparametric Statistics for the Behavioral Sciences. Toronto: McGraw-Hill.Google Scholar
Still, A. and White, A.P. (1981). “The Approximate Randomization Test as an Alternative to the F-test in Analysis of Variance.” British Journal of Mathematical and Statistical Psychology. 34(2), 243–252.CrossRefGoogle Scholar

Technical Resources

Hastings, N.A.J. and Peacock, J.B. (1975). Statistical Distributions. Toronto: Wiley.Google Scholar
Hergert, D. (1988). Microsoft Quick BASIC. Redmond, WA: Microsoft Press.Google Scholar
Microsoft. (1995). “RND and RANDOMIZE Alternatives for Generating Random Numbers.” Microsoft Support Online. Article ID: Q28150.Google Scholar
Press, W.H., Flannery, B.P., Teukolsky, S.A., and Vetterling, W.T. (1986). Numerical Recipes. New York: Cambridge University Press.Google Scholar