Exploratory data analysis in the context of data mining and resampling.

Chong Ho Yu

Abstract


Today there are quite a few widespread misconceptions of exploratory data analysis (EDA). One of these misperceptions is that EDA is said to be opposed to statistical modeling. Actually, the essence of EDA is not about putting aside all modeling and preconceptions; rather, researchers are urged not to start the analysis with a strong preconception only, and thus modeling is still legitimate in EDA. In addition, the nature of EDA has been changing due to the emergence of new methods and convergence between EDA and other methodologies, such as data mining and resampling. Therefore, conventional conceptual frameworks of EDA might no longer be capable of coping with this trend. In this article, EDA is introduced in the context of data mining and resampling with an emphasis on three goals: cluster detection, variable selection, and pattern recognition. TwoStep clustering, classification trees, and neural networks, which are powerful techniques to accomplish the preceding goals, respectively, are illustrated with concrete examples.

Keywords


exploratory data analysis; data mining; resampling; cross-validation; data visualization; clustering; classification trees; neural networks

Full Text:

PDF

References


Altman, D. G., & Royston, P. (2000).What do we mean by validating a prognostic model? Statistics in Medicine, 19, 453-473.

Baker, B. D., & Richards, C. E. (1999). A comparison of conventional linear regression methods and neural networks for forecasting educational spending. Economics of Education Review, 18, 405-415.

Behrens, J. T. & Yu, C. H. (2003). Exploratory data analysis. In J. A. Schinka & W. F. Velicer, (Eds.), Handbook of psychology Volume 2: Research methods in Psychology (pp. 33-64). New Jersey: John Wiley & Sons, Inc.

Behrens, J. T. (1997). Principles and procedures of exploratory data analysis. Psychological Methods, 2, 131-160.

Berk, R. A. (2008). Statistical learning from a regression perspective. New York: Springer.

Breiman, L., Friedman, J.H., Olshen, R.A., & Stone, C.J. (1984). Classification and regression trees. Monterey, CA: Wadsworth International Group.

Carpio, K.J.E. & Hermosilla, A.Y. (2002), On multicollinearity and artificial neural networks, Complexity International, 10, Retrieved October 8, 2009, from http://www.complexity.org.au/ci/vol10/hermos01/.




DOI: http://dx.doi.org/10.21500/20112084.819

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.