Description and discussion of
FACTOR ANALYSIS
LITTLE EXPERIMENTATION IS PERMISSIBLE ON HUMAN PERSONALITY; SO IT IS IMPORTANT TO MAKE THE BEST OF CORRELATIONAL DATA. FACTOR ANALYSIS ALLOWS LARGE CORRELATION MATRICES TO BE SUMMARIZED AND RE-DESCRIBED IN MORE SIMPLE TERMS; AND IN THE COURSE OF THIS IT WILL SOMETIMES INDICATE POSSIBLE CAUSAL STORIES AS TO THE MAIN INFLUENCES THAT MIGHT HAVE BEEN AT WORK TO GENERATE THE CORRELATIONAL ARRAY. HOWEVER, THE METHOD HAS HAD DETRACTORS AS WELL AS SUPPORTERS--AS THE FOLLOWING INTRODUCTORY QUOTATIONS INDICATE.
"In my view the primary object of factorial methods is neither causal interpretation, nor statistical prediction, but exact and systematic description. And I suspect that most of the confusion has arisen because factors, like the correlation coefficients on which they are based, have been invoked to fulfil these three very different purposes, and so have made their appearance at three very different levels of thought-like the famous legal firm of Arkles, Arkles & Arkles, which, 'more to its own satisfaction than that of its clients, canvassed three different lines of business in three small offices on three different floors."
C.BURT, 1940, The Factors of the Mind. London University Press.
"[Factorial analysis is] a brilliant but misguided departure from the central path of empirical psychology."
O.L.ZANGWILL, 1950, An Introduction to Modern Psychology. London : Methuen.
"....factor analysis provides no way to discover or explicate the processes that in combination constitute intelligence."
R.J.STERNBERG, 1977.
"....despite certain limitations, factor analysis can aid in the identification and characterization of cognitive processes."
J.B.CARROLL, 1988.
"[The Cambridge psychologist, Sir Frederick Bartlett] regarded factor analysis as a technical cul-de-sac of psychology."
Paul KLINE, 1993, A Handbook of Psychological Testing.
London : Routledge.
"One issue on which [John Carroll (1994, Human Cognitive Abilities, CUP)] is adamant is that, contrary to the clamor of critics, the track record of factor analysis provides a sound assessment for understanding the nature of cognitive abilities."
Kevin LAMB, 1994, Mankind Quarterly 24.
"The history of the development of factor analysis is not edifying. Pearson and Spearman, who together had the skills to develop it rapidly, quarrelled and feuded. ....Thomson....has been neglected. Garnett....was ignored. The new wave of American factorists in the 1930's lumped all the British workers in the field together, called them Spearman's school, and set out to rubbish them. Thurstone in particular turned the clock back.... The squabbles of the 1930's have continued with scarcely a break.... [Burt's] real influence is felt through the work of those of his students, such as R.B.Cattell and H.J.Eysenck, who went on to build factorial theories that have attracted widespread attention, and through the continuing line of thinking which is prepared to allow both g and more particular kinds of ability.... ....one thinks of the work of Philip Vernon, and more recently of course Jensen has laid emphasis on the value of Burt's contributions." S.F.BLINKHORN, 1995, 'Burt and the early history of factor analysis.' In N.J.Mackintosh, Cyril Burt: Fraud or Framed? Oxford University Press.
"I inspected the subject index of some well-known texts in experimental cognitive psychology and found that the term factor analysis never appears in the subject index."
D.DORFMAN, 1995, Contemporary Psychology 40.
The procedures that are involved in factor analysis (FA) as used by psychologists today have several features in common with the procedures for administering Rorschach inkblots. In both procedures, data are first gathered objectively and in quantity; subsequently, the data are analysed according to rational criteria that are time-honoured while not fully understood by most users; finally, in case it had not been enough to have a choice between various analytic schemes, users are allowed to interpret the 'results' in their own ways. It is probably the combination of these three attractive facilities that defies prophets of doom and accounts for the enduring popularity of FA in psychology (especially when experimentation is impossible).
FA allows psychologists {and other users}:
(i) to handle the almost lifelike complexity of multiple measures while reducing that complexity to slightly more manageable proportions;
(ii) to demonstrate semblances of methodological sophistication, numeracy, computer wizardry and respect for their tribal elders; and
(iii) to enjoy the subjectivity and originality of the artist within the pretty secure framework provided by well-rehearsed routines of reason and science.
These three achievements are loosely associated with the three broad stages of FA:
(i) the collection of multivariate data and its preparation for analysis;
(ii) the transformation (normally reduction) of the correlation (r) matrix to a factor matrix; and
(iii) the final rotation and interpretation of the factors.
Users who take each stage seriously can feel themselves embarked on a voyage of discovery the very conservatism of which makes for outcomes that are at once original and reasonably truthful.
At the same time, both the 'conservative' and the 'creative' aspects of FA can be abused-especially because of the large number of particular steps that are involved. A common accusation of undue 'conservatism' would be that the authorized procedures are too often applied with unreasoning faith in the wisdom of the ages. It is the unthinking, mechanical use of FA that attracts the criticism "Garbage in-garbage out!" On the other hand, the 'creative' options within FA sometimes seem to allow such a wide discretion to the researcher as to make FA "more of an art than a science." Whereas Popper advised scientists to weed out false hypotheses so that truth may flourish, FA can sometimes seems to have more hypotheses at its end than at its beginning (e.g. by allowing rotation of the axes to theoretically favoured positions). Not the least galling to the critic is the discovery that factorists actually rejoice in such liberties. A casual user may want FA to provide the solution-and FA will oblige; but with greater sophistication, a user will end not just with factors but with ideas for what factors could be expected in different, improved data sets.
The aims and limitations of Factor Analysis (FA): an analogy.
{OR: A View from the Motorway Bridge....}
The central aim of FA is to reduce the covariance in a correlation matrix to more manageable proportions. [The covariance is the 'amount of correlation', or, precisely, 'the sum of all the squared correlations between the variables in a table of correlation coefficients (usually Pearson r's)'. (Squaring correlations eliminates minus signs and highlights stronger correlations.)] The idea is to explain the covariance of a relatively large number of variables in terms of a rather smaller number of factors. [Such explanation is not a discovery of causation -except by happy accident. What is being carried out is an accounting exercise in which scores on variables are finally 'accounted for' statistically, and rendered predictable from, scores on hypothetical factors.] It could be said that factors are parsimonious packages, each involving bits -often quite large bits-of the covariance found amongst the larger number of original variables. If the analysis has gone smoothly-without too many departures from conventional assumptions and expectations-the resulting factors will be promulgated, instead of the original variables themselves, as being capable of serving as the source of the original observed covariance. That is, the factors could be causal but certainly enable prediction with economy.
Each resulting factor is thus a hypothetical source of the covariance in the study-and being a mere hypothesis is a headstart over not emerging as a factor at all! Yet a factor's relation to variables is no different from the relations that may be envisaged to exist among variables themselves. When variables X (say, 'students' knowledge of St Augustine') and Y ('knowledge of St Thomas Aquinas') both correlate better with Z ('knowledge of Aristotle') than they do with each other, the principle of parsimony in scientific explanation inclines us to look with optimism at the possibility that Z actually influences both X and Y; or at least that Z taps, or reflects especially well an important underlying factor, say Zf, that itself influences both X and Y. [For example, a fourth variable or factor, 'number of philosophy courses passed by students', might explain, or account for quite a lot of the variation ('differences') between people in their knowledge of all three philosophers; and the 'Aristotle' item might have tapped that knowledge especially well because Aristotle is the best known of the three philosophers-making it especially unlikely that a person could know a lot about philosophy without knowing quite a bit about Aristotle. (By contrast, had the correlations with 'Aristotle' been weak, the covariance in the matrix would be chiefly in knowledge of both Augustine and Aquinas. Why might such differences have arisen, independently of knowledge of Aristotle? Since both Augustine and Aquinas are Saints, individual differences in knowledge of them might have reflected differences in degree of religious interest and exposure.)] A factor is just like a variable that correlates especially well with several other variables.
Suppose we wish to learn from scratch about motor cars-starting from only the most elementary knowledge that differently shaped and titled bodies of metal can be seen moving around the roads. That is to say, suppose that we are able to see some of the differences between different makes of motor car, yet we cannot drive them ourselves, read about them, experiment with them or even measure them in any sophisticated way. (In fact, such restrictions are pretty similar to those under which psychologists labour when trying to find out anything very interesting about people!) Frustrating as our situation may be, we can at least make frequency counts of the appearance of the different makes of car in various locations; and we can do a lot of counting quite easily if we stand on a motorway bridge and count the frequency with which the makes appear in the various lanes underneath. (In such a study, the lanes would be analogues of the psychologist's tests, or variables; the makes of car would be equivalent to individual testees, or 'subjects'; and the correlations between appearance-frequencies-per-unit time (between the makes' scores for frequency-of-appearance-in-Lane-1, frequency-in-Lane-2, Lane-3, etc.) would provide the contents of the correlation matrix to be submitted to FA.)
Now, suppose all the lane-appearance-frequencies (LAF1, LAF2, etc.) are positively correlated (i.e. the same makes appear more often in all the lanes). We can now talk of a factor of 'general frequency'. We will conclude that, no matter which measure we take, one 'source' of each variable's variance is that -for whatever reason-there are simply more of some makes of car than of others. (Which make is more frequent might still differ by location-e.g. in shopping precinct car parks, or on motorways in another country. But the overall positivity of our correlations, the 'positive manifold', offers some assurance as to the generality, or at least the not-complete-specificity of frequency differences.) This is our first discovery. [By contrast, had the average of the inter-LAF correlations (r's) been zero, we would have falsified the hypothesis that there are, quite generally, more of some cars than of others.]
However, even amidst such general positivity of intercorrelation, further factors might be detectably at work. For example, if the r's between LAF1 and LAF6 (between, say, frequencies on the extreme 'inside' and 'outside' lanes of the London-bound carriageway) were significantly lower, we would begin to think that the makes of car also differed in some other respect than just that of 'general frequency'. If we knew the significance of cars' being in such lanes, we might conclude that makes of car differed in the typical speeds at which they are driven-whether from basic capacity differences between them or because they attracted purchasers having different requirements or propensities (such as levels of risk-taking). We might be able to check this hypothesis, perhaps by recording 'speed' as an additional variable, or by making some motorway journeys and counting the appearance-frequencies of various makes on the 'hard shoulder'. For example, if LAF1 correlated with hard-shoulder-frequency more than LAFs generally correlated with each other, we might conclude that there were indeed capacity differences between makes; and we might interpret a factor of 'speediness vs breakdown frequency' as being one of underlying differences in 'roadworthiness'.
Yet further factors might be recognized, even within the constraints of such a ham-strung study. Suppose the LAF r's are generally positive-yielding the first, 'general frequency' factor; and lower (or even negative) between the extreme inside and outside lanes-yielding a second, 'speed/roadworthiness' factor. But now suppose the extreme inside lane, LAF1, correlates significantly less with the other inside lanes (LAF2 and LAF3) than they do with each other. In this case we might note a potentially specific, third source of variance, and perhaps hypothesize that LAF1 is specially influenced by drivers' preparations for turning off the motorway-towards, say, some leafy suburb in which make-ownership was unrepresentative of general motorway prevalence, and not in ways reflecting choice of makes having low capacity for speed. Thus we would resolve that our next 'motorway FA' would include a frequency count from an actual exit lane so as to be able to confirm the existence of a 'leafy suburb' factor and precisify its nature. [The inclusion of such an 'exit lane' variable might have proved particularly valuable if the first FA had thrown up not the two quite intelligible factors of 'general frequency' and 'speed/roadworthiness' but two equally large and independent factors of 'inside lanes frequency' and 'outside lanes frequency'. For, by distinguishing (cf. rotating to) a special 'leafy suburb' factor-which would have 'pulled in' some of the variance on LFA1, and thus some of the negative covariance between the inside lane variables'-we might have strengthened the 'inside lanes factor' and given it a positive r with the 'outside lane factor'. Thus, putting 'exit lane' variance to one side-or extracting that factor-might actually have allowed the more interpretable factors of 'general frequency' and 'speed/roadworthiness' to emerge in their own right.]
[For an actual example of FA on physical objects, see Laumann & House, 1970, Sociol. Res. Here the analysis is of the contents of 800 living rooms. It yielded two factors of 'social class' and 'tradition vs modernity'. Richardson (1978, Neuropsychologia) provides a readily intelligible FA of handedness: he finds writing, hammering and throwing to provide the best items for measure the general factor of right- vs left-handedness.]
Such are the main possibilities of discovery and discourse that factor analysis enables. The provision of a 'summary' is certainly important in much of the work in human individual differences, where the measures that are available can so seldom be described as being established 'tests' of anything that is well understood in its untested form. At the same time, much more interest attaches to summaries of sense than to summaries of nonsense: careful and wide-ranging consideration of which variables to use and of which factors to aim for (at the last stage of factor rotation) will show FA at its best.
References for psychologists
The best introduction for psychology students is probably:
CHILD, D. (1971). Essentials of Factor Analysis.
Shorter (several-page) summaries of FA are provided by:
CRONBACH, L.J. (1961 et seq.) Essentials of Psychological Testing.
CATTELL, R.B. (1971) Abilities.
LEMKE & WIERSMA (1976). Principles of Psychological Measurement, Chap. 8.
CATTELL,R.B. & KLINE,P.(1977) The Scientific Analysis of Personality and Motivation.
KLINE, P. (1981) in F.Fransella, Personality.
RUST, J. & GOLOMBOK, S. (1989). Modern Psychometrics, Chapter 9.
KLINE, P. (1993) Handbook of Psychological Testing.
The classic, comprehensive account is:
HARMAN, H.H. (1976) Modern Factor Analysis, 3rd edn. Chicago University Press.
Readers wishing helpful accounts and demonstrations that bring out the psychological relevance [or, some may cruelly claim, irrelevance!] of factor-analytic procedures and arguments might consult:
THURSTONE, L.L. (1934) Psychological Review 41.
GUILFORD, J.P. & GUILFORD, T. (1936) J. Psychology.
CATTELL & DICKMAN (1962). Psychol. Bull. (re reality of 'oblique factors')
FRANSELLA, F. & ADAMS, B. (1966) Brit. J. social & clinical Psychol.
CATTELL et al. (1970). Brit. J. Psychol.
EYSENCK, H.J. (1971) Brit. J. social & clinical Psychol.
BRAND, C.R. (1972) Brit. J. social & clinical Psychol.
NEISSER, U. (1979) Bulletin of the Brit. Psychol. Socy.
GOULD, S.J. (1981) The Mismeasure of Man.
JENSEN, A.R. (1981) Bias in Mental Testing.
BLAHA & WALLBROWN (1982) J. consulting & clinical Psychol. (re 'hierarchical FA')
CARROLL, J.B. (1982) in R.J.Sternberg, Handbook of Human Intelligence.
KLINE, P. & BARRATT, P. (1983). Adv. in Behav. Res. & Ther. 5. (re 'obliquity')
EYSENCK, H.J. & EYSENCK, M.W. (1985) Personality and Individual Differences:
A Natural Science Approach.
JENSEN, A.R.(1985). Behavioral & Brain Sciences. (re 'hierarchical FA')
McCRAE, R.R. & COSTA, P.T. (1985) J. Personality.
PARKES, K. (1985). Person. & Indiv. Differences 6 (re 'number of factors to extract')
OBRIGG & CHEEK (1986). J. Personality. (re 'confirmatory FA')
EMLER, N. (1987) J. Child Psychol. & Psychiatry.
SCHONEMANN, P. (1987) in S. & C. Modgil, Arthur Jensen:
Consensus and Controversy.
MEYER et al. (1988) Personality & Individual Differences 9.
BRAND, C.R. & EGAN, V. (1989) Personality & Individual Differences 10.
CARROLL, J.B. (1993) Human Cognitive Abilities.
Factor Analysis: its three main stages.
(i) The main issues and disputes that arise regarding the selection and preparation of data for FA are as follows.
(a) There should be some variance-or, quite simply, some differences between testees [or between whatever units have been assessed]. More precisely, there should be an appropriate range of scores on most variables: so, usually, standard deviations should approximate those normally found for the variables to be analysed. This may seem an obvious point; but variance-failures can easily arise in two ways.
(1) The subjects who have been chosen for the study may not differ markedly-by population standards-because of some kind of pre-selection: e.g. IQ's will hardly range below 105 in studies of British university students; and the attitude-scores of students of psychology will seldom range sufficiently widely to allow researchers to explore the further reaches of 'authoritarianism', 'tough-mindedness', 'realism' or 'social conservatism'.
(2) When new, 'experimental' items are used in a questionnaire survey, some of them will have less variance than the researcher had envisaged: e.g. sympathy for 'civil servants', 'teachers' or 'gypsies' may be much higher than the test constructor believed from reading the newspapers.
[A useful rule of thumb might be to arrange visual display of frequency distributions and/or scattergrams before proceeding to FA. An FA may proceed using non-normal variance, but its conclusions will need to be appropriately qualified.]
(b) Flooding the data-matrix with related variables (e.g. with different facets of extraversion, such as liveliness, jocularity, adventurousness, sociability and activity -or even with sensation-seeking, risk-taking and impulsivity) will produce a large extraversion factor, accounting for an impressive percentage of covariation. Conversely, even the most familiar dimensions of differential psychology will have a struggle to 'emerge' if no marker variables have been put in to define them. [Ideally, the FA researcher seeking 'the dimensions of personality / attitudes / abilities / etc.' should be able to say of what population of variables the study's selected variables provide a sample. (A specified sample might involve 'all the tests mentioned by particular authors' or '....mentioned in a particular journal over a period of time.')]
(c) The inclusion of scores that depart extremely from sample means can bias resulting r's and thus factors. Often, such outliers are the result of clerical errors in scoring or at the stage of data input at the computer keyboard. Sometimes they are just that one subject is markedly older, and more traditionally 'religious' than the rest: such subjects should make researchers question 'Of what population are my subjects intended to provide a sample?' [Particularly in small sample (when N < 100), there is a case for setting aside subjects whose scores are more than three standard deviations high or low on any variable-while prominently documenting such exclusion when reporting the study.]
(d) Whether to analyse items or item parcels was long a matter of dispute between the mighty psychometrician-psychologists, Hans Eysenck and Raymond Cattell. Individual items seldom have any substantial percentage of their variance determined by the factor that they are intended to 'measure'. Even IQ items seldom correlate at more than .20 with final IQ; and personality items-such as for extraversion-typically correlate at rather less than .10 with the trait that gleams in the expectant researcher's eye. Analysing individual items can thus lead to peculiar results in small studies and to 'bloated specifics' (e.g. involving two or three items mentioning 'punctuality') in larger studies. Cattell therefore favoured using parcels of four or five items (all of some pre-established broad type) as the unit of analysis. [This enabled Cattell to develop measures of a rather larger number of personality factors and dimensions than Eysenck -indeed, Cattell's usual six, independent, 'second-order' dimensions of personality are the forerunners of the 'Big Six' dimensions of modern personality psychometry (e.g. Ormerod & Billig, 1981, Person. & Indiv. Diffs 2.; Deary & Matthews, 1993, The Psychologist;; Matthews & Oddy, 1993, Person. & Indiv. Diffs 14; Brand, 1994, Psychologica Belgica). However, Eysenck (e.g. 1992, Person. & Indiv. Diffs.) still tends to maintain that his 'Gigantic Three' personality dimensions are more substantial and scientifically well-specified in terms of established underlying mechanisms and processes. At present, it can only be suggested that item parcels will be particularly suitable when there appears to be some special, 'parcel-added' meaning to high or low scores on all the items. For example, individual 'honesty' items ('Have you ever cheated / shouted / lied / etc. ?') may reflect real variance in behaviour; however, when a subject denies virtually any human failing, i.e. 'lies' on quite a number of items, hypocrisy can properly be suspected. Thus a 'parcel' of items may pick up real variance in 'hypocrisy / faking good' that is poorly captured by individual items.]
(e) In order to enable relatively clear isolation of meaningful factors from sheer error variance, it is helpful to include in the FA several variables that have nothing to do with any of the meaningful factors that are expected. Some such mock variables might even be composed of mere random numbers. Provision of such hyperplane stuff was long a feature of Cattell's work; and it will serve to prevent over-enthusiasm about factors that emerge and over-imaginative rotations of factor axes {see (iii) below}.
(f) FA serves none but an illustrative purpose unless the number of subjects is at least four times the number of variables. Correlations themselves are less reliable when N's are low and when the subjects' scores derive from untried tests; and factor calculation involves multiplying variables' correlations and is thus particularly affected by unreliability.
(g) FA only searches for linear relations. To check for U-shaped relations between variables, deviation scores [testees' scores minus the mean score on each variable, disregarding the sign] may be included.
(h) When dichotomous or other non-continuous variables are included in FA (as often happens when test items are among the variables), their correlations should be expressed as phi coefficients or as point biserial coefficients. It is quite in order to combine these measures of association with Pearson r's in the final correlation matrix.
(ii) At the second stage of FA -the analysis of the correlation matrix-the main question is that of which FA package to use.
(a) The first choice is whether to analyse only the covariance in the r matrix, or whether to analyse the total variance and covariance in the data. Normally the latter seems more reasonable in psychology: one hardly wants to forget the unshared variance - which may well be shared with other variables that did not happen to be included in the study.
(b) Making the 'total variance' choice {as above} requires a communality assumption about how much each variable has in common with itself-i.e. about the reliability of each variable.
(1)Perhaps the simplest assumption to make is that, by definition, each variable correlates at unity (1.00) with itself. This is what is assumed in quite the most popular type of FA today, known as Principal Components Analysis. The assumption introduces the maximum possible unshared variance and so requires more computational effort-thus it only became popular as computers became available to perform all the necessary donkey work.
(2)Alternatively, an attempt can be made to provide a realistic estimate of the actual reliability of each variable-rather than flood the FA with specific (and often 'error') variance which then has to be distinguished and set aside in special factors during the course of the analysis. One possibility is to insert for rXX the known reliability of Test X-e.g. as given in the test handbook. However, such knowledge may not be available or securely based; and, even if it is, it will be for a population that probably differs in some significant ways from the researcher's present sample. Thus the more usual procedure, used in Centroid Factor Analysis (and in Principal Factor Analysis, after a previous Principal Component Analysis has been undertaken) is to estimate each variable's reliability initially as being as high as the highest correlation which the variable enjoys with any other variable in the matrix. [This is reasonable enough in principle: variables cannot truly correlate with other variables at a higher level than their own reliabilities allow. Later in the factor analysis {see below} this estimate is corrected if it seems to have been incorrect: this process is called 'reiterating the communalities.']
(c) Here is how centroid analysis proceeds-with the help of a worked example. (The centroid method was the original method that was first widely used in psychology of the 1920's. Though other methods improve on it in particulars, no factor analysis will produce results with most psychological data that are importantly different as far as the general user is concerned.)
Here is a specimen r matrix to be analysed - the Original Matrix 1 -
VIQ PIQ SE BE
Verbal Intelligence (VIQ) .60 .00 .00
Performance Intelligence (PIQ) .00 .00
Social Extraversion (SE) .40
Behavioural Extraversion (BE)
1.For each variable, its r's with all other variables and with itself (in so far as its variance is covariance-the 'communality assumption') are added together. (The communality assumption normally inserts the variable's highest r with any other variable in the leading diagonal of the r matrix.)
Thus is created a matrix with a communality assumption- the Original Matrix 2.
VIQ PIQ SE BE ¦ Sum of r's
VIQ (.60) .60 .00 .00 1.20
PIQ (.60) .00 .00 1.20
SE (.40) .40 .80
BE (.40) .80
------
4.00 = Sum of sums
2.00 = (Sum of sums)
2.Each such sum of its correlations, for each variable, is then expressed as a proportion of the square root of the sum of all such sums, in order to express the loading of the variable on the major centroid factor (i.e. to express the correlation of each variable with the centroid factor).
**Thus: Sum of r's / (Sum of sums) = Loading on
first centroid
factor
VIQ 1.20 / 2.00 = .60
PIQ 1.20 / 2.00 = .60
SE .80 / 2.00 = .40
BE .80 / 2.00 = .40
The first factor has now been extracted. It consists simply of the variance that it shares with the variables that 'load' it, as above.
All there is to 'see' of a factor is its correlation with the variables that yield it. A factor might be said to have 'absorbed' a certain amount of the variance of the variables that load it.
{Note: The major centroid factor here is hardly very 'helpful'. It has no obvious interpretation; and implies that the variables had this mysterious factor 'in common' when the original correlation matrix was largely composed of zeros. In fact the picture will change as the FA proceeds; but this first centroid factor can still serve as a caution against taking loadings of .70 too seriously when trying to understand the 'nature' of a factor.}
3. Next it is asked what the original correlations would have been if only the first centroid factor had been at work to generate individual differences in the data. These new, hypothetical r's are, in each case, the product of the variables' loadings on the first centroid factor.
These hypothetical r's make up the Product of Loadings Matrix.
(E.g. the expected r or VIQ with itself, if only the first centroid had been at work would be the product of VIQ's loadings on the centroid, i.e. .60 x .60 = .36.)
VIQ PIQ SE BE
VIQ (.36) .36 .24 .24
PIQ (.36) .24 .24
SE (.16) .16
BE (.16)
4.Each r in the original r matrix (using Original Matrix 2) is then reduced by the above hypothetical values in the Product of Loadings Matrix. These operations effectively partial out -i.e. set aside-the variance from the first centroid that had led to the original r's being as high and as positive as they were.
The result of this operation is the Residual r Matrix:
VIQ PIQ SE BE
VIQ (.24) .24 -.24 -.24
PIQ (.24) -.24 -.24
SE (.24) .24
BE (.24)
5. The process involved in the above four steps is now repeated upon the Residual Matrix. To speed up analysis, the number of negative r's in the matrix is minimized by the process of reflection: the direction of scoring of a variable can be reversed, and its name changed (e.g. from Extraversion to Introversion). This allows larger 'sums of sums', and thus factors accounting for more variance. (It avoids wasting time-an important consideration in the days of hand-cranked calculation!)
For example, reflection of SE and BE-which thus become Social Introversion and Behavioural Introversion-will yield the following Reflected Residual Matrix:
VIQ PIQ SE BE
VIQ (.24) .24 .24 .24
PIQ (.24).24 .24
SE (.24) .24
BE (.24)
6. Since the above residual values are all the same, analysis of this particular Reflected Residual Matrix will yield the same loading for all four variables on the new, second centroid.
Loading on 2nd Centroid
VIQ PIQ SE BE Sum of r's [= Sum / (Sum of sums)]
VIQ (.24) .24 .24 .24 .96 .49
PIQ (.24).24 .24 .96 .49
SE (.24) .24 .96 .49
BE (.24) .96 .49
3.84 = Sum of sums
1.96 = (Sum of sums)
7. The above 2nd Centroid will not seem any more helpful or interesting to a psychologist than was the first. These two factors will certainly require the extra attention provided by the rotational procedures of Stage (iii). However, an important landmark has in fact been reached. For the next Product of Loadings Matrix will have .49² , i.e. .24 in every single cell, just like the Reflected Residual Matrix above. Hence, subtraction of the new Product of Loadings Matrix from the Reflected Residual Matrix will yield a new, second Residual Matrix having zero in each of its cells. At this point the search for factors can and must halt-for no factors are required to account for a matrix involving no correlations!
[Normally, of course, the conclusion will not be so 'neat'-so the decision to search for no more factors will depend on whether the latest residual matrix is deemed to contain any statistically significant correlations: analysis will thus proceed until there is nothing significant left to analyse.]
The result of this particular example of factor analysis is thus that two centroid factors (each loaded by all the variables to some degree) account statistically for the total variance in the original matrix. This solution is ideal in its neatness-but not in its meaningfulness, as will be considered below.
(d) The chief technical problem that arises in factor analysis (when unities are not inserted in the leading diagonal, as they are in Principal Components) is that of how to stabilize the communalities-i.e. of how to equate the estimated (as above, (b) and (c1)) and the obtained communalities. The obtained communalities for each variable are given by the sum of the squares of its loadings on all the obtained factors.
Thus, above, Social Extraversion has an obtained communality (normally written h²) of
.40² + .49² = .16 + .24 = .40
In this case (because the analysis came out so neatly) the obtained communality is just what had originally been estimated. If that had not been so, however, the correct procedure would have been to repeat the whole analysis again - using the new communality estimate (presumably superior for having been 'obtained' empirically) until stability (or insignificant instability) was achieved.
[Even though an initial guess has been made as to what communality values to place in the leading diagonal, new and better estimates can be made as the analysis proceeds. This is because the obtained communality of a variable with a number of independent factors is equal to the sum of the squares of its loadings on each of them. Thus, at any stage in the analysis, the initial estimate of the communality- whether it was made as in the centroid method, or whether it was the value of unity upon which Principal Component Analysis proceeds until Principal Factor Analysis takes over-may be revised and the whole analysis re-started. Although most psychological researches avoid the issue by settling for Principal Components, it was not uncommonly a matter of concern to Cattell and other factorial specialists that numerous such reiterations should be made to 'stabilize the communalities'.]
(e)There is always a choice for the factorist as to when to stop extracting factors-for real-life matrices hardly ever resolve entirely cleanly. On this matter, disputes between Cattell and Eysenck used to be particularly heated, for Cattell felt that Eysenck systematically underestimated the number of factors worth extracting from 'personality' data. In fact, the criterion that 'none of the r's in the last residual matrix should be significant' became generally displaced by the Kaiser-Guttman Criterion.. This specifies that it is not worth extracting factors whose variance (= the sum of the squares of the factor loadings that are made by the variables, also called 'latent root' or 'eigenvalue') does not reach unity: The criterion effectively ensures that each recognized factor will account for as much variance as any single variable in the study; and it is standardly used in Principal Components Analysis. Another criterion is that each recognized factor should account for at least 10% of the total variance in the study: such a percentage is calculated as: (Latent Root x 100) / (Number of Variables). This provides a stricter criterion when the number of variables exceeds ten-for it recognizes fewer, and not more than ten factors. Cattell, however, always preferred a third criterion, that of the scree test. To apply this test, numerous factors are first extracted and the percentage of the total variance for which each accounts is ascertained as above. It is then demanded that the least substantial of the factors to be recognized should itself account for significantly more variance than the largest factor that is to be rejected as too insubstantial. For example, the factors emerging from a study might be arranged as below.
In such a bar-graph, the last and least substantial factor to be accepted will account for a percentage of variance that puts it clearly above the line or 'scree' made by the many further factors having the gradually diminishing '% variance' that would be expected by chance. In this case, the third factor would be the last to be accepted. The burden of the scree test is that the recognized factors should be clearly distinguishable from the many 'garbage' factors in terms of the percentage of variance for which even the least of the recognized factors accounts. A typical empirical 'scree' is shown by Howarth & Browne (1972, Brit. J. soc. & clin. Psychol. -though these authors themselves used the lenient Kaiser-Guttman Criterion to draw their own conclusion that the Eysenck Personality Inventory reflected many more factors than Eysenck himself typically envisaged. A fourth possibility is to extract a number of factors that has been determined in advance. This feat may be particularly impressive if the hypothesis is that 'there will be a certain number of specified factors' and if subsequent inspection confirms that the main factors are indeed of the particular nature expected. However, it is a trick that will rarely work in everyday empirical psychology-except with a few tests that have already been well refined by factor analysis. {In any case, there is no journal in mainstream psychology that even encourages-let alone requires-authors to submit predictions prior to data-collection.}
(f) The last stage of the factor analysis proper consists of the factor matrix. This shows the factors extracted-constituted as factors are by their collection of loadings from each of the original variables. These loadings are essentially correlation coefficients. Although a factor accounting for a decent proportion of overall variance is a thing of beauty and an ever-present cause for 'further research'...., strings of factor-loadings of less than .50 provide-of themselves- very little direct definition of a factor. Rather, they are a reminder that not a single variable could be found that shared even a quarter of its variance with the putatively meaningful factor - even under favourable conditions in which the variables themselves had every chance of contributing to the very nature of that factor. Again, particular attention should be paid to the final communality (h²) values of variables before hastening to interpret a factor in terms of the variables that load it: for it can easily turn out that a variable that has the highest loading on one of the extracted factors has still more variance that is unaccounted for by any factor at all than variance which it shares with the factor in question. If this is not noticed, a factor can all too easily be 'understood' in terms of those very components of item- and test-variance that it was not in fact reflecting. (Lower factor loadings are, however, of interest when the unreliability of the original variables is high - as with the not-infrequent exploratory items of personality and social psychology that take researchers little effort to think up and testees little time to complete.) Lastly, although a factor is sometimes said, impressively enough, to account for 20% or even 60% of the variance (of an original r matrix, or of the matrix of extracted factors), there is a snag. It must never be forgotten that 'the variance' in question is 'the covariance amongst these variables in the present subjects'-with or without the assumption of Principal Components that each variable has its own true and unique variance. FA will not go out of its way to remind users of real-life variance that was not included in the analysis-such as 'error variance', test-retest variability, variance from other variables, and variance occurring amongst types of testee not included in the present sample.
(iii) The third and last broad stage of FA is that of rotation of the axes.
This serves two purposes. It allows 'psychological' errors that have occurred at the previous stages to be corrected; and it offers a new set of criteria for deciding 'just what the factors really are'. That it should do the latter may seem strange in view of what has already been said. However, the simple point is that, when any two independent factors are plotted orthogonally [at a right angle] to each other - intersecting at the point at which loadings on each of them are zero - it may turn out that most of the variables, when their two loadings are plotted at one point against the independent axes, fall not on either of the axes but towards the middle of the four quadrants.
******************************
Thus, the two centroid factors in the example can be plotted at right angles because they are, by their manner of extraction, independent of each other. The two 'IQ' variables will both fall at the same position in the two-dimensional space, being loaded positively and indeed having identical loadings on both factors.
Centroid 1
|
|
.60 | IQ(V & P)
|
.40 |
|
________________ |__________________ Centroid 2
| .49
|
|
|
|
|
|
To plot the 'Extraversion' variables similarly requires 'reflection' once more-of two of the loadings of +.49 on Centroid 2. These positive loadings were made by variables (originally Social and Behavioural Extraversion) that by then had been reflected and renamed 'Introversion'. After re-reflection, the loadings of the 'E' variables-both Social and Behavioural-can be represented as follows.
Centroid 1
|
|
.60 | IQ(V & P)
|
E(S&B) .40 |
|
________________ |__________________ Centroid 2
-.49 | .49 (after reflecting
| SI and BI)
|
|
|
|
|
Here all the variables (VIQ, PIQ, SE and BE) have their loadings represented by points in two-dimensional space that are well into the quadrants rather than close to the line made by either factor. This is because no single variable had a near-zero loading on either factor.
Moreover, in the example, it can hardly be suggested that either of the factors looks at all 'helpful' or intelligible. Centroid 1 is defined positively by all the variables. And Centroid 2 (after reflection of BI and SI) suggests there is a complementary opposition amongst all the variables - now with intelligence being opposed to extraversion. What can such factors mean? - Especially when one centroid factor seems but a contrast to the other - taking away, as it were, what the other centroid factor put in place?
*****************************
Clearly, what has happened is that many or all of the original variables are loading many or all of the factors to some degree. For sure, the analysis has yielded a 2-dimensional factor space defined by North-South and East-West axes; and these axes have been selected according to a rational criterion - that the first one of them account for 'the most variance' while the other be independent of the first. (In the particular example used, we also know these two factors are exhaustive: they account between them for 100% of the variance.) Yet these may not seem particularly compelling reasons for continuing to describe the 2-dimensional space in terms of such axes. The situation is far from that which makes 'North-South' and 'East-West' singularly useful (and widely understood) ways of describing differences between real-life geographical locations. Particularly when most variables cluster elsewhere in 2-D space than around such axes, it may seem preferable to rotate the axes to some position (of simple structure) such that some variables load primarily on one of the new rotated factors while the remaining variables chiefly define the other.
**********************************
In the example, what is evidently required is an approximate 45 rotation of the axes so that one will line up with VIQ and PIQ while the other will line up with SE and BE.
The precise loadings of the variables can be read off from their positions along each of the rotated centroids. One rotated centroid will be loaded at about .75 by the IQ variables while the Extraversion variables will project on to it at around zero. The other centroid will now be principally defined by Extraversion-with both BE and SE loading it at about .60.
*************************************
Such a rotation of the axes may seem even more appealing when (as in the example) we are used to calling our variables by broad titles such as 'verbal ability' and 'spatial ability' and when we do not generally expect some variables (like IQ and Extraversion) to have much to do with each other. {Even if one large and broad factor brought many mental abilities together as loading on it, some psychologists would still feel tempted to tease them apart rather than struggle to understand a second factor (independent of this broad g factor) that would, by its mixture of positive and negative loadings (as on the second centroid in the example), imply that there was something-some factor-that tended to facilitate some particular abilities while actually impairing others.} However, setting theoretical (psychological) prejudices aside, the 'objective' way to resolve the question of how to rotate is the criterion of simple structure. Here the search is for that rotation (or set of rotations -when more than two factors are involved) that maximizes the number of near-zero, non-significant factor loadings in the factor matrix as a whole. Put another way, the idea is that a majority of variables should end up identified principally with just one factor; and, by the same token, that each factor should be defined in terms of a relatively small number of variables. [The appeal of searching for simple structure will thus be greater as and when the researcher feels sure of what most of the variables were measuring in the first place. By contrast, Arthur Jensen has considered it folly to attempt to break up clear g factors by an obsessional search for simple structure. Parsimony certainly dictates that strong, general first factors should be taken pretty seriously unless simple structure rotation provides clear theoretical improvements.]
The possibility of rotating to well-defined clusters of variables in factor space (like the IQ' and 'E' clusters in the example) will increase enormously if we loosen the shackles of orthogonality. For, if we allow ourselves to rotate axes to positions in which the angles they make with each other depart from 90, we are free to identify whatever factors we like. Of course, such factors will themselves be correlated: so our final psychological theory will have to explain not only the factors we have created by oblique rotation but also the correlations that obtain between them. [The correlation between two oblique factors is equivalent to the cosine of the angle between them.] It is to allow this freedom while imposing at least a certain restraint upon it that criteria are developed allowing 'oblique' rotations with their closer approximation to most naked-eye criteria of simple structure. [The most widely used set of such criteria is that of the Promax programme that can substitute for the orthogonality-maintaining rotations of Varimax. Both rotational programmes endeavour to approximate to ideal simple structure, but Promax necessarily achieves this goal more nearly.] For psychologists, there may often be a certain relief in being able to identify (oblique) factors that are singularly well defined by favourite and putatively 'pure' tests. On the other hand, major arguments frequently ensue as to whether beautiful psychometric positioning of the axes is any substitute for the experimental and biological criteria that might also seem relevant to defining the 'true' dimensions of personality and individual differences. [For example, it was because of his mistrust of simple-structure criteria for rotation that Eysenck (1971, op.cit.) - as he explains - rejected Ferguson's rotational positioning of the dimensions of social attitudes. Ferguson, Thurstone, Adorno, Altemeyer and others have preferred to work with two independent attitude dimensions of
AUTHORITARIANISM vs HUMANITARIANISM
and RELIGIONISM / MORALISM vs HEDONISM / LIBERTARIANISM.
By contrast, Eysenck urged the superiority of a large, general and seemingly familiar factor of SOCIAL CONSERVATISM vs LIBERALISM
together with a less well defined factor of
TOUGH-MINDEDNESS/REALISM vs TENDER-MINDEDNESS/IDEALISM
as a superior rotation by which to describe and 'explain' the same 2-dimensional attitude space that has been so generally found. {See Quotes XXV.}]
However, even if there is a risk of being driven up a psychometrician's blind alley by oblique rotations that initially seem so convincing, the possibility of scaling a tree and surveying the wider picture is still with us. For, since oblique factors are themselves inter-correlated, these correlations can in turn be analysed by FA to yield what are called second-order factors that account for them. Indeed, it often turns out-at least in the Eysencks' work-that second-order factors that emerge in this way are not too different from those that emerge when the search for simple structure is made within the constraints of orthogonality in the first place. [By contrast, Cattell always preferred to suppose that what he expressly called his sixteen primary (often oblique) personality factors did possess a psychological reality that no reduction of them to second-order factors-and no demonstration of their unreliability (Eysenck, 1972, Brit.J.soc.clin.Psychol.)-could dispel.] Most recently, Carroll's (op.cit.) work on human mental abilities has documented the near-universal emergence of general intelligence at the second- or third-order level of analysis even if simple structure criteria has initially broken g up amongst 'primaries'.
Today, it is easier for FA users to experiment with different rotational options for themselves-though the ready availability of Principal Components is quite enough to meet the needs of many users. The main hope of course is to be able to arrive at factorial solutions that seem theoretically convincing, though the failure of many 'educational' and 'cognitive' psychologists to acknowledge even the widespread phenomenon of g shows that this is easier said than done. {See Quotes III and Quotes VIII for discussion of 'the dimension(s)' of human personality and abilities.} Lastly, it should be mentioned that, if obliquity has been adopted, communalities for variables and eigenvalues for factors cannot any longer be calculated because the assumptions on which they depend no longer apply.
Concluding counsels
Thus it is that, after however many false starts, detours and meanderings through seemingly endless reiterative feedback loops, the three broad stages of the action in FA draw acolytes finally towards the goal. The last 'outlier' has been removed, the last 'communality' stabilized, and the last factor rotated by hand to its final artistic position. Yet prudent players of this exquisite game will not lose their heads at this point. Apart from recalling the ways in which their analyses have already failed to live up to the best possible practice, factorists will still show a lively appreciation of several remaining issues as they interpret their factors and dimensions. (The term 'dimensions' tends to be used-rather than 'factors'-when factors are singularly large, relatively independent, and somewhat more likely to emerge at a higher order of analysis.)
(a) Factorists will at least consider Scott Armstrong & Soelberg's (1968, Psychol. Bull.) suggestions as to how they might demonstrate the reliability of their results. Most dutifully, in particular, will they prepare themselves to explain why they did not divide their samples in half, complete their FA on each half separately, and then steel themselves to assess the similarity of the factors thus obtained. [Factor similarity may be estimated by treating the loadings of factors by variables just like the scores on tests by individuals. That is, one correlates the loadings of factors by variables just like one correlates the scores on tests by people. If two factors emerge as strongly correlated, this is because they are both relatively highly loaded by the same variables.] By conventional standards in FA, Eysenck and his co-workers have been exemplary in their frequent presentations of coefficients of factor similarity (CFS's). However, Vagg & Hammond (1976, Brit.J.soc.clin.Psychol.) long ago pointed out that reliability is far from being the only issue. The CFS "simply accounts for the direction of the factors and the shared items: it does not include the relative magnitude of the factors or the item loadings which are shared." Vagg & Hammond's preference was to calculate factor scores [using loadings as weightings for variables] for each subject on the factors that emerge in both the sample of which the subject was a member and in another sample of which the subject was not a member. They then correlated such factor scores, just as if their subjects had taken two tests. As a further sophistication-to guard against individual items getting 'misplaced' in the course of FA-the scores on the original items can be correlated with the factor scores that the subjects obtain on the similar factors that emerge in different samples. Such item-scale r's can then be used to make up scales representing the confactor. Such scales are composed of items that load relatively highly on both of two factors that have been 's found to be similar by the criterion of r's between factor scores. - Needless to say, little modern practice in FA exemplifies such high advices, chiefly because of the limitations imposed by the usual small samples with which psychologists work. [Not that non-factorial psychology is any better! Non-factorial psychologists typically neglect the most elementary examination of the reliability of their findings. They are often content to have come up with a measure that -merely because of its unreliability in the restricted samples used-does not make for political heresy by correlating with g.]
(b) At the same time, consideration should be given to the question of how easily a result might have emerged that the factorist would not have liked from a theoretical point of view. [Brand (1972, op.cit.) provides an explicit example of the comparison of alternative rotational solutions related to different theories.]
(c) As Guilford often stressed with regard to r's, there is seldom any serious substitute for a scattergam: so scatterplots of factor scores against each other should likewise be made to enable U-shaped relations to emerge if they exist and to check for outliers.
(d) Last, although differential psychology has doubtless benefited in some ways from its isolation from the transient schools of thought that have preoccupied experimental (lately 'cognitive') psychology, there are other criteria of psychological validity for tests and factors than those of a purely psychometric nature. Those who find themselves either tired of or over-enthusiastic about FA should therefore try to consult volumes that urge the merits of combining factor-analytic with experimental and psychogenetic understandings of human psychological differences, such as Eysenck & Eysenck (1985, op.cit.) or other volumes from the Reference List [above, pp.5,6.].
Acknowledgments: I am grateful to Boris Semeonoff and Vincent Egan for their
improving comments on an earlier version of these notes.
FINIS
(Compiled by Chris Brand, Department of Psychology, University of Edinburgh.)
For coverage of dimensions of intelligence and personality,
and of the essentials of factor-analytic method, see:
BRAND, C.R. (1996) The g Factor.
Chichester : Wiley DePublisher.
"The nature and measurement of intelligence is a political hot potato.
But Brand in this extremely readable, wide-ranging and up-to-date
book is not afraid to slaughter the shibboleths of modern "educationalists". This short book provides a great deal for thought
and debate."
Professor Adrian Furnham, University College London.
The book was first issued, in February, but then withdrawn, in April, by the 'publisher' because it was deemed to have infringed modern canons of
'political correctness.'
It received a perfectly favourable review in Nature (May 2, 1996, p. 33).
For a Summary of the book, Newsletters concerning the
de-publication affair, details of how to see the book for scholarly purposes, and others' comments and reviews,
see the Internet URL sites:
http://laboratory.psy.ed.ac.uk/DOCS/crb/internet.html
http://www.webcom.com/zurcher/thegfactor/index.html
For Chris Brand's 'Get Real About Race!'-his popular exposition of his views on race and education in the Black
hip-hop music magazine 'downlow' (Autumn, 1996)-see:
http://www.bhs.mq.edu.au/~tbates/intelligence/Brand_downlow.html
A reminder of what is available in other Sections of P, B, & S.
Summary Index for PERSONALITY, BIOLOGY
& SOCIETY
(This resource manual of quotations about individual and group differences, compiled by
Mr C. R. Brand, is kept on the Internet and in Edinburgh University Psychology Department Library.)
Pages of Introduction
3 - 11 Full Index, indicating key questions in each Section.
12 - 14 Preface. - Why quotations? - Explanations and apologies.
15 - 51 Introduction: Questions, Arguments and Agreements in the study of Personality.
-Some history, and a discussion of 'realism vs 'idealism.'
52 - 57 Introductory Quotes about the study of personality.
Sections
General problems
1 'Situational' vs 'personological' approaches to human variation.
2 'Nomothetic' vs 'idiographic', 'subjective' and relativistic approaches.
3 Personality dimensions - by factor analysis and otherwise.
4 'Superstructure' and 'infrastructure' - the 'mind/body problem'.
5 Nature vs Nurture? - Or Nature via Nurture?
6 The role of consciousness in personality and 'multiple personality'.
7 The 'folk psychology' of personality components.
Intelligence
8 The measurement of intelligence. - Does g exist?
9 The bases of intelligence. - What is the psychology of g?
10 The developmental origins of g differences. - The nature and nurture of g.
11 The importance of intelligence. - The psychotelics of g.
12 Piagetianism: Kant's last stand?
13 Cognitivism: 'The Emperor's New Mind?'
Propensities
14 Neurosis, emotion and Neuroticism.
15 Psychosis, psychopathy and Psychoticism.
16 Crime and criminality.
17 Genius and creativity.
Popular proposals - psychoanalytic, phrenological and prophylactic
18 Psychoanalysis: 'Decline and Fall of the Freudian Empire'?
19 Hemispherology: a twentieth-century phrenology?
20 Psycho-social Engineering: therapy, training or transformation?
Group differences
21 Age and ageing - especially, the role of g in 'life-span development'.
22 Psychological sex differences. - Do they exist? Must they exist?
23 Social class. - Does it matter any longer?
24 Racial and ethnic differences. - Their role in 'lifestyles' and cultural attainments.
Ideological issues
25 The psychology of politics and ideological extremism.
26 The politics of psychologists and allied co-workers.
27 Equality and Community: the 'utopian' package of political aims.
28 Freedom and Responsibility: the 'legitimist' package of political aims.
Pragmatic questions
29 Carry on differentializing?
30 Carry on psycho-testing?
Appendix: Factor analysis. - 'Garbage in, garbage out'?
==============================================================