Are the results cross-cultural? For example, would someone who was not from an individualistic, change-focused, and esteem-oriented culture like America agree with the results? Is the Barnum effect more pronounced amongst certain types of people and are some individuals more apt to disagree with the results?
In contrast to the theory of the Barnum test, the Jung Typology posits that there are real differences between persons who are introverted and extroverted, for example, that can be measured on a personality test. Even Jung allows, however, that most individuals are not pure introverts or extroverts, but rather manifest one tendency to a slightly greater or lesser degree. The Barnum inventory, acknowledging this, specifically states that the test-taker is both extroverted and introverted at different times in his or her life. One of the sources of dissatisfaction with the Jung inventory is that most people are extroverted in some situations and introverted in others. Some people are very confident at work, but reserved at home. Others like speaking in front of people but hate having one-on-one conversations with others at a party. This acknowledgement of how contextual personality can be, and the fact that people can manifest a variety of traits, depending on the situation calls into question legitimate personality tests like the Jung Typology test -- it also explains why the vague, false readings of the Barnum test may feel more accurate than the results of a real test.
Personality tests imply that the human personality is a fixed and stable entity. But experiences tend to suggest otherwise. Who we are with can change our personality, as can where...
This is why popular personality test results should never be used in isolation to make decisions. They may be useful in some circumstances as a means of self-examination. But people should not attach too much significance to them, given the difficulty of validating the accuracy of such tests. It is tempting to feel that the tests are valid, because the descriptions are likely to resonate with at least some parts of our psyche but 'feeling' correct is not the same as actual scientific accuracy. Merely because a personality test sounds correct in terms of its results or seems scientific on the surface does not mean it has actual scientific validity. Even when tested by an objective authority, personality tests that are used in business, education, and mental health fields must be rigorously tested for their validity and reliability on large and diverse populations if they are to have any weight in an evaluative process. Personality tests should be used as only one component among many to evaluate a candidate.
References
The Barnum effect. Retrieved September 29, 2011 at http://psych.fullerton.edu/mbirnbaum/psych101/barnum_demo.htm.
Barnum effect feedback: Personality inventory. Retrieved September 29, 2011 at http://psych.fullerton.edu/mbirnbaum/psych101/LittleBig5B.htm.
Whitbourne, Susan. (2010). When it comes to personality tests, a dose of skepticism is a good thing. Psychology Today. Retrieved September 29, 2011 at http://www.psychologytoday.com/blog/fulfillment-any-age/201008/when-it-comes-personality-tests-dose-skepticism-is-good-thing
