Many who want a winning record in the game of life are skipping the shrink and hiring a life coach instead.
BY CECILIA CAPUZZI SIMON, The Washington Post, June 10, 2003 >>
As any parent will attest, predicting the behavior of an evolving, impressionable 17-year-old is a crapshoot. Will he show up at an 8 a.m. class, or party too much? Will he stay in college in the face of personal or financial challenges, or even persevere through a difficult course?
''To stick with college shows remarkable resilience and initiative,'' says Wayne Camara, vice president of research at the College Board, which administers the SAT. ''They are different skills than cognitive abilities.'' The SAT, the admissions exam used by 80 percent of American colleges, addresses only a student's cognitive abilities, or academic aptitude. It does not tell the whole story.
Colleges roll the dice every year around this time, as applications flood admissions offices. Small elite colleges can delve the inner soul with essays, interviews and intuition, but most large universities must rely on cold hard numbers: grade point average and admissions test scores.
To better predict college readiness, Dr. Camara says, ''we need to look at noncognitive factors -- personality, temperament, flexibility, proclivity to learn, ability to adjust, to get along with a roommate, to make appropriate decisions about studying.''
To the tune of almost $2 million so far, the organization is backing two outside research projects to identify and quantify such traits, much as employers do in personnel testing and hiring, and develop a standardized test that might supplement the SAT. Two prominent psychologists head the projects: Neal Schmitt, chairman of the department of psychology at Michigan State University, and Robert J. Sternberg, a professor at Yale whose term just ended as president of the American Psychological Association (where Dr. Camara headed testing before joining the College Board in 1994). Both projects have recently completed initial phases with promising results, but the tests would be at least six years off, and how they would be used has not been determined. While cautious about the noncognitive future, Gaston Caperton, the College Board president, says the board is ''serious about this.''
''We know the value of a test of analytic ability,'' Mr. Caperton says. ''But we also recognize that there are other kinds of intelligence, and that schools would want to know more about them because they are important to students' performance, and to their careers.''
Dr. Schmitt, an expert in personnel selection, borrowed concepts from conventional employment tests that present situations and ask test takers to choose a reaction. Dr. Sternberg's test has similar elements but includes a more controversial component: cartoon captioning and short-story writing to quantify creativity.
Dr. Sternberg says the goal of his research, which he calls the Rainbow Project, is twofold: to increase a college's chance of identifying students who will profit from its environment and to counteract the pronounced gap on SAT scores of white and non-Asian minority applicants. His test challenges traditional ideas about intelligence. In his view, there are three separate intelligences: analytic, creative and practical. Analytic intelligence -- what the SAT measures -- lends itself to memorization and analyzing information; people express creative intelligence by applying knowledge in a novel way; practical intelligence reveals itself in everyday situations. Dr. Sternberg says that analytic intelligence may not be enough to excel in college or in life. A person who is strong in creative or practical skills may thrive in an academic setting even if he doesn't excel in the analytic skills tested by the SAT. That student, however, is derailed by the test.
Perhaps the most elusive aspect of Dr. Sternberg's test is his effort to measure creativity. In Phase 1, students were asked to write short stories based on a title -- say, ''The Octopus's Sneakers.'' Others dictated stories into tape recorders based on a page of loosely linked drawings. Students also wrote captions for old New Yorker cartoons.
The test was given to 973 students at 13 colleges and graded by trained readers. Dr. Sternberg says the test is twice as likely as SAT scores to predict students' first-year grades. It also narrowed the gap between minority and white scores. In Phase 2's longitudinal study, more than 5,000 students will be followed through four years of college. The University of Illinois at Urbana-Champaign, the University of Indiana at Bloomington and Yale have committed to participate.
For his study, Dr. Schmitt searched the mission statements of 35 four-year colleges looking for keywords and goals describing ideals and personality traits the colleges valued. He boiled them down to 12 ''dimensions.'' Among them: motivation to continue learning beyond the classroom, an appreciation of art and music, multicultural tolerance, ability to communicate effectively and get along with others, leadership, social responsibility, perseverance, ethics and adaptability. From those, he developed situational questions. For example: What would you do if you discovered that your roommate was writing essays for other students for a fee? There is even a question about the ''freshman 15'': What if you're already overweight and add 15 pounds? Another part of the test asks biographical questions to tease out leadership traits, community mindedness and desire to learn beyond the classroom.
The test was given to 654 freshmen at Michigan State. Dr. Schmitt found that his test, when combined with ACT and SAT scores, improved the chances of predicting freshman grades by 6 percentage points. The test outperformed the SAT and ACT in predicting absenteeism, and outperformed peer ratings and self-ratings in the 12 dimensions. Dr. Schmitt expects to administer a revised version at about a dozen campuses.
A spokesman for ACT Inc. says it is not developing a similar test and would not comment about the College Board's efforts. But Dr. Schmitt and Dr. Sternberg are not the only educators studying noncognitive measures to gauge college readiness.
William E. Sedlacek, professor of education at the University of Maryland, uses dimensions similar to Dr. Schmitt's but also measures factors like self-image, family support and how a student copes with racism.
A consortium of universities -- North Carolina State University, Appalachian State University, North Carolina A&T State University, University of North Carolina-Pembroke and the University of Maryland -- has applied for a $400,000 federal grant to help create a test based on Dr. Sedlacek's research that would ''more accurately assess university applicants'' than admission exams can do alone, says Raymond Ting, associate professor of counselor education at North Carolina State. Dr. Ting, who coordinated the grant application, says efforts to standardize psychological and personal measures are the new wave, especially as higher education's minority population grows. ''Students have to be considered more fully as individuals, not just test scores,'' he adds.
Dr. Sedlacek finds it pointless to revisit traditional tests of academic readiness. ''We focus too much on how to improve what we already have, like the SAT,'' he says. ''But that's the wrong question. We don't know how to do the verbal or quantitative testing any better than we do.''
Oregon State University is using Dr. Sedlacek's measures to assess applicants' potential, to provide direction once the student is on campus, and in making financial aid decisions. The measures are also used by the Bill and Melinda Gates Foundation and the National Action Council for Minorities in Engineering to select students for scholarships. Dundee Holt, the council's vice president for development and communications, says the council finds such measures a better ''indicator of success than the SAT for minority students'' and doesn't consider entrance exam scores. ''We've had enough students who've earned their doctorate with 900's on their SAT's to know that there are other things going on,'' Mr. Holt says.
SO which student will make it through four years of college? According to a study by the Department of Education, the most significant precollege factor in whether a student graduates is the intensity of high school curriculum. Curriculum ''reflects 41 percent of the academic resources students bring to higher education,'' the report states. Class rank and grade point average -- less reliable benchmarks because of disparities in school quality and standards nationwide -- reflect 29 percent and test scores just 30 percent.
With the SAT under fire from several quarters, including a handful of elite colleges that have recently made scores optional, it's not surprising that the College Board wants to increase its ability to assess students. Complaints about the exam are well documented: It favors whites and males, and an overemphasis on scores has turned the admissions process into the ''educational equivalent of a nuclear arms race,'' as Richard Atkinson, former president of the University of California, once put it. Dr. Atkinson's threat to drop the SAT as an application requirement jarred the College Board into revamping the exam to better reflect high school curriculum. The changes are due in 2005.
In addition, the Supreme Court ruling last June in a University of Michigan case -- which upheld a university's right to consider race in admissions but outlawed quotas -- has admissions officials scrambling for ways to identify promising minority students.
Educators have a new buzzword to describe the admissions process: ''holistic.''
Jerome Lucido, vice provost for enrollment and management at the University of North Carolina at Chapel Hill, says his first thought when hearing a presentation on Dr. Sternberg's test was: ''That's exactly the information we're trying to extrapolate when we read an applicant's portfolio.''
At competitive schools, academic achievement is a given in any application, making entrance exam scores almost beside the point in admissions decisions. ''This generation of students is so good we don't have to worry about academic performance,'' Dr. Lucido says. ''What we need to look at is whether they are good citizens; will they challenge the faculty, will they engage in research?''
MOST large public universities don't have the resources to review portfolios. Penn State, for example, received just over 51,000 applications for its freshman class of 2003. ''We're not 'reading' every application,'' says Randall Deike, its director of research for enrollment management and administration. For such universities, which rely on test scores and grade point average, a test that measures motivation, leadership and good judgment would seem just the ticket.
But the tests face hurdles not only in design but also in institutional and public acceptance. ''I'm a great admirer of their goals,'' says Paul Sackett, professor of psychology at the University of Minnesota and a member of the College Board research advisory committee. ''The real question is: Will they be able to develop a test that holds up under scrutiny?'' Foremost, as Dr. Sackett points out, is controlling for fakery. You can't bluff a math problem. But looking at sample questions, it's not hard to figure out socially acceptable responses. While fabricators may be flagged by tricks embedded in the tests -- asking follow-up questions or multiple questions designed to illicit the same response -- such tests are not prep-proof. Coaching can teach how to write a creative caption or answer questions in a way that reflects desirable qualities.
Educators say they are intrigued by the research but have reservations about using such measures competitively. Dr. Lucido expresses concern about test questions that could eliminate students ''because of some disability that's mental in nature.'' Patte Barth, senior associate with the Education Trust, an advocacy group for poor schoolchildren, calls the whole thing ''squirrelly.'' Even in the SAT reasoning test, Ms. Barth says, there is room for interpretation. ''You take something even more difficult to measure, and I don't think they could hone this accurately for any meaning.''