[RUME] Ratings of US High Schools
Richard Hake
rrhake at earthlink.net
Mon May 8 14:51:38 EDT 2006
If you reply to this long (18 kB) post please don't hit the reply
button unless you prune the copy of this post that may appear in your
reply down to a few relevant lines, otherwise the entire already
archived post may be needlessly resent to subscribers.
*******************************
ABSTRACT: A problem with the "Mathews Challenge Index" (number of
students taking AP or IB tests divided by number of graduating
seniors) used by "Newsweek" to rate the merit of U.S. high schools is
that AP and IB courses are not, in general, well aligned with
established learning principles, as indicated in the NRC report
"Learning and Understanding: Improving Advanced Study of Mathematics
and Science in U.S. High Schools."
*******************************
Dennis Roberts (2006), in his post of 4 May 2006 to EvalTalk and
EdStat titled "Ratings of US High Schools" wrote [bracketed by lines
"RRRRRRR. . . ."; ellipses "..." in the original; slightly edited]:
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
Each year for the past few years, Newsweek has made a big splash with
its annual rankings/ratings of high schools ... which can be found
here:
<http://www.msnbc.msn.com/id/12532668/site/newsweek/>
A Washington Post Education writer, Jay Mathews, came up with a
system that essentially does the following:
HS Rating = (# of HS students in 2005 who have taken Advanced
Placement or International Baccalaureate TESTS) / (# of graduating seniors) ...
This value is calculated ... then the ratings for 1000 HSchools are shown.
CLEARLY, the definition of a great high school is based on TWO things
and two things only:
1. How many students TAKE AP and/or IB TESTS (passing is irrelevant)
2. How many students in the senior class graduated
So, the current values are for 2005 ...
I would be interested in ANY discussion ANY of you have related to this matter
. . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . .
RRRRRRRRRRRRRRRRRRRRRRRRRRRRRR
Mathews calls his rating index the "Mathews Challenge Index" [Mathews
(2006a)] and attempts to justify its use as a gauge of high school
merit in. e.g.:
(a) two articles "Why AP Matters: Test wars: Behind the debate over
how we should judge high schools" Mathews (2006b) and "Four Steps to
High School Greatness" [Mathews (2006c)]; and
(b) the transcript of a live talk on 3 May 2006, that appears at
<http://www.msnbc.msn.com/id/12532668/site/newsweek/>
under "Web Exclusive." There Newsweek writes:
"This week, NEWSWEEK ranks America's best public high schools using a
ratio devised by Washington Post Education Reporter Jay Mathews: the
number of Advanced Placement or International Baccalaureate tests
taken by all students at a school in 2004 divided by the number of
graduating seniors. Mathews is also the author of 'What's Wrong (and
Right) With America's Best Public High Schools.'. . .[Mathews
(1999)]. . . He joined us for a Live Talk on Wednesday, May 3, to
take your questions on NEWSWEEK's list of the best schools, the
importance of AP tests, and what those not listed can do for better
marks. Read the Transcript Below."
In my opinion, one of the problems with the Mathews method of rating
high schools, at least insofar as physics is concerned, is that many
AP physics courses simply mimic traditional passive-student
introductory college courses.
Most of those subscribe to what Arnold Arons (1986) called "the
relativistic model of instruction: based on the premise that, if one
starts with an
E - N - O - R - M - O - U - S breadth of subject matter but passes
it by the student at sufficiently high velocity, the Lorentz
contraction will shorten it to the point at which it drops into the
hole which is the student mind."
Such introductory physics courses are known to be relatively
ineffective in promoting students' conceptual understanding of
physics [see e.g., the reviews by Meltzer & Heron (2005) and Wieman &
Perkins (2005)].
For a rather unfavorable critique (consistent with the above
appraisal) of AP math and science and courses see "Learning and
Understanding: Improving Advanced Study of Mathematics and Science in
U.S. High Schools" [NRC (2002)]. According to the Executive Summary:
"the report presents results of a 2-year effort by a National
Research Council (NRC) committee to examine programs for advanced
study of mathematics
and science in U.S. high schools. The committee focused on the two
most widely recognized programs in the United States, and the only
two of national scope: Advanced Placement (AP) and International
Baccalaureate (IB).
Expertise on the committee included scientist-researchers, secondary
teachers of AP and IB, science and mathematics educators working on
teacher education and issues of access and equity, cognitive
scientists, and educational administrators. Panels of experts in the
disciplines (biology, chemistry physics, and mathematics) also
advised the committee. The four panel reports provided a critical
basis for the committee's analysis and may be used independently of
this volume. . . ."
The panel experts for physics were: Robin Spital, S. James Gates,
David Hammer, Robert Hilborn, Eric Mazur, Penny Moore, and Robert
Morse.
In the section "Analysis of AP and IB Programs Based On Learning
Research," of the Executive Summary of the NRC (2002) report appears
the following [bracketed by lines "NRC-NRC-NRC-. . . "]:
NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC
1. PRINCIPLED CONCEPTUAL KNOWLEDGE - Although the AP and IB programs
espouse an emphasis on concepts and key ideas, this intention is
largely unrealized in the sciences. Excessive breadth of coverage
(especially in 1-year science programs) and insufficient emphasis on
key concepts in final assessments contribute significantly to the
problem in all science fields. Although emphasis on learning concepts
and key ideas is more evident in mathematics, further improvement is
needed, particularly in the assessments, which frequently focus on
procedural knowledge at the expense of conceptual understanding.
2. PRIOR KNOWLEDGE - Except for mathematics, these programs do not
specify clearly what prior knowledge is needed for success or help
teachers to build on what students already know or to recognize
student misconceptions. In all subjects, efforts to prepare students
properly in the years preceding advanced study are often inadequate.
Too many students, especially in physics, take a 1-year advanced
course as their first course in the discipline - an inappropriate
situation.
3. METACOGNITION - Advanced study can increase students'
metacognitive skills, but many programs and courses do not help
students develop these skills.
4. DIFFERENCES AMONG LEARNERS - AP and IB teachers who employ a
variety of pedagogical approaches are likely to reach a broader range
of learners. Using several sources of evidence of student progress
also can provide a more accurate picture of what students know
compared with any single measure, such as an examination. The single
end-of year examinations and summary scores, as found in AP, do not
adequately capture student learning.
5. MOTIVATION - Students have varied motives for enrolling in
advanced study. Designing programs that are consistent with the
findings of learning research can increase students' motivation to
succeed in advanced study, encourage them to believe in their own
potential, and increase the proportion of students who take and
succeed in the course and final examinations.
6. LEARNING COMMUNITIES - Teamwork and collaborative investigation
are especially important in advanced study. The breadth of course
content and the generally short duration of laboratory periods in
many schools may be inadequate for such activities. Better use of the
Internet and technologies for collaborative learning is needed.
7. SITUATED LEARNING - Students need opportunities to learn concepts
in a variety of contexts. The AP and IB programs currently do not
emphasize
interdisciplinary connections sufficiently or assess students'
ability to apply their knowledge in new situations or contexts.
Additionally, advanced study courses might make better use of
laboratory experiences by requiring students to plan experiments,
decide what information is important, select experimental methods,
and review results critically. These courses might also draw upon
local resources (e.g., science-related industries) to give students
experience with varied practices in mathematics and science.
Although AP AND IB PROGRAMS CURRENTLY ARE NOT WELL ALIGNED WITH LEARNING
PRINCIPLES (My CAPS), they can be revised with this research in mind.
The resulting transformations are likely to make the programs more
successful in enhancing deep conceptual learning and make them more
accessible to additional students.
NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC-NRC
For discussion of AP course and the NRC (2002) report see Inside
Higher Ed's David Epstein (2005a,b), and Hake (2005).
Richard Hake, Emeritus Professor of Physics, Indiana University
24245 Hatteras Street, Woodland Hills, CA 91367
<rrhake at earthlink.net>
<http://www.physics.indiana.edu/~hake>
<http://www.physics.indiana.edu/~sdi>
REFERENCES [Tiny URL's courtesy <http://tinyurl.com/create.php>]
Arons, A.B. 1986. "Conceptual Difficulties in Science," in
Undergraduate Education in Chemistry and Physics: Proceedings of the
Chicago Conferences on Liberal Education," No. 1, edited by M.R.
Rice. Univ. of Chicago. p. 23-32.
Epstein, D. 2005a. "The New AP," Inside Higher Ed, 26 May; online at
<http://insidehighered.com/news/2005/05/26/ap>.
Epstein, D. 2005b. "Earning the AP Name," Inside Higher Ed, 15
August; online at <http://insidehighered.com/news/2005/08/15/ap>.
Hake, R.R. 2005. "The Role of Advanced Placement Courses," online at
<http://listserv.nd.edu/cgi-bin/wa?A2=ind0508&L=pod&P=R5582&I=-3>.
Post of 15 Aug 2005 13:34:29-0700 to AERA-J, AERA-H, Math-Learn,
Phys-L, Physhare, PhysLrnR, & POD.
Heron, P.R.L. & D. Meltzer. 2005. "The future of physics education research:
Intellectual challenges and practical concerns," Am. J. Phys. 73(5):
459-462; online at
<http://www.physicseducation.net/docs/Heron-Meltzer.pdf> (56 kB).
Mathews, J. 1999. "Class Struggle: What's Wrong (and Right) with
America's Best Public High Schools." Three Rivers Press. Amazon.com
information at <http://tinyurl.com/hf22m> .
Mathews, J. 2006a. "Jay Mathews's Challenge Index," Washington Post,
May 1; online at
<http://www.washingtonpost.com/wp-srv/education/challenge/2006/challengeindex01.html>,
or more compactly at <http://tinyurl.com/jo8bc>.
Mathews, J. 2006b. "Why AP Matters: Test wars: Behind the debate over
how we should judge high schools," Washington Post, May 1; online at
<http://www.washingtonpost.com/wp-dyn/content/article/2006/05/01/AR2006050100399.html>,
or more compactly <http://tinyurl.com/ptbl4>.
Mathews, J. 2006c. "Four Steps to High School Greatness," Washington
Post, May 2; online at
<http://www.washingtonpost.com/wp-dyn/content/article/2006/05/02/AR2006050200567.html?sub=AR>,
or more compactly <http://tinyurl.com/gd24n>.
NRC. 2002. National Research Council's Committee on Programs for
Advanced Study of Mathematics And Science in American High Schools,
co-chaired by Jerry Gollub and Philip Curtis, "Learning and
Understanding: Improving Advanced Study of Mathematics and Science in
U.S. High Schools," American Academy Press; online at
<http://books.nap.edu/catalog/10129.html>. An Executive Summary is at
<http://fermat.nap.edu/execsumm_pdf/10129.pdf> (968 KB).
Roberts, D. 2006. "Ratings of US High Schools," EvalTalk/EdStat post
of 4 May 2006 21:16:10-0400 online at the EvalTalk/EdStat Archives
<http://bama.ua.edu/archives/evaltalk.html>/<http://lists.psu.edu/archives/edstat-l.html>.
One must evidently subscribe to EvalTalk/EdStat to access their
archives, but it takes only a few minutes to subscribe by following
the simple directions at
<http://bama.ua.edu/archives/evaltalk.html>
/<http://lists.psu.edu/archives/edstat-l.html>."Join or leave the
list (or change settings)" where "/" means "click on." If you're
busy, then subscribe using the "NOMAIL" option under "Miscellaneous."
Then, as a subscriber, you may access the archives and/or post
messages at any time, while receiving NO MAIL from the list!
Wieman, C. & K. Perkins. 2005. "Transforming Physics Education,"
Phys. Today 58(11): 36-41; online at
<http://www.colorado.edu/physics/EducationIssues/> / "Papers" (where
"/" means "click on").
More information about the Rume
mailing list