Montana Politics

Assessing Standardized Assessments

Shares

The school district just finished a week of standardized testing for many students. I could complain about what this testing cost in funding or school time or schedule disruption, or launch a discussion about the veracity of these tests, their effect on standardizing instruction and squashing individual creativity, and their use as ammunition in the battle to discredit public education. But I’m not really in the mood for any of that – I think its more productive to look at the very concept of standardized testing itself to determine what value it still has in society.

I admit it’s taken a long time for me to get to this point, because I’m exceptionally good at taking tests. They’ve gotten me good grades, thousands of dollars in scholarships, and have also to a large extent validated my work in recent years. And there is still some solid evidence that standardized tests can be a meaningful predictor of student achievement in college – that was certainly true in my case, where my college GPA correlated much more closely with my SAT scores than with my high school GPA. But I think its worth asking if the basic qualities that define a standardized test – consistency, objectivity, and control of the testing environment – are in fact the qualities that make them increasingly inappropriate assessment tools.

First, everyone loves a consistent test, because it is the same throughout the country. But that consistency leads to predictability, and that predictability is perhaps the greatest weakness of standardized tests: the stakeholders (either schools or students) know what to expect, and so can ‘study’ for it. The millions of dollars this country pumps into test prep are clear evidence that this consistency has led to an uneven playing field – those millions have to come from somewhere, and its certainly not from the underprivileged. But worse than that, standardized testing is preparing kids for a world where the challenges they face will be clearly defined beforehand, with guidebooks available for their use several months in advance of encountering a problem. This is clearly not the world into which they are entering.

The supposed objectivity of the tests encounters the same problem: I would guess most adults (Except professional ‘Who Wants to be a Millionaire’ contestants) rarely encounter a problem where there is one definitively superior answer, presented as one of four options Life generally consists of choosing from a variety of subjective answers to a problem, weighing their strengths and weaknesses, and choosing the ‘best’ course of action for a particular goal. In general, any problem that can be presented in multiple choice format can be answered in 45 seconds on Google. Real problem solving takes a bit more finesse.

Which brings us to the final problem: a closed and sterile testing environment is perhaps the situation most dissimilar from the one kids will encounter in the world. A person with immense stores of memorized knowledge is great entertainment at a dinner party, but both academic and professional use of knowledge needs to be coupled with the ability to answer questions and prove points using the resources available to a modern person. However, it is also the case the a sterile testing environment may give too optimistic a view of the challenges of post-secondary work or education. Perhaps the best predictor of future success would be a test that consisted of a difficult but boring and unpleasant task on the left, and a bottle of whiskey and a Playstation on the right. It doesn’t really matter how much knowledge or intelligence a student has if they lack the intellectual curiosity to seek out more, the focus and drive to do so in a world designed to distract them, and the resilience to keep doing so in the face of unexpected challenges.

Obviously, not everything needed for success can be taught in school. But by having our children forever taking tests, both to judge their schools and them as students, we as a society are giving the impression that what is important for success is similar to what is seen on tests. In fact, it would seems as though these tests are at best poorly aligned with the world students will need to face if they plan on being successful in the real world, and it is high time we supplemented or replaced them with a more realistic method of assessment.

About the author

The Polish Wolf

5 Comments

  • “… it is high time we supplemented or replaced them with a more realistic method of assessment.”

    Apart from having students choose between a difficult, unpleasant, task and a bottle of Wild Turkey and Playstation, what more realistic method of assessment do you propose, and why?

    • There may well be a place for tests, but look at how successful companies choose employees – there is some written testing, but there’s also oral interviews, controlled group projects, portfolios of work, etc. Tests are used for two distinct purposes: to judge students, and to judge schools. Colleges (and the military) largely dictate testing on high school students, and it’s up to colleges to decide how important those tests are, and what other characteristics they could use. Personally, I think colleges are ducking their responsibility here: They blame high schools for not preparing kids for college, trying to explain away why so many start but don’t complete their degrees, when in fact this is pretty strong evidence that colleges are failing to select the students with the best chance of completing their degrees (as there is little incentive to do so – tuition cash is still green, whether you hand out a degree or not). Students spend an inordinate amount of time and money on test prep, which might get them into college but will do little good once they are there. It’s the prerogative of colleges to decide how they want to accept students, but it seems their current method is not working, and is wasting millions of hours and dollars of societal time into learning a skill (standardized test taking) that will not translate into academic success if it does get them accepted, when it is quite clear that retention of new students is miserably low today.

      The second use of testing is to judge schools, and here I think it is even less appropriate. It rewards those schools that focus most extensively on a limited number of skills and punishes those schools that allow their the students to concentrate more on equally beneficial but harder-to-test skills, like welding, mechanics, business management, teamwork (the fact that standardized tests by their nature require no cooperation with others alone is enough to cast doubt on the efficacy in assessing real world skills), artistic endeavors, computer science, etc. Ultimately though, a thorough knowledge of business software, auto mechanics, welding/carpentry, or a strong command of a foreign language (or two) is a better guarantee of gainful employment than what ends of getting tested.

      So, how do you replace standardized tests with something that rewards schools that actually prepare students for the world outside? You can’t do so completely if you insist on having an objective measure of school performance. But you’ll notice that when people look at the critical statistics for a college, they rarely focus on what college produces the best test takers. Instead, we already have metrics in place for judging success that can be applied to high schools: percent of students graduating, percent of students fully employed after five years, you can look at their incomes, their incarceration rate, their dependence on government aid. In other words, you can look at real-world outcomes rather than measures of acumen in abstract tasks. And I imagine you’ll find that when schools focus on those outcomes, you will see real innovation in how schools prepare students for life after high school.

  • “…the fact that standardized tests by their nature require no cooperation with others alone is enough to cast doubt on the efficacy in assessing real world skills…”

    (1) I see no reason why standardized tests that measure a student’s grasp of a subject should be expected to assess real world skills. A test measuring a student’s grasp of, say, algebra, should attempt to measure that and nothing more.

    (2) To me, it is a virtue of standardized tests that they measure an individual, not a group of individuals, or how an individual interacts with others.

  • “A test measuring a student’s grasp of, say, algebra, should attempt to measure that and nothing more.”

    I guess the question is, what is really the value of measuring a student’s grasp of Algebra? There is some, but the problem is that math tests tend to be used inappropriately to 1) asses a school’s overall quality and 2) to assess a student’s chance of succeeding in college. The basic question is whether the student body’s grasp of algebra is the best way to assess a school or a student. There are tests designed to evaluate a student’s math achievement, and they are used to place a student in math courses when they enter college. That’s all well and good.

    But it quite possible for a school to excel at producing students who are ‘proficient’ in the four or five skills really tested on a standardized test but fail miserably at training students for the life ahead of them, and vice versa. And the danger is that the incentive of standardized testing, if successful, will produce schools exactly like that, where every student, regardless of their skills and interests, is expected to learn what’s on the test. This is especially problematic when the tests themselves mirror what is expected on a test to enter a four year university – and when those tests leave out entirely many highly valuable skills that as a nation are in dire need of: mechanical thinking, teamwork and leadership qualities, foreign language proficiency, etc. When we base our assessments on standardized tests, we send a strong message that those are the skills our education system still values above others, and therein lies the problem.

  • Whether a school’s curriculum well prepares a student for success in life is a question separate from whether that school successfully teaches the curriculum. Standardized tests measure how well students have learned what they studied. As you note, it is certainly possible for a curriculum to be competently taught and well mastered by students, and for that mastery to be confirmed by standardized tests, and for the curriculum not to do a good job of preparing students for life in the real word. In that case, the problem is the curriculum, not the tests.

    I believe there should be a standardized curriculum for all schools, not just in Montana, but across the nation, so that a course in, say, chemistry, is the same in Alabama as in Montana, and that the same standardized test should be administered to all students to (1) measure how well students have mastered their subjects, (2) compare schools so that lessons can be learned from high performing schools and assistance provided to low performing schools, and (3) identify high and low performing teachers.

    My opinion is that we don’t test enough in schools, and that we should lengthen the school year to accommodate more testing.

Leave a Reply

%d bloggers like this: