Using Flawed Tests to Make Flawed Decisions

As I feared would happen, we’re now starting to see the results of so-called common assessments of Prince Edward Island grade 3 students used to advocate:

No matter what your position might be on the tests and whether or not the school-by-school results should have been released, surely we can all agree that it’s not proper to extrapolate anything profound from how kids respond to a multiple choice test about Bobby’s Big Toe.

There may indeed be problems with our schools, but these test results aren’t a useful measure of what those problems are: they are a flawed statistical snapshot of a very thin slice of what’s “important” about education. To use them to make policy and resource decisions runs the risk of ignoring where and what the real educational challenges are.


Marian's picture
Marian on January 28, 2008 - 15:44 Permalink

I think “A Sore Toe For Bobby” is a good alternate title for that story.

Seriously though, the test is okay as long as you don’t panic when interpreting the results. But it’s been my experience that common sense seems to have totally gone out the window lately when it comes to raising/teaching children. So when people aren’t panicking about the schools they’re panicking about the children themselves or safety in the schools or whatever. Here in Ontario, I think one out of every five or six boys has attention deficit disorder or bi-polar disorder or some other diagnosis?! (Don’t get me wrong, I think there are such things as learning disabilities and real psychological problems, I just don’t think that one out of five/one out of six boys has one.) At my son’s school there is also a drill (like a fire drill) where they simulate intrusion by some ‘Columbine’ style killer! I don’t know if you have any of this stuff on PEI, but here at least, we seem prone to panic.

Make Teachers Accountable!'s picture
Make Teachers A... on January 28, 2008 - 16:37 Permalink

If not extrapolating test scores from multiple choice tests (i.e level playing field for ALL students and provided in equal settings with equal time allottments), WHAT would you suggest for honest measurement?

Surely not a ‘Red Rover Circle’ replent with subjective factors such as parental employment, whether the toast was cold on the test day in question, or some other irrelevant, ridiculous stetch.

Measurement provided in equal settings should tell us something important about teachers and students. Otherwise, we’re on on slippery slope to the educational abyss.

I wouldn’t mind seeing some tenured teachers knocked down a pay scale or two for poor standards and poor performance by students. Without some level of accountability, being a teacher is a ‘lifestyle choice’ not a higher calling…

A lot of teachers a great golfers and have the darkest tans on the east coast. But what about the ‘clients’ — society’s children.

Leo's picture
Leo on January 28, 2008 - 17:35 Permalink

I think all level governments need to take very seriously early childhood education and the work of such people as Dr. Fraser Mustard (you may have heard a regional radio documentary about this topic on Sunday, January 20). Canada is one of the few industrialized countries without a national childcare program — It is needed so we do not incur other longer term costs.

Marian's picture
Marian on January 28, 2008 - 18:08 Permalink

Mustard is very popular these days, but I think his analysis is flawed. His conclusion that those who do poorly in preschool testing also tend towards deviant behaviour in later life is contested. I think we need to get back to treating children as people, not as statistics and potential deviants.

Kevin's picture
Kevin on January 29, 2008 - 03:02 Permalink

I’m glad my kids are done school — this isn’t a good time to have a kid starting out in the system.

Ok, the tests are a blunt instrument. Very blunt — useless. But, it seems lots of people don’t see it that way and many of them are in important positions with respect to education. Is there a way to convince them, or do we just have to let them do what they’re going to do, mess things up, and let the next generation fix it up (if they’re any smarter)?

oliver's picture
oliver on January 29, 2008 - 04:07 Permalink

I think we need to get back to treating children as people”

I’m not sure what people are, but children aren’t people, and it’s not even true that “a child is a child,” in that for two children of different ages it ought not to be assumed as a rule that the same educational policy will apply, because they are rapidly maturing targets—(neuro)anatomicaly, socially, psychologically, etc.. The only thing staying the same into high school is their rights under the Constitution…or Charter or whatever. My credo would be “model person-hood for a child in parallel with treating that child like whatever animal with whatever capacities he or she has and with spotting and coaching toward your best guess of a natural next goal en route to whatever maturity they seem to be disposed toward, unless sociopathological, and then seek help. Plus be careful.” Not that I’ve ever raised one myself.

oliver's picture
oliver on January 29, 2008 - 04:10 Permalink

Of course, that’s when price is no object.

Marian's picture
Marian on January 29, 2008 - 16:15 Permalink

All of this assumes that we have some idea of what their capacities are. I’m not convinced that we do, but certain educational tools are making us overconfident and that’s bad. Also, in attempting to measure ‘capacities’ in pre-school children (which frankly I don

Andrew MacPherson's picture
Andrew MacPherson on January 29, 2008 - 22:07 Permalink

A lot of teachers a great golfers and have the darkest tans on the east coast.”

As a parent or a student I would rather have a relaxed teacher than a stressed out one any day.

Again I think were taking all of this too seriously. Unfortunately we can’t simply engineer a perfect education system into existence. There will be great teacher and mediocre teachers, good days to take a test and rotten days to take a test. As I have stated here before the sample sizes on all of these results are too small to make any conclusions.

That doesn’t mean we don’t keep working to make our education system better…