Maclean’s report on Canadian universities has been criticized since its inception more than 10 years ago. Are these complaints valid, or are low-ranking universities trying to save face?
"There are a lot of people who would say it’s silly to rank universities," commented Doug Shale of University of Calgary’s Office of Institutional Analysis. "They’re disbelieving, they’re cynical, they realize it’s a game."
Shale and his team spent three years analyzing Maclean’s method and the University of Calgary’s performance, ranked 14 out of 15 in 2002. He doesn’t feel the study gives an accurate representation of the respective quality of Canadian universities.
"Getting reliable data," said Shale of improving credibility. "At least then the data would be usable."
Universities currently supply their own data, which is not audited. He also questions the weight assigned to some parameters. The most heavily weighted parameter, currently 15 per cent, is determined by a reputation survey sent out to select community members. Students are not allowed to participate. This survey alone moved McGill University from sixth to second place in 2002, while the University of Western Ontario dropped from second to seventh.
"It oversimplifies the complexity of education," explained Concordia University Sociology professor Benet Davetian in the November 1998 issue of Atlantis-A journal of social theory and critique. "It does so by failing to take into account realities that play an important part in the life of a university."
These realities include tuition, student opinion and extra curricular activities such as varsity sports. The study also ignores the relative strengths and weaknesses with different departments. The study goes further and separates schools offering Medical programs, so that Concordia is incomparable to McGill because of McGill’s medical school.
"I have difficulty accepting the legitimacy of this study," he said. "And I hesitate to condone the insidious damage it is doing to university morale."
In the past, universities such as the Universities of Montreal, Manitoba and Sherbrooke, as well as Laval University, have voiced their displeasure by abstaining. However, due to the overwhelming popularity of the study, this proved to be a costly statement. Manitoba was inferred by some to be an unaccredited institution due to its exclusion.
The study accounts for factors such as class size, budget per student, percentage of budget for student services and student awards, which all contribute to the learning environment. However, universities are graded on a curve, so when classes get larger at Dalhousie University, apparently, overcrowding doesn’t seem so bad here at the U of C.
"This study, although it includes interesting journalism, is methodologically unsound," concluded Davetian.
Many people feel biases within the study are not random but tend reflect the fact Maclean’s is a markedly East Coast publication. The university issue also led to an increase in revenue for Maclean’s. In comparison with 1991, the 2000 edition had almost twice as much advertising (99 pages compared with 52), which generated an estimated $3 million in advertising, while the same 41 pages were devoted to the universities.
For further information on the studies’ methodology and how the U of C has done in the past, visit the Office of Institutional Analysis online at www.oia.ucalgary.ca