The recent reform Gelmini "has placed, at least for the moment, the university system at the center of media attention. Commenters profession by the columns of the major newspapers have long launched the attack of the university system, with a view to reform and introduce merit in the management mechanisms.
Criticizing is very easy to criticize an institution and decrepit, inefficient, and in many theater-documented cases of malpractice and wrongdoing on the part of so-called "barons", is particularly easy. But when it comes time to to put on the table concrete proposals for reform, and the chaos lord it seem amateurish. While everyone agrees on the need for an abstract concept of meritocracy, then move on to the measures to be implemented in practice is another matter.
One proposal that is on the table for some time now firmly in the desires of the ministry, is to link the salary of each teacher its "productivity". They talk about it, for instance, Checchi and Jappelli colleagues on Lavoce.info :
should be defined once and for all the legal status of teachers, and details about course loads and periodic checks of scientific productivity, which affect the economic progression , now only based on seniority. They talk about
distinguished teachers and talk about it students (see the episode on the University of Annozero a few days ago.) Above all, this proposal can be found in Guidelines government for the university:
D.1 review the mechanism of automatic salary, which does not necessarily reward the quality of research and commitment in teaching, gradually replacing with regular assessments of the activity;
D.2 develop shared parameters of scientific qualification for access to the different roles of the teaching, even with the use where possible, internationally recognized indicators of scientific quality (impact factor, citation index): The CUN is already at work in this direction; "
All agree, then: the automatic seniority will soon be replaced by centralized periodic checks, the result of which bind the progression of salary (Incidentally, please note that these assessments would be in addition to periodic evaluations of university and departments: the first while you check the salaries of teachers, with the second one controls the amount of funds for each university).
We're sure it's a good idea? Roberto Perotti thinks not:
[...] many pursue in more or less explicit, direct differentiation of wages from the center, on the basis of strictly defined parameters. But a central differentiation may not work: it is impossible to determine in advance what is a researcher in every situation and circumstance. Even the single objective minimum criterion, the bibliometric, can be a useful aid, but it would be unwise and dangerous to use mechanically as the only determinant of wages.
Roberto Perotti, University rigged, Ostriches Einaudi, 2008, p. 112
Perotti Maybe too far. Taken from his fury liberal, wants to liberalize the last stronghold Financo State University, or the centralization of salaries? After all if so many influential figures tell us that the solution is right, and is in the guidelines of the government ...
As I was taught not to trust too much about what I say, I'll try to do here, however, a small exercise in style. Suppose that the minister Gelmini call me tomorrow as a consultant to the ministry
"Joseph, find me a way to implement the guideline D1!" (D2 of the CUN is dealing with it already).
"Ok, minister, but it is a difficult and dangerous, I will draw criticism from all!"
"In return I will cover you with gold."
"So, I accept." For the good of the country, of course.
OBJECTIVE OF THE MISSIONIn lines guide, in fact, it says little. Then do the objectives and specifications I write them alone, will probably be easier to reach!
What is this reform? To increase the productivity of Italian teachers, putting a mechanism directly rewarding salary. So this will be the main yardstick of our simple rules: in the end we check whether the productivity will be increased.
How much money are we talking about? Follow this way. I would say that if the goal is to gradually replace the mechanism of seniority with a mechanism based on the evaluation (as they state guidelines), more or less we are talking about the same order of magnitude rather than give it to you automatically with the passage of time, you may submit to an evaluation.
It just goes to go up or down? If we replace the old age "as it is," agree only small steps to climb. But I do not think a great idea. In fact, if the salary increase was in proportion as it is now shooting for seniority, age play a factor, however: would reward those who do research for some time, while the young researcher after the first evaluation would still pay with a restricted even if it were a new Einstein.
Perhaps more to boost the quality would be better to tie a substantial part of the evaluation and salary will also be expected to fall next time. For example, every 3 years I rate you, if valid, give you a nice little 'extra salary for the next 3 years (for example 30% -40% more) if you are worth, nothing. After 3 years, but we start again from scratch. And of course you can have mixed systems. In addition, agrees that the salary increase is proportional to the result. You could assign each teacher a score of 1 to 10, to be used to calculate the amount of the increase.
course, the system must be "fair", that is as fair as possible. I have to avoid actually reverse the scale of values \u200b\u200bbetween two researchers, in order to avoid injustices, grievances, feuds. We're going to get their hands into the pockets of Italian teachers, after all (for some reason, this sentence seems to have already heard). I'm not saying that it should be perfect, but in short, almost.
After five minutes I call the minister: "I forgot to tell you, whatever you decide to do, must be at no cost to the country: every euro spent in the assessment will be taken by the Ordinary Financing Fund (FFO).
For some reason I expected. AN ASSESSMENT OF TEACHER
A teacher is evaluated based on three core activities:
1) Quality and quantity teaching.
2) Research.
3) organizational activities.
last point, for convenience, we will put all those activities not directly falling within paragraphs 1 and 2, such as coordination of research projects, fund raising , editing of journals, peer review , conferences, participation in decision-making bodies of the University (faculty meetings, etc.)..
All three aspects are important, of course. With regard to education, some say (eg Perotti, ibid) that working in a university system does not affect the institutions assess regularly teaching, because the students themselves to assess the market. I do not know how true this is, but the issue is much debated and I will leave aside for the moment.
Point 3 is the most controversial. From personal experience we get to bitter infighting between departments rather than scientific and technical (engineering in particular) on the one hand, funds that are attracting a large amount of public and private institutions, and humanities departments still chronically short of funds other . Enter the item amount of research funds collected in the evaluation of every single teacher could lead to significant distortions, as well as to asprissime controversy. Then I will avoid taking for now consider this item.
then concentrate on research.
Research Evaluation
"There are now internationally accepted methods to measure the scientific productivity of a researcher."
This sentence is firmly on the lips of all the advocates of meritocracy (ie almost everyone). It would therefore seem a trivial task: we use these indicators blessed. Unfortunately, few of those who pronounce it they have never laid eyes on one.
Basically, we measure the number of articles published by a researcher in journals or conference proceedings, and the number of times each of these items has been cited in articles by other researchers. The sheer number of articles is a purely quantitative measure, which says nothing about the quality and the number of citations shows instead (or should indicate) indirectly "rating" that has received the item in the international scientific community. From these two measures, bibliometric indicators are constructed more or less complicated. Here are some here.
The limits of these indices are significant and well known. First, the database on which to make measurements is not unique. In science there ISI . Another is Scopus . In economics have EconLit . Unfortunately none of these good covers Information Science and Engineering. There are systems "free" as Google Scholar or Citeseer . But neither covers all human knowledge. And in all there are any errors, omissions, shady areas. If two databases cover the same field, the probability that it gives the same measure on the same research is practically nil.
In addition, scientific fields / disciplines have different rules are different: in some fields are usually publish articles with many authors, in others the number of authors is usually very high. There are fields for which they value above all the magazines (see Economics, as well as science), while in other fields have certain conferences sometimes more than a prestigious journals (see Computer Science & Engineering). In short, untangle complicated. You might think a different system for each sector. I already imagine the specification, hundreds or even thousands of pages of regulations! (Just to write it will take years of work ...)
also focusing on a specific field, there are technical difficulties, not indifferent. Suppose, for example, to evaluate a teacher every three years for the last 3 years of publications. If we started to evaluate my publications in February 2009, taking into account the period 2005-2008, almost certainly the most articles have a citation next practically zero. I know, I'm a poor researcher. But maybe also depends on the fact that if I published an article in 2008, the chance that someone will have read, I have mentioned in an article that was later accepted by a peer-review process at another journal and has been published and put into the database, it is practically nothing. Usually spend at least 5-6 months after writing an article to its publication (and in some cases well over a year). Worse, the process of reviewing have time varies depending on the field! In physics are usually fast; Engineering are usually very slow (even 2 years).
therefore need to evaluate research older. But so is reduced by the effect of "reward." Be awarded today to an article of 5-10 years before it can give pleasure, but perhaps does not work so well as a mechanism rewarding. More than anything else a bonus for his career.
also count the number of citations can be misleading. In some rare cases it happens that an article receives a huge number of citations because it contains an error that everyone mentions how to fix. It is certainly an article to be awarded. In many cases, an article receives citations "negative" by researchers who complain bitterly of the method. It 's very common in the case of articles summarize the state of the art in a field: are-quoted by everyone, but do not contain any original contribution.
In addition, some indices may be somewhat artificially manipulate. For example, between two groups by agreeing to be mentioned all the items to each other. It is common practice then (and incorrectly) in many departments that the "head" sign, however, the articles of all his subordinates, even if he has never even read. He he gets the honors (and increases in salary) like its subject, not even doing research for decades. Or, an important work could be split into multiple articles, each proposing a small increase over the previous year. With a job is out 5-6 articles a minimum.
The exclusive use of automated measuring systems is therefore not recommended. David Parnas lo dice molto francamente in un suo recente articolo ("Stop the numbers game: Counting papers slows the rate of scientific progress.", Communications of the ACM, Volume 50, Number 11 (2007), Pages 19-21, web link ):
Paper-count-based ranking schemes are often defended as "objective." They are also less time-consuming and less expensive than procedures that involve careful reading. Unfortunately, an objective measure of contribution is frequently contribution-independent.
e ancora:
The widespread practice of counting publications without reading and judging them is fundamentally flawed for a number of reasons: It encourages superficial research [...] It encourages repetition [...] It encourages small, insignificant studies [...]
and finally:
Evaluation by counting the number of published papers Corrupts Our scientists; They learn to "play the game by the rules. " [...] Those Who Want to see computer science and progress That Contribute to the society pays for it must object to rating-by-counting schemes Every Time They see one Being Applied.
In practice, if we only use automated systems for measuring, instead of encouraging the pursuit of excellence are likely to encourage poor search, or even dirty tricks to manipulate the numbers. It is certainly not what we want. Two notes before
move on. First, you are not here arguing that bibliometrics is a useless discipline. Quite the contrary: it gives us an enormous and valuable statistical support at the country level, or to compare institutions of other countries. Even in the evaluation of the curriculum of a teacher's reference lists give an initial framework, a rough estimate. This assessment must always be complemented by a careful reading of publications by experts in the field, just to avoid making blunders and to screen the wheat from the chaff. Incorrect use of the reference lists would adversely affect the long assessment.
Second note: it seems that the Ministry are aware Parnas's thought, as they have instructed the CUN to study ways of integrating bibliometric indices in the regulations for the competitions!
With indexes we went wrong. It remains the only peer-reviewed . That is the assessment by peers. For each teacher we make a commission of three experts who assess the scientific output. I can already feel, "a commission for every teacher? Composed of teachers? End that will evaluate all the others! The Italian teachers will assess each other all by themselves! What will be the most likely outcome? Self-promote all! ". I admit that the risk is very real. We may enter foreign experts ... but those do not come to the evaluators in Italy in exchange for a simple "thank you". We must pay them. And who assures us that these foreigners are good evaluators? Say that you need to call the best? It starts to become too expensive.
And then the assessment is not uniform and objective: there are good evaluators tend to give more good grades and carcass evaluators that also discard Shannon (really happened!). Fairness should be to hell. Might as well throw of the dice.
So how is it done? Perhaps he was right Perotti?
EPILOGUE"Dear Minister. The only thing that can work is that each teacher is assessed by his own university. If the University receives funds mainly based on scientific assessment of research, and if every university has freedom to determine the level of remuneration of its teachers, the best teachers will of course where there is more chance of seeing recognition of their professionalism, course and receive a higher salary. The University will decide whether to evaluate the capabilities of most fund-raising, or the ability to create a group, or the scientific output. The system regulates itself. "
" Dear Joseph, you reason well, do not say no, but if I propose this thing eat me alive. "
" I understand. And then let the matter drop. At the bottom are the guidelines, no? There is no obligation to turn them into law. The country is in recession, in a little 'forget it all. Rather, if I were you, push forward the evaluation of universities. On that thing there, see, you can do it. "
" You're right, you probably let the matter drop. A big thank you for the excellent work done for the country and goodbye! "
" Um, excuse me, minister, there would be one thing hold gold ... "
" Oh yeah, I forgot. Let's do this, call Julie at, say I sent you. "
Mmm, I know that bad throws. It will be for next time. HOT NEWS
seems that the ministry will work hard (from Corriere.it ):
the anti-"BARONI" - Among the changes introduced in the Senate committee, the anti-barons is provided for the establishment of a register (updated annually) in the Ministry with the names of teachers and researchers, and publications. To obtain the biannual salary teachers should try to have done research and publications can be obtained. If there's no trace two years shooting salary brackets was halved and teachers can not be part of committees that take new staff. Professors and researchers who do not publish for three years are also excluded from PRIN, those of national importance in the search.
Who knows ... I think David Parnas
