Opinions differ on usefulness Recognition and Rewards program
Still a long way to go with expected obstacles
Read more- The University , People
- 13/07/2021
Opinions differ on usefulness Recognition and Rewards program
Beginning of last week, Cursor published a summary of the interview that rector Frank Baaijens gave about the progress of the Recognition and Rewards program. After the summer holidays, the Executive Board will discuss this further with the three interfaculty committees. Baaijens also presented the six guiding principles for the further roll-out of the program, such as focusing on quality instead of quantity, assessing on more competences than just publications and obtaining grants, and promoting open science. Cursor asked for opinions about this within the university.
“Scientists do so much more than ‘just’ research. They teach, some lead their department, share their insights with the general public. But they are mainly judged on their research output. This is going to change thanks to the new Recognition and Rewards system, which is being rolled out university-wide in the Netherlands”, was stated in a box at the bottom of an interview with educational innovator Rick de Lange on the TU/e website at the beginning of February this year.
Cursor is curious what professors and Human Resource think about the new way of evaluating, the system and the workload. Ironically, several of the requested professors were unable to speak to Cursor at short notice due to a busy schedule and the aforementioned workload. Luc Brunsveld, Willem Mulder, Tom de Greef and Jaime Gómez Rivas and HR policy advisor Ineke van der Vegt managed to find a spot in the agenda and share their opinion. The fact that, at this moment, the TU/e plans are still not very concrete is an important comment to take into consideration here, as the interviewees responded to the main principles here. “The concept vision developed by the taskforce Recognition and Rewards is an important first step,” Van der Vegt says. “This now has to be translated into the criteria on which we at TU/e assess our current scientific staff and new talent still to be recruited.”
Narrative CVs
Tom de Greef, associate professor of synthetic biology at the department of Biomedical Engineering and professor at Radboud University and member of De Jonge Akademie, believes there are two discussions. “I understand that at the university a differentiation has to be made regarding profiles: some people are better at education, others excel in research. Preferably you can do both, but it's okay to focus on one. To become a professor you have to excel on at least one of those topics, which is also logical. What I do not understand is the role of NWO in this process. NWO has recently switched to narrative CVs and no longer wants to assess researchers on quantitative indicators. In the proposals, publishing innovative research in high-impact journals is no longer included in the assessment at all. These high-quality publications are so important for our international competitive position. NWO simply has to say: we want the best researchers here, and numbers are the best way to measure that, especially in the beta and biomedical sciences. After all, for most European grants it is the case that you are judged by the number of publications, their impact, the number of invitations to give lectures, etc. Isn't it strange not to do that here anymore all of a sudden? I understand that you want more options to value people, but I don't think NWO should take on such a role, that's the role that the universities should take on."
Willem Mulder, professor of precision medicine at the department of Biomedical Engineering thinks it’s simple: “There is a group that is enthusiastic about Recognition and Rewards and a group that is not. Many beta scientists are not enthusiastic about it. That says enough. It's fine that different disciplines want to define their own performance indicators, but the current one-size-fits-all restructuring comes from social sciences and forces the beta scientists to work in a non-quantitative way.” But then rector Baaijens, who’s leading the taskforce Recognition and Rewards at TU/e, he has a beta background, right? Luc Brunsveld, professor of chemical biology at the department of Biomedical Engineering: “That's politics.”
Level drops
Brunsveld agrees with De Greef. “It is fine to make someone a professor based on his or her leadership or teaching qualities. But that does not mean that that person should be allocated a research budget based on that. This is a university and our first priority must be top research, because only then we have the necessary foundation for top education. Weakening the focus on top research is disastrous for the competitive position of our country in the long term. We are forced to work together, purely for the sake of working together. And that does not mean that collaboration is not good, we do that here (at this department, ed.) regularly, but that is bottom-up and because there are two good individual groups next to each other. If you put two teams from the B-class together, that will not lead to an A-class. In fact, if you put a B-class team and a C-class team together, the level drops.”
De Greef adds: “A publication in Nature or Science is really a scientific breakthrough, no tinkering and so much better than in other magazines.” Brunsveld: “Willem and I both started a company and we really wouldn't have gotten the funding for it because we are so good at talking about our research on Twitter, but because of the science and the team behind it. That’s all that matters if you want to do real valorization. Of course you have to share it with the public, but its importance for the business of our country is heavily overestimated.”
Jaime Gómez Rivas, professor of surface photonics at the department of Applied Physics is happy with the concept of the Recognition and Rewards system. “I signed the San Francisco Declaration on Research Assessment that focuses on this topic many years ago. It says among other things that the outputs from scientific research are many and varied and that we should not just focus on the impact factor (IF) of a journal while evaluating a scientist, so I’m happy that this Recognition and Rewards system is coming and we can stop the IF non-sense. But we have to stay critical: will this metric not be replaced by another metric in this new system? And who will remove it from the evaluation criteria of the commissions?”
Van der Vegt agrees with Gómez that we need to look more broadly at the scientific output. “Recognition and Rewards focuses on a more narrative support of the CV, so that numbers of publications and your h-index become less decisive. In the assessment it is a switch from quantity to quality. This is a big change and also quite a puzzle, because not everything is objectively measurable and that makes assessment and comparison difficult.”
Impact factor harmful
“The concept of the IF of journals is damaging science terribly,” Gomez says. “I have been in many commissions and that factor was almost always mentioned. The focus on that metric takes focus away from the quality of the research. It seems to be more important to publish a bad article in a high IF journal than to publish an excellent article in a lower IF journal. And then scientists really want to publish in specific journals because of their IF , which pushes for fraud. People who have to evaluate just don’t have time to read into the actual research the candidate has conducted. We look at the impact factor of the journals they published in because that’s the simplest and fastest way, but also the worst. We need to look at the impact of the research, not the journal!”
Work pressure not a problem
De Greef, Brunsveld and Mulder do not see the workload in science as a problem. Brunsveld: "I don't have a problem with workload, but with nonsense that increases workload, such as the new HR system InSite." De Greef understands his colleague: “I used to be able to mark an exam and submit it to the education office, but now I have to enter all the results in Osiris myself.” Brunsveld: “Some here say: 'Having to work more than 40 hours is outrageous'. But nobody says that about a soccer player, or an entrepreneur. Some jobs you have to do fully or not at all.” When asked whether you can be a good scientist if you want to work a maximum of 40 hours a week, all three professors unanimously answer 'no'. De Greef: “Some people are willing to go all out and give up their free time, and those people will get more funding and publications in better magazines.”
Mulder: “I just came back from the US where the awarding of research funding for biomedical research is organized in a super competitive system by the National Institutes of Health (NIH). And yes, that is very stressful. Everything, including your own salary and job, even at tenured full professor level, depends on the research funding you gathered and the scientific output of your lab. Top institutes in the US are working very hard and that results in more groundbreaking research than we have here. I don't want to say that it is better there than in the Netherlands, but the real top universities in the US dominate Europe. Nevertheless, we have an exceptional position in Europe. Look how successful the Netherlands is, we even win Nobel prizes. And that well-functioning system now has to be broken down?! If you use this vision and policy at NOC-NSF, we will have zero medals at the next Olympics.”
I don't have a problem with workload, but with nonsense that increases workload, such as the new HR system InSite
Science becomes politics
If we look at the implementation of Recognition and Rewards within TU/e, should that program be the same for all departments or should it be implemented differently per department? “There can be differences,” Gómez believes. “For example in valorization, as not all departments are equally close to the industry. And you need to look at individual circumstances, unfortunately we don’t have time for that. A machine can do evaluation based on metrics. But a machine can’t look at the personal circumstances of each individual. The only fair evaluation is a personal one. Should we judge only by output and metrics a researcher with four young kids trying to have a balanced private-professional life and a researcher fully devoted to work, the same? Or a Dutch researcher with a strong network the same as a foreign one just entering the country without a network? If the answer is yes, then we will make a non-diverse university of nerds.”
“More individual tracks means more customization, and at the same time we want to assess everyone objectively and based on the same criteria,” says policy advisor Van der Vegt. “I can imagine that with more individual career paths we will eventually get a system in which we mainly assess someone for a promotion on the individual (performance) agreements in the Personal Development Plan, instead of on a list of generic criteria. I also expect that competences will play a greater role, but I think it is important that assistant professors, associate professors and professors think along with them on what is the best way to do this.”
Brunsveld: “You may need to diversify a bit per department or research field. And also, if we do it the same everywhere, you have to again lower your standards and go for the lowest level. Alpha and gamma problems are now being projected onto the beta sciences. Mulder: “The desired change from NWO comes from social sciences. A narrative CV is a recipe for mediocrity, so that anyone, regardless of academic background, can judge. “We are not telling them that they should stop doing those surveys and that they should look at the facts, either,” Brunsveld says.
Risk of metrics
Education is another topic in the new system, but Gómez wonders how we will measure that properly. “How will you measure being successful in education? By the student’s grades? Via student evaluations? That’s not right either as there is discrimination in those evaluations of female and foreign teachers. We are all (unconsciously) biased, it’s well known. Are we going to create more discrimination in the new system? And how will you measure valorization? By the amount of patents? Or the profit of startups? In the end you’re back at metrics and that could be a serious risk. Metrics are perverse. They do not make the system better, just the opposite, and sooner or later everybody learns how to manipulate them with negative, even dramatic, consequences: A rating system of surgeons in the United States is based on the survival rate of patients a thirty days after surgery” he cites the book The Tyranny of Metrics by Jerry Z. Muller and says. “That causes hospitals not to want to take in the old and/or questionable patients, because if they do the chances are higher for the surgery to go wrong and that will bring their rate down. That risk applies here too.”
Van der Vegt acknowledges that the focus is still on research: “In practice, my colleagues and I see that the focus of assessment committees (BACs) is predominantly on research, for the candidates themselves and the committee members alike. An example: candidates give a presentation during such a BAC and for the majority of them, more than eighty to even ninety percent of that presentation is about research, while, for example, people spend at most forty to fifty percent of the time on research. I hope that Recognition and Rewards will lead to more equal attention for 'the other activities', such as teaching, supervising PhDs and PDEngs and committee work, and that, in addition to paying attention to the 'what', there will also be more room for discussion about the 'how'.”
She adds: “Creating more space for the own, unique talent of our academic staff: that is the goal of Recognition and Rewards. This is a great challenge for the HRM department. Years ago, we took the first steps in the policy 'Excellent People attract Excellent People'. When selecting and assessing assistant professors, associate professors and professors, the committee is already asked to take a broad look at four domains: research, education, valorization and organization & management. This has now been translated by some departments in the Recognition and Rewards program as: research, education, impact and leadership. Including the team contribution here is a new element.”
Recognition and Rewards will make the university more diverse and even more attractive as an employer, but it is really a cultural shift
Need for diversity
“We tend to reward people for things we are excellent at ourselves (again that dammed unconscious bias),” Gómez says. “So we need to make those commissions more diverse in order for the evaluation to be more fair. We should also have tenure trackers and postdocs in evaluation commission, and more women and non-western staff. Young people can evaluate better on topics that will be more important in the future and where seniors are not good at, like if the person being evaluated is good at communicating science in social media for example. That is more important than before, but in commissions and evaluations this is never considered, probably because these commissions are formed only by senior people.”
“We say we want diversity,” he continues “but with the hard numbers we use to evaluate, we are creating the opposite. We only get one profile that way. If you want diversity, you need to remove the numbers. Many students in my group are extremely cooperative. They put the interest of the group before their own. They help other students or colleagues with problems and help me without the management asking. Maybe they publish a couple of articles less than others, but does this make them less good students? No, they are at least as good and they make the group stronger. Maybe we should also start evaluating the evaluators, and base a reward on the result. That will provide an extra incentive to do better and to take the time to look further than the metrics.”
Van der Vegt sees that the movement towards diversity has been started: “Ultimately, TU/e wants to stimulate and facilitate the development of everyone's unique talents. The great thing is that this movement towards more diversity in career paths is already happening; for example, the department of Applied Physics has already taken steps towards a career path focused on education. Ultimately, Recognition and Rewards will make the university more diverse and even more attractive as an employer, but it is really a cultural shift. It’s therefore a long-term process, there are still a lot of obstacles on the way and we really won't finish within a year. There are no easy solutions. The most important thing is that all those involved continue to have a dialogue with each other and that we think in terms of solutions and opportunities. If we keep taking small steps, we can make a big change.”
Discussion