RT. A Journal on Research Policy and Evaluation
Authors who publish with this journal agree to the following terms: Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
Stato giuridico, reclutamento ed evoluzione della docenza universitaria (1975-2015)
The changes in the legal status and in the recruiting procedures of Italian University professors and researchers occurred in the last forty years are summarily described. The related recruiting policies are analyzed in detail, reporting yearly data on recruitment and on the (growing) average age of access to the different roles. Data on the quantitative evolution of the academic staff and on its constant aging are presented (and discussed) on a yearly basis from the early Eighties to the year 2015. The distinct evolution of different scientific areas is analyzed. Gender effects (and in particular the “glass ceiling”) are considered. The most significant changes in the geographic distribution of Universities and of their teaching body (from the year 1939 to the present day) are described. Some general conclusions on the Italian University system and its recent evolution are drawn
Copyright (c) 2016 RT. A Journal on Research Policy and Evaluation
Le politiche italiane sull’università: un serio ostacolo sulla strada della ripresa e della crescita del paese
The Italian university, after the birth of the Republic, continued for many years to be characterized by a marked centralism, despite the wide autonomy recognized by the Art. 33 of the Italian Constitution. Since the last eighties a progressively increased autonomy has been granted, initially only statutory and then also regulatory, financial, didactic and in recruiting, which provided the universities with the tools to move quickly towards the European and international competition. However, starting from the financial crisis in 2008, all the governments have chosen to dramatically change the course by starting a path of downsizing the university system, with a gradual subtraction of financial resources into the universities and a parallel limitation of autonomy. The impact of the policies adopted has been a disaster and has pushed Italy into the last positions among the European Union countries for funding at universities and last place by number of graduates in the age range of 30-34 years. In line with this approach is also framed the law of December 30, 2010, no. 240, the so-called Gelmini Law, with which a radical reversal of the system has been settled with respect to the previous autonomy policies on the university. The substantial carelessness towards the university, when not the hostility, shown by the policy and the widespread idea that its funding is not seen as an investment, but rather as an expense and a luxury that we cannot afford, leaves little hope for the future of our country and lets us imagine more and more obscure scenarios. Political choices, whether deliberately or not, are progressively leading to the dismantling of the Italian university.
Copyright (c) 2017 RT. A Journal on Research Policy and Evaluation
Un elemento di valutazione della qualità della didattica universitaria
Da molti anni le Università italiane hanno l'obbligo di somministrare questionari sul gradimento degli insegnamenti. Non è chiaro tuttavia come utilizzare questi questionari. In questo lavoro si propone un metodo per selezionare l'enorme informazione contenuta nei suggerimenti di studentesse e studenti. L'unico obiettivo che appare utile è migliorare i servizi offerti dall'Università. Le risorse di personale a disposizione di coloro che coordinano le attività didattiche sono inadeguate ed a volta risibili. Pertanto sembra necessario concentrare l'attenzione degli organi di coordinamento sulle maggiori ciriticità. Nel nostro lavoro dimostriamo che un metodo quasi automatico permette di isolare una porzione molto limitata (il 7% circa nel caso della Facoltà di Scienze della Sapienza) di insegnamenti drammaticamente critici. L'obiettivo politico è rispondere, con chiarezza, nella massima trasparenza, alla domanda che viene da studentesse e studenti di migliorare quei particolari insegnamenti. Non si postula un'attestazione di sacralità alle opinioni espresse, ma si vuole praticare e diffondere il rispetto assoluto della richiesta di verifica, implicita in tali opinioni. Il nostro approccio suggerisce anche in che modo sarebbe utile modificare leggermente la formulazione delle domande per rendere l'analisi statistica ancora più accurata. Nel complesso, per la realtà studiata in questo lavoro emerge un gradimento molto alto, se si guarda soltanto a valori medi. Tuttavia esiste una porzione di insegnamenti per i quali l'azione di verifica puntuale delle affermazioni contenute nei questionari appare utile e doverosa. Tecnicamente, l'uso del cosiddetto "trimming method" appare particolarmente efficace per la selezione degli insegnamenti problematici. In sostanza si tratta di un metodo, molto adoperato in situazioni molto diverse, per ripulire il campione statistico dalla possibile presenza, in questo caso, di questionari che potrebbero essere stati compilati con scarsa attenzione. Nel nostro caso, per ogni insegnamento, e per ogni singola domanda posta nei questionari, eliminiamo il 50% dei questionari. Più esattamente, un 25% è composto dalle valutazioni più favorevoli e un 25% è composto dalle valutazioni più sfavorevoli. La nostra analisi mostra una grande stabilità dei risultati se si mantiene l'obiettivo della ricerca degli insegnamenti più critici.
L'università italiana in Europa: riflessioni sull'equipollenza dei titoli e sulla libertà di movimento all'interno dell'Unione
In this paper I discuss aspects of the problems arising when a recognition of qualifications for the profession of university professor acquired in foreign states members of the European Union is attempted in Italy. I show that different obstacles still exist to the achievement of the free movement of workers -a fundamental principle of the Union enshrined in article 45 of the Treaty on the functioning of the European Union. In this sense I also comment on the obligation for university professors of Italian university to reside within the municipality of their university.
La valutazione d’impatto socio-economico e culturale della ricerca nelle scienze umane e sociali: uno studio di caso in una Università del Regno Unito
The aim of this paper is to describe the impact of research on society, economy and culture in the Humanities and Social sciences. In the first part of the paper, we describe the principal characteristics of the evaluation of research impact in the United Kingdom (UK), with a special focus on the new national exercise of research evaluation: the ‘Research Excellence Framework’ (REF2014). In the second part of the paper we present a case study in a British university campus where good practices of impact evaluation have been recently introduced. The main results of this study show that impact evaluation of Social sciences and Humanities research is a today’s key challenge for the academia, especially due to the difficulty in identifying the appropriate indicators for research impact measurement. Finally, we present the main themes of the current debate about the evaluation of the impacts in different fields of research.
Recruitment of professors and researchers by Italian public Universities is limited not only by the financial constraints imposed by National laws, but also by the conversion of the financial resources into positions according to a rule esatablished a few years ago by the Ministry of Education and espressed by the so-called “Punto Organico” (P.O.). The origin of the notion of P.O. is traced back to some statistical studies performed more than a decade ago . Its subsequent application to university recruiting follows from a sequel of administrative actions taken by the Ministry but it is not based on an explicit (and formally required) legal formulation. The recent structural evolution of the università system and some changes in the legislation have further weakened the justification for such a constraint. An alternative (and legally consistent) mechanism for keeping under control the projected expenses for personnel is proposed.
Post-publication peer review in biomedical journals: overcoming obstacles and disincentives to knowledge sharing
The importance of post-publication peer review (PPPR) as a type of knowledge exchange has been emphasized by several authorities in research publishing, yet biomedical journals do not always facilitate this type of publication. Here we report our experience publishing a commentary intended to offer constructive feedback on a previously published article. We found that publishing our comment required more time and effort than foreseen, because of obstacles encountered at some journals. Using our professional experience as authors’ editors and our knowledge of publication policies as a starting point, we reflect on the probable reasons behind these obstacles, and suggest ways in which journals could make PPPR easier. In addition, we argue that PPPR should be more explicitly valued and rewarded in biomedical disciplines, and suggest how these publications could be included in research evaluations. Eliminating obstacles and disincentives to PPPR is essential in light of the key roles of post-publication analysis and commentary in drawing attention to shortcomings in published articles that were overlooked during pre-publication peer review.
Copyright (c) 2018 RT. A Journal on Research Policy and Evaluation
Supervision, mentorship and peer networks: how Estonian early career researchers get (or fail to get) support
The paper analyses issues related to supervision and support of early career researchers in Estonian academia. We use nine focus groups interviews conducted in 2015 with representatives of social sciences in order to identify early career researchers’ needs with respect to support, frustrations they may experience, and resources they may have for addressing them. Our crucial contribution is the identification of wider support networks of peers and colleagues that may compensate, partially or even fully, for failures of official supervision. On the basis of our analysis we argue that support for early career researchers should take into account the resources they already possess but also recognise the importance of wider academic culture, including funding and employment patterns, and the roles of supervisors and senior researchers in ensuring successful functioning of support networks. Through analysing the conditions for the development of early career researchers – producers of knowledge – our paper contributes to social epistemology understood as analysis of specific forms of social organisation of knowledge production.
Managing the Future Imaginary: Does ‘Post-Normal’ Science need Public Relations?
Contemporary conditions of so-called ‘post-normal’ science characterised by fundamental uncertainty and high decision stakes have been met by the call for an ‘extended peer community’ to include a full range potential stakeholders in the assessment and evaluation of future research policy (Functowicz and Ravetz, 1993; 1994). Correspondingly, the term ‘Anticipatory Governance’ (AG) has entered currency within Science and Technology Studies (STS) circles, where the phrase refers sympathetically to the fields involvement with an array of novel practices routinely carried-out in the name of increasingly public-focused, conscientious management of emerging science and technology. Existing literature in this area has typically focused on perceived benefits of social-scientist driven AG as ‘Real Time Technology Assessment’ (RTTA), rather than address how such participation — in line with STS’s contemporary post-social, object-centred, anti-normative research character — relates to a lack of institutional protection for most STS practitioners today. I argue the activities of social science researchers enrolled in AG-styled programmes appears to closely resemble those of PR professionals, and as such, in today’s knowledge economy the field could have much to gain by turning to clarify and formalise the unique cognitive-base and normative horizons befitting of a closed occupational group. I suggest an occupational restructuring in line with the ‘professional project’ (Macdonald, 1995) could bring about increased autonomy for STS practitioners, as well as purposeful direction for future research.
Policy Considerations for Random Allocation of Research Funds
There are now several proposals for introducing random elements into the process of funding allocation for research, and some initial implementation of this policy by funding bodies. The proposals have been supported on efficiency grounds, with models, including social epistemology models, showing random allocation could increase the generation of significant truths in a community of scientists when compared to funding by peer review. The models in the literature are, however, fairly abstract (by necessity). This paper introduces some of the considerations that are required to build on the modelling work towards a fully-fledged policy proposal, including issues of cost and fairness.
Invention through bricolage: epistemic engineering in scientific communities
It is widely recognised that knowledge accumulation is an important aspect of scientific communities. In this essay, drawing on a range of material from theoretical biology and behavioural science, I discuss a particular aspect of the intergenerational nature of human communities – “virtual collaboration” (Tomasello 1999) – and how it can lead to epistemic progress without any explicit intentional creativity (Henrich 2016). My aim in this paper is to make this work relevant to theorists working on the social structures of science so that these processes can be utilised and optimised in scientific communities.
Social Epistemology at Work: from Philosophical Theory to Policy Advice
In Praise of Precipitatory Governance as a (Meta-)Principle of Responsible Innovation
How accessibility influences citation counts: The case of citations to the full text articles available from ResearchGate
It is generally believed that the number of citations to an article can positively be correlated to its free online availability. In the present study, we investigated the possible impact of academic social networks on the number of citations. We chose the social web service “ResearchGate” as a case. This website acts both as a social network to connect researchers, and at the same time, as an open access repository to publish post-print version of the accepted manuscripts and final versions of open access articles. We collected the data of 1823 articles published by the authors from four different universities. By analyzing these data, we showed that although different levels of full text availability are observed for the four universities, there is always a significant positive correlation between full text availability and the citation count. Moreover, we showed that both post-print version and publisher’s version (i.e., final published version) of the archived manuscripts receive more citations than non-OA articles, and the difference in the citation counts of post-print manuscripts and publisher’s version articles is nonsignificant.
Errors and secret data in the Italian research assessment exercise. A comment to a reply
Italy adopted a performance-based system for funding universities that is centered on the results of a national research assessment exercise, realized by a governmental agency (ANVUR). ANVUR evaluated papers by using “a dual system of evaluation”, that is by informed peer review or by bibliometrics. In view of validating that system, ANVUR performed an experiment for estimating the agreement between informed review and bibliometrics. Ancaiani et al. (2015) presents the main results of the experiment. Alberto Baccini and De Nicolao (2017) documented in a letter, among other critical issues, that the statistical analysis was not realized on a random sample of articles. A reply to the letter has been published by Research Evaluation (Benedetto et al. 2017). This note highlights that in the reply there are (1) errors in data, (2) problems with “representativeness” of the sample, (3) unverifiable claims about weights used for calculating kappas, (4) undisclosed averaging procedures; (5) a statement about “same protocol in all areas” contradicted by official reports. Last but not least: the data used by the authors continue to be undisclosed. A general warning concludes: many recently published papers use data originating from Italian research assessment exercise. These data are not accessible to the scientific community and consequently these papers are not reproducible. They can be hardly considered as containing sound evidence at least until authors or ANVUR disclose the data necessary for replication.
Is partial behaviour a plausible explanation for the unavailability of the ICMJE disclosure form of an author in a BMJ journal?
This case study about the ethical behaviour in the field of scholarly publishing documents an exception on the rule for research articles in the medical journal BMJ Open that ICMJE disclosure forms of authors must be made available on request. The ICMJE, the International Committee of Medical Journal Editors, has developed these forms for the disclosure of conflicts of interest for authors of medical publications. The case refers to the form of the corresponding author of an article in BMJ Open on retraction notices (Moylan and Kowalczuk, 2016). The corresponding author is a member of the council of COPE, the Committee on Publication Ethics. I will argue that the unavailability of the form relates to personal conflicts of interest with the corresponding author about my efforts to retract a fatally flawed study on the breeding biology of the Basra Reed Warbler Acrocephalus griseldis. I describe my attempts to get the form and I will argue that its unavailability can be attributed to partial behaviour by BMJ, the publisher of BMJ Open. This study complements other sources reporting ethical issues at COPE.
The Social Science Centre, Lincoln: the theory and practice of a radical idea
The Social Science Centre, Lincoln (SSC), is a co-operative organising free higher education in the city of Lincoln, England. It was formed in 2011 by a group of academics and students in response to the massive rise in student fees, from £3000 to £9000, along with other other government policies that saw the increasing neo-liberalisation of English universities. In this essay we chart the history of the SSC and what it has been like to be a member of this co-operative; but we also want to express another aspect of the centre which we have not written about: the existence of the SSC as an intellectual idea and how the idea has spread and been developed through written publications by members of the centre and by research on the centre by other non-members: students, academics and journalists. At the end of the essay we will show the most up to date manifestation of the idea, the plans to create a co-operative university with degree awarding powers where those involved, students and academics, can make a living as part of an independent enterprise ran and owned by its members for their benefit and the benefit of their community and society.
University ranking methodologies. An interview with Ben Sowter about the Quacquarelli Symonds World University Ranking
University rankings represent a controversial issue in the debate about higher education policy. One of the best known university ranking is the Quacquarelli Symonds World University Rankings (QS), published annually since 2004 by Quacquarelli Symonds ltd, a company founded in 1990 and headquartered in London. QS provides a ranking based on a score calculated by weighting six different indicators. The 2015 edition, published in October 2015, introduced major methodological innovations and, as a consequence, many universities worldwide underwent major changes of their scores and ranks. Ben Sowter, head of division of intelligence unit of Quacquarelli Symonds, responds to 15 questions about the new QS methodology.
Science Policy for an increasingly diverging Europe
This paper argues about the need for a revisited public policy framework giving priority to knowledge and technological change across the entire Europe, by discussing new cumulative data on R&D expenditure and the qualification of human resources across Europe. It takes a wide international comparison after a decade hit by recession and economic and budgetary problems, which shows an increasing internal divergence on knowledge investments across Europe, beyond the increasing gap between Europe as a whole and North America. As a result, the paper argues that new paradigms and conditions for responsible science and innovation policy across EU require the collective action of R&D institutions and a system approach to higher education, together with new initiatives towards international cooperation across an enlarged Europe. Analysis shows that chronic backwardness in science and technology in many European peripheries, including in EU southern and eastern regions, have been significantly overcome over the last decade. Nevertheless, their growing scientific and technological capacity is now associated with an increasing vulnerability as a result of the growing international competition for qualified human resources. Additionally, the comparative analysis of levels of economic diversification and sophistication across Europe, suggests the need to insist on qualification and institutional strengthening. This should consider active public policies to attract and retain qualified human resources all over Europe, as well as considering public actions towards promoting new markets. The way in which the economic fabric may gain competitiveness and access to external markets may require enhancing the degree of internationalization of the scientific community and encouraging international knowledge and innovation networks.
Some remarks on the division of cognitive labor
Since the publication of Kitcher’s influential paper The Division of Cognitive Labor, some philosophers wondered about these two related issues: (1) which is the optimal distribution of cognitive efforts among rival methods within a scientific community?, and (2) whether and how can a community achieve such an optimal distribution? Though not committing to any specific answer to question (1), I claim that issue (2) does not depend exclusively on an invisible hand like mechanism, since both intra-scientific and extra-scientific institutions may play a major role. Finally, I examine some practical difficulties of reallocating scientists from a method to another, which leads to stress the importance of well-planned training and recruitment.
Selecting applications for funding: why random choice is better than peer review
A widely-used method of research funding is through competitive grants, where the selection of which of the applications to fund is made using anonymous peer review. The aim of the present paper is to argue that the system would work more efficiently if the selection were made by random choice rather than peer review. The peer review system has defects which have been revealed by recent criticisms, and the paper gives one such criticism due to the Nobel prize winner Sir James Black. It is then shown, in support of Sir James' position, that the use of anonymous peer review leads to a systemic bias in favour of mainstream research programmes and against minority research programmes. This in turn leads to the stifling of new ideas and of innovation. This thesis is illustrated by the example of the recent discovery of the cause of cervical cancer – a discovery which has generated substantial profits for pharmaceutical companies. It is then shown that selection by random choice eliminates this systemic bias, and consequently would encourage new ideas and innovation
Metajournals. A federalist proposal for scholarly communication and data aggregation
Abstract While the EU is building an open access infrastructure of archives (e.g., Openaire) and it is trying to implement it in the Horizon 2020 program, the gap between the tools and the human beings – researchers, citizen scientists, students, ordinary people – is still wide. The necessity to dictate open access publishing as a mandate for the EU funded research – ten years after the BOAI - is an obvious symptom of it: there is a chasm between the net and the public use of reason. To escalate the advancement and the reuse of research, we should federate the multitude of already existing open access journals in federal open overlay journals that receive their contents from the member journals and boost it with their aggregation power and their semantic web tools. The article contains both the theoretical basis and the guidelines for a project whose goals are: making open access journals visible, highly cited and powerful, by federating them into wide disciplinary overlay journals; avoiding the traps of the “authors pay” open access business model, by exploiting one of the virtue of federalism: the federate journals can remain little and affordable, if they gain visibility from the power of the federal overlay journal aggregating them; enriching the overlay journals both through semantic annotation tools and by means of open platforms dedicated to host ex post peer review and experts comments; making the selection and evaluation processes and their resulting data as much as possible public and open, to avoid the pitfalls (e.g., the serials price crisis) experienced by the closed access publishing model. It is about time to free academic publishing from its expensive walled gardens and to put to test the tools that can help us to transform it in one open forest, with one hundred flowers – and one hundred trailblazers.
Reporting—the final phase of scientific research—can and should be supported. A case for integrating language professionals into the research setting
Writing for peer-reviewed research journals is difficult and requires specialized skills and knowledge—in language, logical argumentation, data presentation, publication ethics and more. The task is especially challenging for researchers who use English as an additional language. In this discussion paper, I illustrate how research writing in non-anglophone settings can usefully be supported by three types of language professional: teachers of academic writing, authors’ editors, and academic translators. Reviewing the situation in Italy, I observe that Italian researchers have limited access to the best forms of writing support, in part due to misconceptions and complex hiring rules. Finally, and based on the higher educational trends in northern Europe, I envisage a future scenario for Italy where university-wide academic writing centers will be established, language professionals with disciplinary knowledge will become part of research institutes’ staff, and researchers will have facilitated access to the services of authors’ editors and academic translators on a per-manuscript basis. As research writing support becomes integrated into the university setting, Italian researchers’ productivity will increase and the profile of Italian reporting in the international literature will be raised.
What ranking journals has in common with astrology
This paper describes the results of a survey regarding the incorporation of societal impacts considerations into the peer review of grant proposals submitted to public science funding bodies. The survey investigated perceptions regarding the use of scientific peers to judge not only the intrinsic scientific value of proposed research, but also its instrumental value to society. Members of the scientific community have expressed – some more stridently than others – resistance to the use of such societal impact considerations. We sought to understand why. Results of the survey suggest that such resistance may be due to a lack of desire rather than a lack of confidence where judging impacts is concerned. In other words, it may be less that scientists feel unable to judge broader societal impacts and more that they are unwilling to do so.