Critically assess the impact of the
Any information on the context of the data will be valuable to understanding the degree to which impact has taken place.
Effects of politics on health
The REF will therefore assess three aspects of research: Outputs Impact Environment Research impact is assessed in two formats, first, through an impact template that describes the approach to enabling impact within a unit of assessment, and second, using impact case studies that describe the impact taking place following excellent research within a unit of assessment REF a. This presents particular difficulties in research disciplines conducting basic research, such as pure mathematics, where the impact of research is unlikely to be foreseen. Click an approach on the left to navigate to it Beneficiary Assessment An approach that focuses on assessing the value of an intervention as perceived by the intended beneficiaries, thereby aiming to give voice to their priorities and concerns. Click an approach on the left to navigate to it Social Return on Investment SROI An participatory approach to value-for-money evaluation that identifies a broad range of social outcomes, not only the direct outcomes for the intended beneficiaries of an intervention. Differences between these two assessments include the removal of indicators of esteem and the addition of assessment of socio-economic research impact. This raises the questions of whether UK business and industry should not invest in the research that will deliver them impacts and who will fund basic research if not the government? HEFCE developed an initial methodology that was then tested through a pilot exercise. If metrics are available as impact evidence, they should, where possible, also capture any baseline or control data. The risk of relying on narratives to assess impact is that they often lack the evidence required to judge whether the research and impact are linked appropriately. It can be seen from the panel guidance produced by HEFCE to illustrate impacts and evidence that it is expected that impact and evidence will vary according to discipline REF SROI aims to provide a valuation of the broader social, environmental, and economic impacts, providing a metric that can be used for demonstration of worth. In endeavouring to assess or evaluate impact, a number of difficulties emerge and these may be specific to certain types of impact. Metrics in themselves cannot convey the full impact; however, they are often viewed as powerful and unequivocal forms of evidence. At least, this is the function which it should perform for society. Click an approach on the left to navigate to it Success Case Method The Success Case Method SCM involves identifying the most and least successful cases in a program and examining them in detail.
This is recognized as being particularly problematic within the social sciences where informing policy is a likely impact of research. Throughout history, the activities of a university have been to provide both education and research, but the fundamental purpose of a university was perhaps described in the writings of mathematician and philosopher Alfred North Whitehead The range and diversity of frameworks developed reflect the variation in purpose of evaluation including the stakeholders for whom the assessment takes place, along with the type of impact and evidence anticipated.
How to do impact assessment
Studies Buxton, Hanney and Jones into the economic gains from biomedical and health sciences determined that different methodologies provide different ways of considering economic benefits. A discussion on the benefits and drawbacks of a range of evaluation tools bibliometrics, economic rate of return, peer review, case study, logic modelling, and benchmarking can be found in the article by Grant Click an approach on the left to navigate to it Developmental Evaluation An approach designed to support ongoing learning and adaptation, through iterative, embedded evaluation. In terms of research impact, organizations and stakeholders may be interested in specific aspects of impact, dependent on their focus. Aspects of impact, such as value of Intellectual Property, are currently recorded by universities in the UK through their Higher Education Business and Community Interaction Survey return to Higher Education Statistics Agency; however, as with other public and charitable sector organizations, showcasing impact is an important part of attracting and retaining donors and support Kelly and McNicoll While looking forward, we will be able to reduce this problem in the future, identifying, capturing, and storing the evidence in such a way that it can be used in the decades to come is a difficulty that we will need to tackle. The traditional form of evaluation of university research in the UK was based on measuring academic impact and quality through a process of peer review Grant It is acknowledged in the article by Mugabushaka and Papazoglou that it will take years to fully incorporate the impacts of ERC funding.
To allow comparisons between institutions, identifying a comprehensive taxonomy of impact, and the evidence for it, that can be used universally is seen to be very valuable. In endeavouring to assess or evaluate impact, a number of difficulties emerge and these may be specific to certain types of impact.
Where quantitative data were available, for example, audience numbers or book sales, these numbers rarely reflected the degree of impact, as no context or baseline was available.
These metrics may be used in the UK to understand the benefits of research within academia and are often incorporated into the broader perspective of impact seen internationally, for example, within the Excellence in Research for Australia and using Star Metrics in the USA, in which quantitative measures are used to assess impact, for example, publications, citation, and research income.
The main risks associated with the use of standardized metrics are that The full impact will not be realized, as we focus on easily quantifiable indicators We will focus attention towards generating results that enable boxes to be ticked rather than delivering real value for money and innovative research.
In many instances, controls are not feasible as we cannot look at what impact would have occurred if a piece of research had not taken place; however, indications of the picture before and after impact are valuable and worth collecting for impact that can be predicted.
In demonstrating research impact, we can provide accountability upwards to funders and downwards to users on a project and strategic basis Kelly and McNicoll As a result, numerous and widely varying models and frameworks for assessing impact exist.
Here we address types of evidence that need to be captured to enable an overview of impact to be developed. Johnston Johnston notes that by developing relationships between researchers and industry, new research strategies can be developed.
Social factors affecting health and illness
To achieve compatible systems, a shared language is required. Two areas of research impact health and biomedical sciences and the social sciences have received particular attention in the literature by comparison with, for example, the arts. The difficulty then is how to determine what the contribution has been in the absence of adequate evidence and how we ensure that research that results in impacts that cannot be evidenced is valued and supported. Again the objective and perspective of the individuals and organizations assessing impact will be key to understanding how temporal and dissipated impact will be valued in comparison with longer-term impact. Incorporating assessment of the wider socio-economic impact began using metrics-based indicators such as Intellectual Property registered and commercial income generated Australian Research Council If knowledge exchange events could be captured, for example, electronically as they occur or automatically if flagged from an electronic calendar or a diary, then far more of these events could be recorded with relative ease. The Payback Framework systematically links research with the associated benefits Scoble et al. While defining the terminology used to understand impact and indicators will enable comparable data to be stored and shared between organizations, we would recommend that any categorization of impacts be flexible such that impacts arising from non-standard routes can be placed. The introduction of impact assessments with the requirement to collate evidence retrospectively poses difficulties because evidence, measurements, and baselines have, in many cases, not been collected and may no longer be available. The point at which assessment takes place will therefore influence the degree and significance of that impact. Consortia for Advancing Standards in Research Administration Information, for example, has put together a data dictionary with the aim of setting the standards for terminology used to describe impact and indicators that can be incorporated into systems internationally and seems to be building a certain momentum in this area. Replicated from Hughes and Martin As part of this review, we aim to explore the following questions: What are the reasons behind trying to understand and evaluate research impact? Click an approach on the left to navigate to it Innovation History A way to jointly develop an agreed narrative of how an innovation was developed, including key contributors and processes, to inform future innovation efforts. Any information on the context of the data will be valuable to understanding the degree to which impact has taken place.
based on 89 review