This report describes a comparative exercise to investigate the robustness and reliability of using generative AI to help produce rapid evidence reviews, compared to those produced only using human input. Two researchers separately conducted reviews on the topic ‘how technology diffusion impacts UK growth and productivity’. Both received the same briefing and inclusion criteria, but one review was produced by ‘human-only’ means whilst the other was ‘AI-assisted’, using a mix of AI tools supplemented by manual checks and edits to the AI-written output.

The AI-assisted output was completed in 23% less time and sped up the process of analysing and synthesising studies. However, the initial draft of the AI output was also judged to be less fluent and required more revisions than the ‘human’ version. This study is effectively a case-study, and therefore the results are not generalisable, but the authors conclude that AI has the potential to enhance the process of conducting rapid evidence reviews. 

The report is accompanied by the AI-assisted rapid evidence review.