Drawing on her experiences, Philippa Kirkpatrick, Chief Digital Information Officer at South East London ICB reflects on the challenges and opportunities AI presents to the NHS and shares insights into creating a practical framework for the ethical, safe, and effective implementation of AI in healthcare
As the NHS grapples with unprecedented demands, the integration of Artificial Intelligence (AI) offers a tantalising promise: a more efficient, responsive, and patient-centred system. Yet, from my perspective, we are far from ready to fully embrace the opportunities that AI offers. There are key gaps in understanding, expertise, and governance that need addressing if we are to harness AI’s full potential while safeguarding against its risks.
Beyond the buzzwords
One of the challenges we face is a lack of clarity around what constitutes AI. The ethical and safety considerations are very different when thinking about generative AI compared with advanced analytics and automation. This creates confusion and dilutes the governance processes necessary for technologies that learn and adapt over time.
I’ve encountered scenarios where AI is used to tackle bottlenecks in a workflow only to shift the problem further down the pathway. This highlights a key lesson: understanding the problem in its entirety is critical before introducing AI into the solution mix. Without this, the risk of misaligned implementations rises exponentially.
Building multidisciplinary teams to address bias and risk
Effective AI implementation requires more than technical expertise. It demands a multidisciplinary approach, bringing together operational leads, technical, communication and ethical experts, and those who will ultimately use the technology. To this end, South East London ICB has initiated a series of roundtable workshops designed to address key concerns such as bias, workforce alignment, and patient engagement.
These workshops have revealed another critical challenge: the need for ongoing monitoring and governance for AI solutions. AI’s adaptive nature means it cannot be treated as a ‘set it and forget it’ technology. Continuous evaluation is essential to ensure AI remains aligned with clinical needs and does not inadvertently embed bias into decision-making processes.
Perhaps one of the most significant barriers to AI adoption is a lack of trust. Patients and staff alike need to understand when, where, and how AI is being used. Clear communication and transparency are paramount.
We’re building on a strong foundation of community engagement in South East London and will be leveraging platforms like ‘Let’s Talk South East London’ to gather feedback and demystify AI for our communities. It’s equally important to address workforce concerns – ensuring staff see AI as an enabler rather than a threat.
Collaboration is key
AI implementation in healthcare must be a collaborative effort. Smaller providers and voluntary, community and social enterprise organisations often lack the resources to develop AI expertise internally. At South East London ICB we’re looking into establishing a centralised expert review group to support organisations across the region. This group will provide guidance, oversight, and shared resources to ensure AI projects are implemented safely and effectively.
Moreover, we’re advocating for greater collaboration across ICSs and the NHS more broadly. Sharing insights, frameworks, and even pitfalls can prevent duplicative efforts and accelerate progress. For instance, sharing best practices in areas like documentation or data governance can save valuable time and resources for organisations just beginning their AI journeys.
In South East London our priority is the development of a practical framework for AI implementation. This framework will provide clear guidance on every stage of the AI lifecycle – from ideation and proof of concept to evaluation and scaling. Crucially, it will balance the need for innovation with robust safeguards to protect patients, staff, and the wider community.
Looking ahead
I envision an NHS where AI plays a pivotal role in shifting care from hospitals to communities, improving efficiency, and enabling more proactive, prevention-focused care. For example, AI could help identify patients at risk of escalation, allowing for timely interventions that keep them out of hospital. However, achieving this vision depends on strong data platforms and integrated systems that support seamless collaboration across providers.
AI holds immense potential to transform healthcare, but its implementation must be underpinned by responsibility, collaboration and a clear understanding of both its capabilities and limitations. As we prepare to share our framework at Digital Health Rewired my hope is that these practical insights will help healthcare leaders navigate the complexities of AI and unlock its benefits for patients and providers alike.
The journey to integrating AI in healthcare is an exciting we are pleased to be leading the way on this in South East London ICB. It is not a sprint but a marathon – one that requires patience, collaboration, and an unwavering commitment to ethical and equitable care.
