Data privacy is top risk for AI according to recent FCA survey

On 21^st November 2024, the Financial Conduct Authority (FCA) released the results from Its latest survey across UK financial services on artificial intelligence (AI), which revealed that four of the top five perceived risks of AI are data-related, with data privacy and protection being at the top of the list.

For any organisations looking to leverage the power of AI, it is crucial to mitigate these types of risks.

Getting data privacy wrong can be costly…

With the growing prominence of data misuse, identity theft, scams, hacking, and ransomware, regulators are understandably concerned about how organisations handle personal information. This has resulted in large fines for firms that fail to implement the appropriate measures and processes in place to ensure compliant data privacy practices.

Although the number of daily reported data breaches have levelled out at around 335, the number of fines, and the respective value of each, continue to rise, further emphasising how regulators are increasingly clamping down harder on data privacy non-compliance.

Some of the largest technology and FTSE 100 organisations have been hit with numerous fines ranging from £20m to £1bn for non-compliance in areas such as:

• Insufficient legal basis for data processing
• Non-compliance with general data processing principles
• Insufficient fulfilment of information obligations
• Insufficient technical and organisational measures to ensure security
• Insufficient fulfilment of subjects’ data privacy rights
• Data security breach of customer personal data

What are the key data privacy challenges?

Here is an overview of some of the common data privacy pitfalls organisations need to be aware of.

1. Generative AI – rapidly accelerating adoption of foundation large language models (LLMs) brings unique risks that include increased data security risk due to 3rd party information transfer, data processing causing copyright infringement, and false ‘fact’ creation.
2. Insufficient data protection measures – organizations often lack adequate security measures like encryption and firewalls, leading to vulnerabilities in data handling.
3. Insufficient legal basis for data processing – processing personal data without consent or for unintended purposes can lead to legal repercussions.
4. Lack of transparency and accountability – not providing clear privacy notices or conducting impact assessments can harm individuals’ rights and trigger fines.
5. Failure to comply with data subject rights – denying access and erasure requests or failing to notify individuals of data breaches can result in significant penalties.
6. International data transfers – transferring data to countries with weak protection laws, without proper safeguards, increases the risk of fines.

The evolving data privacy landscape

It can be difficult keeping up with everything happening in the world of data privacy.  Organisations hoping to achieve a firm grip on the rapidly shifting environment should be aware of the following regulatory, technological, and societal factors.

• AI policy and governance – the integration of AI and ML in data processing introduces complexities regarding user privacy, prompting regulators to enforce responsible and transparent use. Recent examples include EU’s AI act.
• Enhanced consumer rights – recent privacy laws empower individuals with rights to access, correct, and delete their personal data, increasing accountability for data handlers.
• Tougher enforcement – application of fines, both in size and frequency are expected to increase as regulators focus on AI related data practices.
• Global convergence and divergence – while many privacy laws align with EU General Data Protection Regulation (GDPR) standards, adapting to variations in requirements and enforcement creates challenges for businesses operating in multiple jurisdictions.
• Cybersecurity and data breaches – as cyber threats continue to evolve, ongoing data breaches intensify the focus on robust cybersecurity strategies and incident response protocols to safeguard sensitive information.
• New business models – consumer preferences and privacy concerns are shaping new models such as paid subscriptions for ad free services, and investment in localised data servers.

Reducing risk through data governance

Organisations that are in the process of introducing new AI solutions should also take the opportunity to implement strong data governance practices. Firms can stay one step ahead by proactively putting in frameworks to mitigate data privacy risks, while embedding best practices on data usage, ownership, and protection.

Effective data governance for AI helps in building trust, reducing risks, and ensuring that AI systems are used responsibly. Well organised and secure AI models will allow systems to be efficient and well controlled.

Understanding what data needs to be protected is critical. By implementing robust data security and cybersecurity measures, organizations can protect their AI systems from threats and ensure they operate securely and ethically.

Ensuring well defined policies and standards for sharing, usage and access to sensitive data can go a long way in mitigating data privacy risks.

Make architectural decisions that mitigate the risks associated with moving data outside your controlled environment.

Ensuring that the data used for training AI models is robust as poor data quality may introduce bias or impact model outputs negatively.

How to manage data privacy risks

How we can support your privacy requirements

Get in touch if you’d like to learn more about how we can support your organisation implement robust data privacy processes – info@4-most.co.uk.

You can also learn more about each of our data privacy services here.