Securing Data for the Future: AI x FHE Overview and Thesis
We need to secure our data for the future. FHE is part of a toolkit to enable that.
By Kushagra Aryal
Image generated by Midjourney.
Introduction
Amara's Law states: "We tend to overestimate the effect of a technology in the short run and underestimate the effect in the long run."
Most of the applications and intersections of crypto and AI are overstated. First, most people do not believe in decentralization. Second, there have to be viable use cases and applications besides DePIN and the financialization of data, which is the common thesis being shared in crypto circles.
Tokenization of data and enabling decentralized computing are certainly interesting applications of crypto. However, in this memo, I will focus on a technical deep dive and use case of a single method to apply crypto tools and technologies in the AI landscape: Fully Homomorphic Encryption (FHE) applications in ML contexts, specifically those dealing with Neural Networks.
Problem Statement
AI is eating the world. The number of parameters, which are numerical values learned by ML models, is increasing exponentially, especially for industry-based applications. The cost to access these models is increasing. Stable Diffusion is estimated to have cost $8M and the Megatron-Turing NLG at $11.35M to train. Not to mention, the number of AI mishaps have grown to 260 in 2021. Many involve misuse of data, tampering, and copyright infringement.
According to a McKinsey survey in 2022, the most commonly adopted AI use cases are Service Operation Optimization (24% of respondents) and Creation of New AI-Based Products (20% of respondents). Interestingly, while mainstream media and news outlets fearmonger about AI taking over the world, only 12% of the Healthcare Systems and Pharmaceutical industry have adopted AI in 2022. While some other industries like Telecom have seen large double-digit growth in AI usage (38%), large behemoth industries that handle sensitive patient data or financial services data have not pushed ahead in adopting AI technology.
There are three reasons why large enterprises with sensitive data are avoiding processing data (especially in Fintech and Healthcare):
1) Cost to train proprietary models
2) Security issues for sensitive data
3) Cost-effective compute power
In the following memo, I will focus on how FHE is preparing to help solve the key security issues for data training. I will explore the current technical breakthroughs, areas of exploration, and the future of commercially viable use cases for FHE in relation to applications in ML and Neural Networks.
Thesis: With Fully Homomorphic Encryption (and its variants), we can unlock the accessibility and usage of both computing power and data. This area is emerging with new research, use cases, and innovations, highlighting its potential for certain applications.
What is FHE and how does FHE work?
FHE is a lattice-based cryptographic security algorithm that enables data to be encrypted via end-to-end processes. Essentially, the data can be operated on whilst remaining encrypted.
First, let's cover the different types of Homomorphic Encryption. There are three main types of Homomorphic Encryption: Partially Homomorphic Encryption (PHE), Somewhat Homomorphic Encryption (SHE), and Fully Homomorphic Encryption (FHE).
Partially Homomorphic Encryption (PHE)
Definition of Partially Homomorphic Encryption, which enables us to perform one operation (either multiplication or addition) on encrypted data. (Source)
Somewhat Homomorphic Encryption (SHE)
(E(1) * E(2)) + (E(3) + E(4))... = E(5)
SHE builds on the application of PHE by allowing more operations (up to 5) to be performed on the data. Nonetheless, you can combine multiple operations here.
Fully Homomorphic Encryption (FHE)
Source FHE combines the ability to do addition and multiplication for more arbitrary (and more secure) calculations on data, as illustrated above.
The main difference between PHE and SHE is that they are limited in the number and type of computations that can be done on the data.
PHE limits computation to either addition or multiplication on ciphertexts (encrypted texts).
SHE limits computation to a limited number of multiplications, where polynomial degrees are a low number.
Source The various layers of using a FHE and ML system from the primitive layer to the application layer. The Basic primitives and the Arithmetic layer are the core parts that enable FHE function under the hood for the Application Layer.
FHE combines both of these systems by enabling encryption of plaintext into a ciphertext via both addition and multiplication, enabling a greater degree of security and creating complex computational logic. However, as we add more operations to the encryptions, it increases the level of "noise". At a certain level, if the noise exceeds a threshold, you can't decrypt the ciphertext.
A solution to reducing the level of noise is Bootstrapping. Bootstrapping FHE means modifying the level (performing operations on the ciphertext like multiplication) and reducing noise (if the "noise" of data rises to a certain level, the cipher text fails to decrypt). This has its own set of complexities and considerations, depending on the type of scheme and computational model being used.
Here is an overview of the current timelines and progress made on FHE
Source Evolution of FHE from inception to the current state.
Going deeper, there are a few different types of schemes that can be used for FHE:
Source This is a set of schemes that have been researched and developed for FHE. It summarizes the main points and features of these schemes.
The most efficient system (in terms of throughput), is Threshold FHE (not to be confused with TFHE-rs, which is a modified version of FHE iterated on by Zama.ai). Threshold FHE combines MPC, with the node validation system requiring a set of votes, and FHE. The other commitment schemes tend to have higher latency and are less suited for enterprise use cases, especially for ML in large data contexts.
Zama.ai’s usage of the TFHE, which is the implementation of TFHE-rs (Fully Homomorphic Encryption over the Torus), is currently one of the major design shifts in the FHE space. TFHE uses a concept called Programmable Bootstrapping, which enables low latency, low cost (compute resource), and faster lookups. This means TFHE requires fewer operations to compute, and is generally more scalable than other schemes. However, this is still under research, and the Zama team is continuing to work in these areas.
Recent developments and research in the FHE & Neural Network space
There are a few interesting research papers exploring the intersection between Backpropagation and FHE, as it relates to the use of FHE in Neural Networks on various applications.
Research Design: This paper was a key part of introducing many ideas of this paper. The paper addresses how Neural Networks over Encrypted Data can be used to deploy a private ML without compromising sensitive health data. The researchers chose to explore this field by designing an experiment that used Neural Networks and Homomorphic Encryption for privacy. This paper has the strongest ties back to the book, "The Master Algorithm" by Pedro Domingos because it emphasizes the practical use of the connectionists' methodology with novel encryption technology.
Contributions: Through this study, researchers were able to find that it is indeed possible to build a secure cloud-based Neural Network using Homomorphic Encryption. It helped to inspire and pave the way for future research in this field. Below are two recent papers (last 5 years) that utilize different mathematical techniques and approaches to further validate and enhance the knowledge of this field.
Research Design: These researchers decided to use Backpropagation (BP) as it is more powerful than most traditional linear or logistic regression models. Since BP is a pivotal part of Deep Neural Networks, they decided to use it as the point of testing. They used the CKKS scheme for bootstrapping FHE. CKKS is a type of FHE scheme that enables you to continuously perform homomorphic computation, via modifying the level (multiplying the cipher text) and reducing noise (if the noise rises to a certain level, the cipher text fails to decrypt). It is a mechanism used specifically in BGV, BFV, and CKKS schemes.
Contributions: Specifically, they made two contributions to the FHE and BP space. First, a polynomial approximation technique that enables you to reduce the error of neurons of FHE ciphertexts in a neural network setting. Instead of using Taylor Polynomials, which are computationally expensive to increase error rates, and are traditionally used for the sigmoid function (linear outputs for error rates of 0 and 1), they developed a method to use Chebyshev polynomials. These tend to be more computationally efficient (especially in hardware-intensive settings) and offer better approximation ability for error rates. They discovered a positive correlation between the number of homomorphic operations (via polynomials) and the time it takes to train the model. Thus, a tradeoff has to be made between training efficiency and accuracy.
Research Context: Gentry introduced the first FHE in 2009, allowing people to perform both multiplication and addition while reducing noise through bootstrapping. Many previous papers have focused on improving bootstrapping efficiency. This paper is geared towards FHE with no computational limitations. They designed logic circuits to implement FHE computation on Convoluted Neural Networks with MINST handwritten data (a commonly used ML training set).
Results: While they were able to achieve a full classification accuracy with 6 bits of decimal precision, the computation time was on the magnitude of days. These systems, while reliable, have extremely slow latency in current stages. Nonetheless, this paper proved that privacy-preserved deep learning is possible.
Key Takeaways from Research Papers
The research area for FHE is still early. Many developments are still in the proof of concepts for ML training. Researchers are utilizing different methods to incorporate FHE into ML, such as changing the scheme type, bootstrapping methods, and mathematical designs of experiments. The current results, while highly accurate, are extremely high latency and require long time frames.
The privacy-preserving FHE industry is still in its infancy, as shown by the studies above. While in industry, there have been leaps made by Zama.ai, there are still many significant bottlenecks highlighted above.
Analysis of Alternative Cryptographic Solutions
There are a few technologies that are adjacent to FHE, such as MPC, AES, and ZK-based technologies. However, these technologies achieve drastically different objectives. While each one has served its purpose for data, I believe that using FHE for encrypted data can improve security in applications in the long term for the AI/ML industry.
AES & SHA-256
Advanced Encryption Standard (AES) and SHA-256 (Secure Hash Algorithm) are older, yet powerful forms of encrypting data. However, these traditional forms of encryption suffer from key management integrity, are vulnerable to quantum computing attacks, and have suffered attacks in the past. Simply, they are not secure or private enough to use in private ML settings, where data privacy is important.
MPC
Commonly used in private key management and custodial wallets, Multiparty Computation (MPC) allows for computations on private data without revealing inputs. MPCs use communication protocols across multiple servers to hold partial keys and validate transactions via an agreed-upon consensus. However, there are bottlenecks: DDoS attacks (single-party failure for a server), scaling costs (as you scale the number of servers, you face latency challenges), and communication issues across servers, meaning that it's still not a scalable solution for most cases.
ZK/ZK-ML Adjacent Technologies
ZK and ZK ML technologies are interesting areas of exploration. ZK, however, is not similar to FHE solutions. ZK enables scalability (via L2s) and some provable guarantees. Inherently, the data is not encrypted during the whole process. If a party wanted to perform a computation on a piece of data, they would need to know what that is. In ZK-based proofs, computations are not private data.
Note: There have been some interesting developments in zkML-based technologies that are looking to solve interesting problems. In AI/ML contexts, zkML enables provability for model correctness while hiding model weights and inputs, which offer better transparency of data and models being used.
Why do we need FHE for ML?
FHE has a myriad of use cases across Health, Identity, and Smart Contracts. Nonetheless, it will be even more important to enable FHE on Machine Learning Models.
Source. Each computational model (how data is organized and represented) serves a different purpose and there are different schemes that you should use in your given context. In the context of ML, TFHE seems to be working the best given Zama.ai's (leading FHE SDK provider), design choice.
FHE is powerful for three main reasons:
1) Training models on encrypted data across Healthcare (Sensitive patient data), financial applications (proprietary order flow, trading strategies), and smart contracts
Legal concerns are a major issue. Ensuring GDPR compliance among other regulatory data issues across the world can be difficult. By training models with private data, it removes barriers for dealing with sensitive data.
2) Attestation of identity via FHE to prevent AI Agents from manipulating data
Google's MedPalm2 limits access to models only to people with medical backgrounds
Serves as an alternative tool to protect identity with privacy systems
3) Currently, the only public cryptographic technology to be quantum resistant.
Currently, we don’t have quantum technology that can break the current status quo of encryption algorithms. However, in the future as developments in quantum technology progresses, there will be a need for highly secure data pipelines.
While FHE is currently slow (high latency and low throughput), I believe it will serve as the foundation for data infrastructure for ML. Through FHE, it helps to solve the problem of adversaries to protect data, people, and companies.
There are three levels of cryptographic guarantees that you can ensure for privacy
Tier 1: Complete strong formal privacy guarantees
Tier 1.5: Towards strong formal privacy guarantees
Tier 2: Realistic privacy guarantees
Tier 3: Weak to no formal guarantees
Source: Adapted from the 2023 Berkley RDI Summit, Shafi Goldwater Keynote
FHE helps to move towards Tier 1 of cryptographic guarantees for data applications.
Source: 2023 Berkley RDI Summit, Shafi Goldwater Keynote
The above chart shows the different regulatory implications that data can have across the Al/ML stack. There has been a rise in lawsuits, reaching 250 cases in 2021. Logically, as AI/ML proliferates across industries and sectors, we will see a rise in the number of cases dealing with these issues. It becomes even more tangible and necessary to use FHE across sensitive data because companies and people want to safeguard their data more than ever. FHE provides future-proof data ready for computation–even with the inevitable advent of quantum computing.
Constraints for FHE and ML & future needs of development
As noted above, there are a few bottlenecks that are currently preventing FHE from scaling and becoming enterprise-ready.
1) As highlighted by the technical challenges of implementations in Neural Networks, FHE requires a lot of computational power. The hardware and cloud systems are not yet good enough to scale and deploy to modern applications. Some solutions to solve this bottleneck include New computing architecture, Circuit-specific applications and libraries for FHE from Intel, and Application Specific Integrated Circuit (ASCI) physical implementation on FHE.
2) Cost reduction: Due to the expensive nature of FHE, for wider adoption, we need other tools and communities to help develop. There are communities such as
https://fhe.org/
that are helping to spur community-based learning and discussion throughout the ecosystem.
3) Adoption and awareness of use cases: FHE is still a novel technology primitive. Due to the smaller niche understanding compared to other privacy-based paradigms (i.e. zk and zk-ML, enterprise-grade technology, such as AES and SHA256), we need more open-source resources to help grasp and use these technologies for ML and crypto applications.
Companies in the space that are working on FHE-based and ML technology
1) Zama.ai
Zama is one of the pioneers of using FHE in ML applications along with applications of privacy in smart contract development and web2 applications. Their most interesting and applicable product relevant to the article is Concrete ML, which enables researchers and scientists to turn ML models into their FHE equivalent using simple and popular Python libraries like sci-kit learn and PyTorch. However, these models are limited to working only with 16-bit integers, which can lead to a loss of accuracy in the models. Concrete ML currently only supports FHE inference, meaning that models are trained on unencrypted data and then converted to FHE equivalent for encrypted inferencing.
Source: Zama's Gitbook on the application of FHE via their Concrete ML SDK for Neural Networks (Click here to read more)
Zama, while being on the cutting edge of FHE research, is still limited in the robustness and fully encrypted usage of FHE's power. These issues are on the edge of what is possible at the current state of application of FHE on ML-based data.
Similar to Zama, Sunscreen.tech is working on bringing the vision of FHE and ML to life. The team has built an FHE compiler that enables you to build on top of the technology with simple primitives like public key cryptography (key generation, encryption, and decryption). FHE is hard to build on because it's resource-intensive, requires deep expertise in cryptography and mathematics, and is not accessible for most applications. Sunscreen is researching these areas to make a fast, efficient, and usable FHE compiler. They are also working on other cool technologies like a ZK-proof compiler and decentralized storage that fall outside of the scope of this paper.
3) Fhenix
Fhenix, a collaboration between Zama and SCRT Labs, is building tool kits and services for FHE-based applications for blockchains and smart contracts. It’s an EVM chain that enables smart contracts to use FHE encryption. The project is still in development and is planning to launch in the near future. While they are not working directly with AI/ML applications, there use cases for smart contracts can potentially extend to AI applications on-chain.
4) Mind Network
Mind Network is building a decentralized data lake, enabling the storage of any type of data. They are building a patented Adaptive FHE technology that allows for end-to-end encryption and storage for private data. This is quite a technological feat because most public research shows the constraints of data, speed, and latency in FHE technology. If Mind Network can deliver on their promises, it can potentially empower a wide range of use cases across Private Blockchains, AI/ML, and data security.
5) Privasea.ai
Privasea is building between the intersection of FHE and ML with their Privasea AI network. The network offers computing resources along with ML models, such as Neural Networks, Decision Trees, Clustering Analysis, and other models. Users can upload their data, which is then encrypted with FHE. From there, they can utilize the various resources mentioned above to do data analysis and sharing.
Open source projects and major companies working on FHE and ML
Microsoft’s SealPIR (Github)
Microsoft’s research team has been helping advance the field of FHE with open source libraries like SealPIR and research papers that can be found here. Their application is focused on building out Private Information Retrieval (PIR) Systems that enable transfer of private data via the cloud.
Google’s FHE advancements (Link)
Similar to Microsoft, Google is working on building out infrastructure to make FHE more accessible for practitioners. Recently, Google launched a set of open source tools–libraries, hardware, compiler, and crypto libraries–to enhance the developer experience for applications in AI/ML, data infrastructure, and more. You can access those resources here.Google’s conviction in developing tools in FHE point to the adoption of more secure data environments in the future.
MarbleHE is a set of open source development tools geared towards building with Homomorphic Encryption, including FHE. It is a set of open source libraries that were developed by Alexander Vinad and team from Intel Labs. You can look at his talks, current research focus, and contributions to the space on his personal website.
O(1) Labs, known for developing the Mina Protocol, released a set of tools for development of Partially Homomorphic Encryption of Elgamal (an encryption scheme that was developed by Taher Elgamal in 1985). You can use TypeScript to use this scheme and build with smart contracts.
‘
Communities that are passionate about working with FHE
For all things FHE (including research papers, libraries, and meetups), you can go to
https://fhe.org/
. Many of the recent updates, enthusiasts, and practitioners of the FHE-space collaborate there.
You can also go to
https://homomorphicencryption.org/
, where the conversation centers around standardization of FHE. As can be observed throughout this paper, FHE has variations and a multitude of applications; therefore, it is important to establish standards for how the researchers and developers can continue to contribute to the rich open source ecosystem.
Conclusion
Overall, after conducting a deep dive into FHE, I am bullish on this technology. There is an inherent need for this level of security across ML-based applications and could be applied to any industry where data is valuable, needs protection, and is quantum resistant. Once we address the current constraints that exist–high latency, cost, and speed–this technology will serve as part of a design choice for the data infrastructure layer. FHE will not become the de-facto technology for private data applications because there are different trade-offs and considerations one has to make. Nonetheless, FHE will serve a purpose in helping to train Neural Networks and other ML applications with end-to-end encrypted data in the medium to long term.