AI x DePIN: What new opportunities will emerge from the collision of hot tracks?

Written by: Cynic, Shigeru

This article is Web3 x AI The second series of research reports, the lead article is detailed in "From parallel to intersection: Exploring the new wave of digital economy led by "Web3 and AI integration"》

随着世界迈向数字化转型的步伐不断加快,AI 和 DePIN( 去中心化物理基础设施 ) 已成为推动各行各业变革的基础性技术。AI 与 DePIN 的融合，不仅能够促进技术的快速迭代和应用广泛化，还将开启更为Safety, transparent and efficient service model, bringing profound changes to the global economy.

DePIN: Decentralization from virtual to real, the mainstay of the digital economy

DePIN is the abbreviation of Decentralized Physical Infrastructure. In a narrow sense, DePIN mainly refers to the distributed network of traditional physical infrastructure supported by distributed ledger technology, such as power network, communication network, positioning network, etc. In a broad sense, all distributed networks supported by physical devices can be called DePIN, such as storage network and computing network.

from: Messari

If Crypto has brought about a decentralized revolution in finance, then DePIN is a decentralized solution in the real economy. It can be said that PoW mining machines are a kind of DePIN. From day one, DePIN has been the core pillar of Web3.

DePIN has two of the three elements of AI: algorithm, computing power, and data.

The development of artificial intelligence is generally considered to rely on three key elements: algorithms, computing power and data. Algorithms refer to the mathematical models and program logic that drive AI systems, computing power refers to the computing resources required to execute these algorithms, and data is the basis for training and optimizing AI models.

Which of the three elements is the most important? Before chatGPT appeared, peopleXiaobai NavigationIt is usually considered to be the algorithm, otherwise academic conferences and journal papers would not be filled with fine-tuning of algorithms one after another. But when chatGPT and the large language model LLM that supports its intelligence were unveiled, people began to realize the importance of the latter two. Massive computing power is the prerequisite for the birth of the model. Data quality and diversity are crucial to building a robust and efficient AI system. In contrast, the requirements for algorithms are no longer as demanding as usual.

In the era of big models, AI has evolved from meticulous craftsmanship to powerful brick-moving, and the demand for computing power and data is increasing day by day, which DePIN can provide.TokenIncentives will be given to leverage the long-tail market, and massive consumer-grade computing power and storage will become the best nutrients for large models.

Decentralization of AI is not an option, but a necessity

Of course, some people may ask, computing power and data are all available in AWS data centers, and are better than DePIN in terms of stability and user experience, so why choose DePIN instead of centralized services?

This statement is certainly reasonable. After all, looking at the current situation, almost all large models are directly or indirectly developed by large Internet companies. Behind chatGPT is Microsoft, behind Gemini is Google, and almost every Chinese Internet giant has a large model. Why? Because only large Internet companies have enough high-quality data and computing power supported by strong financial resources. But this is wrong. People no longer want to be manipulated by Internet giants.

On the one hand, centralized AI has data privacy andSafetyOn the one hand, there are risks that AI may be subject to censorship and control; on the other hand, AI created by Internet giants will make people more dependent on it, lead to market concentration, and increase barriers to innovation.

from: https://www.gensyn.ai/

Humanity should not need a Martin Luther in the AI era. People should have the right to communicate directly with God.

DePIN from a business perspective: cost reduction and efficiency improvement are the key

Even if we put aside the debate over the values of decentralization and centralization, from a business perspective, using DePIN for AI still has its merits.

First of all, we need to clearly realize that although Internet giants have a large number of high-end graphics card resources in their hands, the combination of consumer-grade graphics cards scattered among the people can also form a very considerable computing power network, that is, the long tail effect of computing power. The idle rate of such consumer-grade graphics cards is actually very high. As long as the incentives given by DePIN exceed the electricity bill, users will be motivated to contribute computing power to the network. At the same time, all physical facilities are managed by the users themselves. The DePIN network does not need to bear the unavoidable operating costs of centralized suppliers, but only needs to focus on the protocol design itself.

For data, the DePIN network can release the availability of potential data and reduce transmission costs through edge computing and other methods. At the same time, most distributed storage networks have automatic deduplication functions, which reduces the work of cleaning AI training data.

Finally, the Crypto economics brought by DePIN enhances the system's fault tolerance space, and is expected to achieve a win-win situation for providers, consumers, and platforms.

from: UCLA

In case you don’t believe it, a recent study from UCLA shows that decentralized computing achieves 2.75 times the performance of traditional GPU clusters for the same cost. Specifically, it is 1.22 times faster and 4.83 times cheaper.

The road ahead is tough: What challenges will AIxDePIN encounter?

We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard. - John Fitzgerald Kennedy

There are still many challenges in building artificial intelligence models trustlessly using DePIN's distributed storage and distributed computing.

Work Verification

In essence, deep learning model calculations and PoW mining are both general calculations, and the bottom layer is the signal changes between gate circuits. From a macro perspective, PoW mining is a "useless calculation" that attempts to obtain a hash value with n zeros in the prefix through countless random number generation and hash function calculations; while deep learning calculations are "useful calculations" that calculate the parameter values of each layer in deep learning through forward and reverse deduction, thereby building an efficient AI model.

The fact is that "useless calculations" such as PoW mining use hash functions. It is easy to calculate the image from the original image, but it is difficult to calculate the original image from the image, so anyone can easily and quickly verify the validity of the calculation; but for the calculation of deep learning models, due to the hierarchical structure, the output of each layer is used as the input of the next layer. Therefore, verifying the validity of the calculation requires executing all the previous work, which cannot be verified simply and effectively.

from: AWS

Verification of work is critical; otherwise, the provider of the computation could simply not perform the computation at all and submit a randomly generated result.

One idea is to have different servers perform the same computing tasks, and verify the effectiveness of the work by repeating the execution and checking whether they are the same. However, most model calculations are non-deterministic, and even in the same computing environment, the same results cannot be reproduced, and similarity can only be achieved in a statistical sense. In addition, repeated calculations will lead to a rapid increase in costs, which is inconsistent with DePIN's key goal of reducing costs and increasing efficiency.

Another idea is the Optimistic mechanism, which first optimistically believes that the result is effectively calculated, and at the same time allows anyone to verify the calculation result. If an error is found, a Fraud Proof can be submitted. The protocol will fine the fraudster and reward the whistleblower.

Parallelization

As mentioned before, DePIN mainly leverages the long-tail consumer computing power market, which means that the computing power that a single device can provide is limited. For large AI models, the training time on a single device will be very long, and parallelization must be used to shorten the training time.

The main difficulty in parallelizing deep learning training lies in the dependency between previous and subsequent tasks, which makes parallelization difficult to achieve.

Currently, the parallelization of deep learning training is mainly divided into data parallelism and model parallelism.

Data parallelism means distributing data on multiple machines, with each machine storing all the parameters of a model, using local data for training, and finally aggregating the parameters of each machine. Data parallelism works well when the amount of data is large, but synchronous communication is required to aggregate the parameters.

Model parallelism is when the model size is too large to fit on a single machine, the model can be split on multiple machines, with each machine storing part of the model parameters. Forward and backward propagation require communication between different machines. Model parallelism has advantages when the model is large, but the communication overhead is high during forward and backward propagation.

The gradient information between different layers can be divided into synchronous update and asynchronous update. Synchronous update is simple and direct, but it will increase the waiting time; the asynchronous update algorithm has a short waiting time, but it will introduce stability issues.

from: Stanford University, Parallel and Distributed Deep Learning

privacy

The trend of protecting personal privacy is rising around the world, and governments around the world are strengthening their supervision of personal data privacy.SafetyAlthough AI makes extensive use of public data sets, what really distinguishes different AI models is the proprietary user data of each company.

How can we benefit from proprietary data without exposing privacy during training? How can we ensure that the parameters of the constructed AI model are not leaked?

These are the two aspects of privacy, data privacy and model privacy. Data privacy protects users, while model privacy protects the organization that builds the model. In the current situation, data privacy is much more important than model privacy.

Various solutions are trying to solve the privacy problem. Federated learning ensures data privacy by training at the source of the data, keeping the data locally, and transmitting the model parameters.Zero knowledge proofMight become a rising star.

Case analysis: What are the high-quality projects on the market?

Gensyn

Gensyn is a distributed computing network for training AI models. The network uses a layer based on PolkadotBlockchainTo verify whether the deep learning task has been executed correctly and trigger payment through commands. Founded in 2020, it disclosed a $43 million Series A financing in June 2023, led by a16z.

Gensyn uses metadata from a gradient-based optimization process to build a certificate of work performed, and is consistently executed by a multi-granular, graph-based precision protocol and cross-evaluator to allow verification work to be rerun and compared for consistency, and ultimately confirmed by the chain itself to guarantee the validity of the computation. To further strengthen the reliability of work verification, Gensyn introduces staking to create incentives.

There are four types of participants in the system: submitters, solvers, verifiers, and whistleblowers.

• Submitters are end users of the system who provide tasks to be computed and are paid for completed units of work.

• The solver is the main worker of the system, performing model training and generating proofs for the verifier to check.

• The verifier is the key to bridging the non-deterministic training process with deterministic linear computation, replicating parts of the solver proof and comparing the distance to an expected threshold.

• Whistleblowers are the last line of defense, checking the work of validators and raising challenges, and receiving rewards when the challenges are passed.

Solvers need to pledge, and whistleblowers will check the solvers’ work. If they find any malicious behavior, they will challenge them. After the challenge is passed, the solvers’ pledgedTokenThe property was confiscated and the informant was rewarded.

According to Gensyn's forecast, the solution is expected to reduce training costs to one-fifth of those of centralized providers.

from: Gensyn

FedML

FedML is a decentralized and collaborative machine learning platform for decentralized and collaborative AI anywhere and at any scale. More specifically, FedML provides an MLOps ecosystem that can train, deploy, monitor, and continuously improve machine learning models while collaborating on combined data, models, and computing resources in a privacy-preserving manner. Founded in 2022, FedML disclosed a $6 million seed round in March 2023.

FedML consists of two key components: FedML-API and FedML-core, which represent the high-level API and the low-level API respectively.

FedML-core includes two independent modules: distributed communication and model training. The communication module is responsible for the underlying communication between different workers/clients and is based on MPI; the model training module is based on PyTorch.

FedML-API is built on top of FedML-core, which makes it easy to implement new distributed algorithms by adopting a client-oriented programming interface.

The FedML team’s latest work demonstrates that using FedML Nexus AI to perform AI model inference on a consumer-grade GPU RTX 4090 is 20 times cheaper and 1.88 times faster than A100.

from: FedML

Future Outlook: DePIN brings democratization of AI

One day, when AI further develops into AGI, computing power will become the de facto universal currency, and DePIN enables this process to happen in advance.

The integration of AI and DePIN has opened up a new technological growth point and provided huge opportunities for the development of artificial intelligence. DePIN provides AI with massive distributed computing power and data, which helps to train larger models and achieve stronger intelligence. At the same time, DePIN also makes AI more open andSafety, reliable direction, and reduce dependence on a single centralized infrastructure.

展望未来,AI 和 DePIN 将不断协同发展。分布式网络将为训练超大模型提供强大基础,这些模型又将在 DePIN 的应用中发挥重要作用。在保护隐私和Safety的同时,AI 也将助力 DePIN 网络协议和算法的优化。我们期待着 AI 和 DePIN 带来更高效、更公平、更可信的数字世界。

The article comes from the Internet:AI x DePIN: What new opportunities will emerge from the collision of hot tracks?

相关推荐: MT Capital 研报：Fair Launch，Token发行方式的深刻变革

Without relying on platform coins or investors, the earliest participants have completed the project's investment, distribution, marketing and evangelism. Author: Xinwei, Ian TL;DR In the past few years,BlockchainThe industry has undergone tremendous changes, especially in the way tokens are issued. From the initial simple ICO to the complex DeFi structure, and then to today’s…