Dialogue with ScaleBit: Interesting things about Web3 security audit

Interviewer: Faust, Wuyue, Geek Web3

Interviewee: Luis, ScaleBit

Editor: Faust, Jomosis

On July 1, Geek Web3 invited Web3 SafetyLuis, co-founder of audit company ScaleBit, answers questions about code auditing and Web3 SafetyDuring the period, both parties discussed code auditing and Web3 security, as well as ZK,AI, Bitcoin ecology, etc., involving issues including:

• Why ScaleBit initially chose the Web3 security direction and took root in the MOVE ecosystem;

• The business logic and customer tier division of the code audit industry;

• The difference and connection between Web3 security and Web2 security;

• What are the complexities of ZK circuit auditing and what efforts Scalebit has made to achieve this?

• Views on the Bitcoin ecosystem and Layer 2 from an operational and technical perspective;

• Chatgpt et al. AI The impact and help of such tools on the code auditing industry;

This article is a text version of the interview, about 7,000 words, during which Luis combined his own experience to provide a relatively detailed popular science on many issues in the Web3 security industry. For those who are not familiar with the Web3 security and code auditing industry, this article will be an excellent opportunity to learn about auditing agencies. It is strongly recommended to read, collect and forward.

1. Faust: The first question I want to ask is about the direction of entrepreneurship. When ScaleBit was first founded, why did you choose Web3 security as the direction?

Luis:We initially focused on Web3 security for the following reasons:

First, many of our team members came from Silicon Valley.BlockchainThe circle is early, and they hope to find long-term and stable user needs.Code auditing happens to be a very basic track that can survive for a long time.So we chose this direction, and what we hope more is to become a security company respected by the industry.

Second, we believeWeb3 security is at a very early stageHowever, in Web3, security issues are much more important than traditional Internet security, because the former is directly related to financial assets and is definitely more valuable than traditional Internet security.

Although some people now believe thatcontractAudit-related business ceiling is limited, but in fact, Web3 securitycontractIn addition to auditing, more new businesses are constantly emerging, and various new demands will continue to emerge, so we think that the Web3 security/code auditing track is still very good.

third,Background and code audits of our team members / BlockchainMore safety intersectionsWe have a lot of experience in the security industry. I was previously aBlockchainThe founding member of the security company, our chief scientist, Mr. Chen Ting, has also been doing research related to blockchain security. He is a professor with a lot of experience in the field of Web3 security. Other members also haveexchangeBackground in security, formal verification, and static analysis.

For these reasons, we finally chose the Web3 security path.

2. Faust: I heard that ScaleBit was originally called MoveBit, and later upgraded its brand to ScaleBit. Can you tell us why you chose the Move ecosystem in the first place, and why you changed the name later?

Luis: This is actually our brand upgrade. The parent brand is BitsLab. We are using all three sub-brands: MoveBit, ScaleBit, and TonBit.It’s a multi-brand strategyWe have also expanded from the Move ecosystem to more ecosystems, and our overall positioning is on the security and infrastructure of emerging ecosystems.

As for why we chose the Move ecosystem in the first place, here’s a little story: around 2022, when we started to work on security audits, we spent three months researching the most suitable niche areas. At the time, we thought,If you run a full-category security company, it will be difficult to surpass your competitors in the market, so you have to find a separate track to enter.

We discussed several directions, one was Move, and then ZK. In addition, we also considered creating a vertical audit brand, for example, focusing on GameFi or a certain type of business. In short, the strategy was to make a breakthrough in a single point. Later, we comprehensively considered various factors and chose the Move ecosystem.

At that time, we were only seven or eight people, and we were quite successful in the Move ecosystem. Of the top 20 projects in the Move ecosystem by TVL, about 80% to 90% were audited by us. We also audited important underlying chain components such as MoveVM and Aptos Framework, and discovered many underlying chain vulnerabilities.In the Move track, we have a high market share.

We are also currently accepting code auditing business in the Move ecosystem, which currently accounts for about 50% of our revenue, and auditing workload accounts for about 40% of ours. Because there are more overseas customers with code auditing needs in the Move ecosystem, the unit price of this type of business will be higher.

Currently, TonBit is mainly engaged in Ton ecosystem auditing, and ScaleBit is engaged in BTC Layer2 ecosystem, ZK and other new ecosystems. The overall positioning of our BitsLab is to focus on emerging ecosystems and those with the potential for Mass Adoption.

3. Faust: I would like to ask a question about ZK. Auditing related to ZK is very troublesome. Vitalik also said before that the circuits of systems like zkEVM are too complicated. Even with functional testing or auditing, there is no guarantee that there will be no problems with the circuits. Can you talk about this based on your own experience?

Luis: ZK-related audits are divided into many aspects, mainly circuit audits, source language audits and general computing audits. Let me talk about circuit audits first.

One of the major difficulties in circuit auditing is that circuit codes are less readable than traditional programming languages. In addition, the ecosystem related to circuit languages is very fragmented. Currently, there may be more than a dozen languages and frameworks for writing circuits.These include Circom, Halo2, Artwork, Bellman, etc. Therefore, there is currently no unified standard for circuit writing.

Obviously, it is basically impossible for any security organization to be proficient in all circuit languages at the same time. Therefore, we have selectively entered the field of ZK. At present, we have mainly done two things in the ZK track. One is to co-organize the ZK Security Capture the Flag Competition (zkCTF) with Scroll, EthStorage and Mr. Guo Yu of Anbi Lab. This competition is held approximately once a year. The main purpose of holding this competition is to cultivate more ZK security talents.

Another thing is,We have developed a vulnerability detection tool - zkScanner, which mainly uses some formal and static analysis methods to scan for vulnerabilities in ZK circuits.After the zkScanner initially scans the circuit, it will find some suspicious points, which will then be handed over to manual confirmation, which can serve as a supplement to manual auditing. Of course, this automated auditing tool cannot completely replace manual auditing, but it is still relatively effective in discovering some more hidden constraints, that is, constraint problems.

Wu Yue: Is the automated audit tool you mentioned a static detection tool like the one that detects ERC-20 Tokens?

Luis: A bit like it, but not quite right. They are similar in terms of workflow: scanning static code and giving vulnerability locations and causes. The difference is that the errors that circuits focus on are mainly divided into two categories, namely Under-Constrain and Over-Constrain; at the same time, ordinary lexical analysis has difficulty finding these errors.

Wu Yue: If we just talk about Constraint, it is a bit abstract. Can you give an example of what it is like?

Luis: I think we can look at this from the side.In essence, circuits are actually smarter thancontractThe language is more of a mathematical expression.It ultimately needs to be converted to R1CS, which can be understood as a pure polynomial expression. Therefore,Some problems that may occur in conventional programs are relatively rare in circuits.

becauseThe circuit is actually "not wrong". Each circuit needs to use the correct input and output to generate the corresponding proof. If there is an error in the circuit, it will not pass the compilation.This ensures that the result of the circuit calculation is "correct". However, being "correct" is not enough for a circuit. It needs to be "correct" in all cases, which leads to the two constraints mentioned above.

If it is Over Constraint, some inputs that meet the requirements will not be able to pass through the circuit; if it is Under Constraint, those that do not meet the requirements will be made to meet the requirements, which are both fatal problems.

Wu Yue: So these problems cannot be discovered by the compiler. They are the prerequisites that the compiler sets before designing the circuit. There are problems when it expresses them. Is that right?

Luis: Yes, because these problems are not entirely at the grammatical level, they also involve the developer's intentions and some common specifications in cryptography. Specifically, these problems often require formal tools such as SMT-Solver to discover them.

4. Faust: From a business perspective, what do you think of ZK-related auditing business?

Luis:The ZK-related audit business is worth paying attention to in the long term. We have been accumulating experience in ZK auditing. Later, we did auditing in the Bitcoin ecosystem because we found that the Bitcoin ecosystem is well integrated with ZK.The ZK Layer2 narrative of the Ethereum ecosystem has receded somewhat, and the next wave of narrative on the ZK track has not yet officially arrived, and may be related to FHE.

Of course, it is neither too early nor too late for us to officially enter the ZK audit field. It is more of a stage of long-term attention and long-term accumulation. At the business level, we continue to focus on the two things mentioned above, one is zkCTF, and the other is zkScanner, which is the ZK circuit vulnerability detection tool mentioned above.

Wu Yue: Can you briefly introduce the zkCTF event?

Luis: This is a CTF (Security Capture the Flag) related to the ZK field that we initiated. It is held once a year and we invite the industry's top security researchers and ZK researchers to participate.We will work with Scroll, EthStorage and Professor Guo Yu from Anbi Lab to set topics for the contestants. We have also received strong support from Ingonyama, zkMove, HashKey and other institutions.

We have counted the list of contestants, and they are basically from all over the world, and their level is also top-notch, including:

OpenZepplin, Offside, Salus, Amber Group, Sec3, etc., as well as some PhD students in security and ZK from Georgia Tech and Berkeley.

5. Faust: I would like to ask ScaleBit about your views on the Bitcoin ecosystem. I heard that you have audited more than 30 Bitcoin ecosystem projects before. What do you think of Bitcoin's second layer?

Luis: The more than 30 Bitcoin ecosystem projects mentioned here include projects on the second layer or within the second layer ecosystem, such as UniSat, Arch Network, Merlin Chain, RGB++, B Network, etc., as well as projects related to inscriptions and runes such as Liquidium. Many other projects are Defi protocols within the second layer ecosystem.

Regarding the second layer of Bitcoin, I agree with the previous view of Kevin He, co-founder of Bitlayer, thatThe competition in the second layer of Bitcoin will be divided into three stages: the first stage is to increase TVL, the second stage is to attract developers, and the third stage is the competition of technical routes. I think we are still at the end of the first stage, or at the beginning of attracting developers and building an ecosystem.

Faust: When you audit projects in the Bitcoin ecosystem, what aspects do you mainly focus on, or what indicators do you use to audit?

Luis: If we are talking about Bitcoin Layer 2, we divide it into several dimensions. For example, some projects need to review their scripts on the Bitcoin chain, and some need to review their contracts deployed on Bitcoin Layer 2. Some audit objects are cross-chain bridges or the bottom layer of the chain. Some Layer 2 may not use EVM. There are audit work to be done at these levels.

We mainly look at the attack surfaces in the code of these projects and see if there are any vulnerabilities from various dimensions.This is actually very complicated because Bitcoin Layer 2 is a system similar to a public chain.We will look at the points that the industry audits the public chain to examine, such as whether this project has some double-spending attacks, eclipse attacks,WitchAttacks, external dependency security, centralization issues, man-in-the-middle attacks, etc.You have to go and see them all. If I want to talk about them in detail, it will be very detailed. We can talk about this topic specifically another day.

Our ScaleBit chain auditing capabilities are at least first-class in Asia. Team members have discovered vulnerabilities in well-known public chains such as Sui, OKX Chain, GalaChain, and Nervos. Recently, they also discovered a High and a Low level vulnerability in Babylon's public audit competition.

6. Faust: Based on your experience in security audits, is the cross-chain bridge the most vulnerable place to security vulnerabilities? As far as I understand, many cross-chain bridges are actually extensions of Defi, so they are as vulnerable to attacks as Defi protocols. What do you think about this?

Luis: In terms of frequency, DeFi-related places are the easiest to lose money, but in terms of amount, the amount lost after the cross-bridge attack is the largest, and once there is a problem, it will be a big problem. Of course, when I talk about DeFi, I am more referring to the contract level. As long as there is a problem with the contract of the DeFi protocol, it can be attacked, and there are relatively few remedial measures.

As for cross-chain bridges, they are indeed the most prone to problems, because the amount of funds usually associated with cross-chain bridges is relatively large, and many of them use multi-signatures, which can be easily messed up.

7. Faust: How much help or impact do you think LLM tools like Chatgpt can have on code auditing?

Luis: It’s actually quite helpful, but it’s more auxiliary.Sometimes when auditors look at some codes, in order to quickly understand the purpose of these codes, they will use Chatgpt to parse what the code roughly does. Of course, this is only auxiliary, and ultimately it depends on people to determine many details.

Another area is writing documents and audit reports, especially writing in English. Some auditors are not very native in English, so they will ask Chatgpt to polish these documents, which is quite helpful.

However, from an auditing perspective, we are also training some specific LLMs internally, using some open source large language models for training, but this is currently only for auxiliary purposes.Although it can improve work efficiency, it cannot be completely relied on AI Auditing can only improve efficiency by about 20%.We are still a long way from a massive reduction in the number of auditors.

At present, LLM still has two obvious shortcomings. The first is the problem of missed reports, and the second is false reports. We can use LLM to do some vulnerability mining, but we must pay attention to the false report rate. If you use AI to help you find code vulnerabilities, but the false report rate is very high, it will waste your time and bring burdens.We will continue to pay attention to the progress of AI, such as whether we can achieve efficient vulnerability mining at the tool level. This area is still relatively cutting-edge and everyone is still exploring, but we have not seen any company say that it can really achieve the above effects.

Wu Yue: Regarding AI automated code auditing, do you think this will be a focus in the future? As far as I understand, AI can read code almost instantly, and it has accumulated much more experience and can exhaustively enumerate more states than humans, which is a great advantage of AI. If a security company has a deep layout in this area, trains special AI to do automated code auditing, and surpasses its competitors, what do you think of this?

Luis:We have been paying attention to this direction. I think we should look at it from two aspects:

First, if you believe that AI automated auditing is really feasible, then a situation will arise:

Theoretically, LLM can harmonize the entire code auditing industry.Because if everyone uses LLM to generate code, LLM can ensure that the code it generates isXiaobai NavigationIf there are no problems or bugs in the code, then you don’t need an audit. At this point, he not only kills the audit industry, but also programmers. But it is very difficult to achieve this effect.

If we think that LLM can get rid of auditors, it must be more difficult than getting rid of developers.Compared to simply implementing code that meets the requirements, it is much more difficult to write code without vulnerabilities, soI think it will be more difficult for AI to replace auditors than programmers.

Another aspect is thatAI does not solve security audits directly, but achieves breakthroughs in certain areas first.For example, as mentioned earlier, AI can help you discover vulnerabilities. Although it cannot help you find all the bugs, it can help you find one or two types of problems that manual audits may overlook. Application scenarios like this are exactly what we focus on.

8. Faust: I would like to ask you about your views on the audit work itself. When you do an audit, what does the specific workflow include? It is not as simple as issuing a certificate, right? In the process of reviewing the code, will you help the project party optimize the code?

Luis: This depends on customer needs. Sometimes we will help customers optimize their original code, such as making certain DeFi operations consume less gas.

As for the audit process, let me briefly introduce it.We have at least two independent audit processes: initial review and review.During the initial review, a group of people will do a separate audit, and the project party will modify the initial code; then it will enter the review, and another group of people will continue to check the code. The final effect is that there will be at least two groups of people to cross-audit the code.

Speaking of the difference between ScaleBit and other auditing companies, we are best at auditing innovative businesses. We like to recruit people with a CTF (Capture the Flag) background to do audits. They have strong learning abilities and a strong understanding of various attack methods.

In addition, we are a little different from other audit companies, that is, we prefer the boutique audit route. If the code we audited missed Major or above vulnerabilities, we will refund 30%~50% of fees. This is something other audit companies dare not promise.

9. Faust: Some people believe that security audits are actually about brand endorsement, just like Wall Street rating agencies. The Matthew effect is very strong in this area. Old companies like SlowMist have first-mover advantages and a strong moat. It is difficult for up-and-coming companies to compete with old powerhouses like SlowMist. What do you think about this?

Luis: I partially agree with this view, but it depends on different situations.We divide our clients into three types according to their audit needs, namely low, medium and high levels.The lowest level is the local dog project, and the next level is the medium-level project, which is the kind of project with certain strength but not a star team. The highest level is the kind of star team with relatively strong financial strength.

Let's start with the highest-level customers. These customers usually look for 2-3 auditing companies and are very concerned about the quality of code audits. They may first look for some of the world's top auditing agencies, but these agencies have too much business to handle and may not be able to prioritize the needs of certain customers.Many star project teams will also find some less well-known but high-quality auditing institutions to supervise at the same time.

The type of clients mentioned above are the favorites of audit companies because they are rich, but this type of clients are also very concerned about the quality of audits, so they generally look for multiple audit companies. As long as you have strong auditing capabilities, you have the opportunity to reach this type of clients, so this type of client is one of our main customer groups.

The second tier is the mid-tier clients, who are “more concerned about audit quality but not necessarily have a lot of money”, but have the potential to become top clients in the future. Although they hope to find a star auditing agency, they may not be able to afford it.

There are only 4 to 5 companies in this circle that can truly be called "top security companies", similar to OpenZeppelin and Trail of Bits.Everyone knows that these institutions are very good, but their prices are also very expensive, which may be 3-10 times more than that of regular auditing companies.

Even if a mid-level client goes to the top auditing firm, they may not accept the offer.For mid-tier clients, instead of spending all their budget on a top audit firm, it is better for them to give the budget to an audit agency with excellent audit quality, or they can find multiple audit firms.This type of customer is our largest group and we hope to grow with them.

The last type of client is the one mentioned earlier, the cheap project at the bottom. This type of client will basically look for whoever is cheap, or spend money to find a well-known auditing agency to review the contract.

So based on the points mentioned above, your statement about the audit industry makes sense.For audit firms with more history and better reputation, there is indeed a Matthew effect.But you can see that some old-established auditing companies, although well-known, have suffered terrible setbacks in recent years and have encountered various problems.

but,As an audit company that emerged later, we must have differentiated advantages.So our strategy is to break through at a single point:

First enter a certain segment and gain an absolute advantage.For example, in the Bitcoin Layer2 ecosystem, our current coverage has exceeded 50%, and our coverage in the Move ecosystem exceeds 80%. Even top auditing agencies like OpenZepplin may not be able to compete with us in these segments.So the so-called "Matthew effect" depends on the environment.

10.Faust: From your personal perspective, Web2 and web3 What is the biggest difference in safety? You can talk about it based on your past experience.

Luis: First of all, I thinkIn terms of development stage, Web3 security is in a very early stage, and the market for Web3 security must be larger than that for Web2 security because Web3 has stronger security requirements.

Here I will tell you a story. There was a Chinese executive in the Silicon Valley security circle who once worked as a VP of a Silicon Valley listed company. He said that the two main groups of people working on security in Silicon Valley are Chinese and Jewish. So why are Chinese people so good at security? BecauseThe security industry is a typical industry that has no presence in normal times, but has to take the blame when something goes wrong. Indians and white people are unwilling to do this, so Chinese people emerge. This is for Web2 security companies.

butWeb3 security is different because blockchain directly deals with money, so the presence of Web3 security is many orders of magnitude greater.Moreover, in this field, many "security practitioners" can directly cash in. Someone once joked that hackers are the most successful in the transition from Web2 to Web3.

From a technical perspective, Web3 security includes parts of Web2 security and reuses many of the latter's technologies.Many systems now, for example, many DeFi applications have servers and interfaces, which also require traditional penetration testing, DoS defense, etc., which is actually part of Web2 security.

The article comes from the Internet:Dialogue with ScaleBit: Interesting things about Web3 security audit

Related recommendations: AO is about to be launched, a comprehensive interpretation of AO's technical principles and ecological potential

This article will mainly introduce the technical principles of AO and related ecology. Author: Charlotte, Kevin; Metrics Ventures 1 Starting from storage, AO helps Arweave regain its momentum. The Arweave mainnet was launched on November 18, 2018. In more than five years, it has undergone 13 major upgrades. The business direction is permanent…