Cryptocurrency exchange Coinbase tests ChatGPT to analyze cryptocurrency security

Cryptocurrency exchange Coinbase has conducted a test for ChatGPT — AI was asked to analyze the security of ERC-20 tokens. Although the algorithm did not meet expectations, Coinbase noted that the chatbot «shows potential.»

Coinbase compared the results of ChatGPT’s analysis with the results of its internal team that checks blockchain projects. In total, the company compared 20 risk assessments of smart contracts by ChatGPT and humans. In 60% of cases, ChatGPT’s results coincided with the assessments from the crypto exchange’s staff. However, in 40% of cases, the chatbot called risky tokens reliable.

The exchange noted that such an error makes it impossible to integrate ChatGPT into the token audit process. Coinbase will continue to explore the chatbot’s capabilities. When the accuracy of ChatGPT reports becomes higher, AI will become a permanent «assistant» to the Coinbase team. Currently, the company is not considering the option of fully automated auditing.

Coinbase noted: «The main problems with verifying tokens through a chatbot are inconsistent results and their error due to insufficient context.»

This is not the first time Coinbase has experimented with ChatGPT. Last week, former crypto exchange director Conor Grogan tested a real Ethereum smart contract in GPT-4 and the chatbot instantly pointed out several «security vulnerabilities» and described the scenarios of code usage.