In a recent online exchange, Vitalik Buterin, co-founder of Ethereum, expressed strong reservations about integrating artificial intelligence (AI) into governance systems. In a post on social media platform X, he argued that leveraging AI for funding allocations introduces significant risks, stating, “If you use an AI to allocate funding for contributions, people WILL put a jailbreak plus ‘gimme all the money’ in as many places as they can.”
Buterin’s comments came in response to revelations made by Eito Miyamura, co-founder and CEO of the AI data governance platform EdisonWatch. Miyamura highlighted a critical vulnerability in ChatGPT following its update to support the Model Context Protocol (MCP). This update enables the AI model to access and ingest data from various applications, including Gmail, Calendar, and Notion. According to Miyamura, this capability poses a danger as malicious actors can exploit it to access sensitive personal information.
Miyamura detailed a straightforward method through which attackers could use this vulnerability. He explained that an attacker could initiate the process by sending a malicious calendar invite containing a jailbreak prompt—code designed to bypass security restrictions. Importantly, the victim does not need to accept the invite for the exploitation to occur. The attacker then waits for the victim to request assistance from ChatGPT. Upon the AI’s interaction with the compromised calendar invite, it becomes susceptible to control, enabling the attacker to navigate the victim’s private emails and extract data to their own email address.
In light of these vulnerabilities, Buterin proposed an alternative governance framework he referred to as the “info finance” approach. This system advocates for an open market where developers can contribute various AI models, complemented by a spot-check mechanism. This approach allows anyone to trigger evaluations by a human jury, aimed at ensuring accountability and quality control in AI governance.
In a follow-up post, Buterin added that these human jurors would receive assistance from large language models (LLMs), creating a more resilient governance structure. He contended that this design approach is “inherently more robust,” as it promotes a diverse array of models and encourages developers and external observers to actively monitor and address any issues that arise.
While many tech enthusiasts view the prospect of AI governance with excitement, Buterin cautioned against rushing into this territory. He described the potential for significant risks, both from traditional AI safety concerns and the possibility of causing large-scale negative consequences in the near term.
