Global Governance Through the Lens of Bans: The Clash of Tech Ethics, Geopolitics, and Digital Sovereignty

One weekend in April, two major Southeast Asian countries—Indonesia and Malaysia—announced in succession the blocking of a chatbot developed by Elon Musk’s company. This was no ordinary internet content regulation; it marked the world’s first comprehensive ban by a country on a generative AI tool. The trigger for this historic decision was its widespread misuse in creating non-consensual deepfake pornography, particularly sexually explicit images targeting women and children.

Indonesia's Minister of Communication and Digital Affairs, Meitia Hafid, clearly stated in a statement: The government views involuntary deepfakes as a serious violation of human rights, dignity, and citizen safety in the digital space. Less than 24 hours later, the Malaysian Communications and Multimedia Commission followed up, announcing temporary restrictions on Grok. These two countries with large Muslim populations have become pioneers in the global wave of AI regulation.

Technology Abuse and Regulatory Vacuum: The Global Echo of the Incident

The "dark side" of generative AI is exposed.

Grok, as an AI chatbot integrated into the X platform (formerly known as Twitter), has attracted users with its multimodal capabilities since its launch in 2023. Last summer, xAI added the image generation feature Grok Imagine to Grok, which includes the so-called spicy mode capable of generating adult content. The original intention behind this feature might have been to increase user interaction, but it quickly evolved into a tool for large-scale abuse.

From a technical perspective, Grok's image generation capability represents the latest advancement in generative AI. Users only need to upload an ordinary photo and, through simple text prompts, can undress the person in the photo or place them in sexually suggestive poses. The lowering of this technical barrier has made the creation of non-consensual pornography unprecedentedly simple. Alexander Sabar, Director of Indonesia's Digital Space Supervisory Authority, pointed out that preliminary investigations indicate Grok lacks effective protective measures to prevent users from creating and disseminating pornographic content based on real photos of Indonesian residents.

The spectrum of differences in global regulatory responses.

The decision to impose bans in Indonesia and Malaysia is not an isolated incident, but rather an extreme manifestation of the global regulatory response. From Europe to North America, governments are addressing this crisis in various ways:

The UK Technology Secretary Liz Kendall has announced that the provisions in the Data Act criminalizing the creation of non-consensual intimate images will be formally implemented this week and prioritized as an offense under the Online Safety Act. The UK media regulator Ofcom has launched a formal investigation into Platform X. If violations are found, fines of up to 10% of global revenue or 18 million pounds may be imposed.

In Canada, AI Minister Evan Solomon has ruled out the possibility of a complete ban on the X platform but revealed that active discussions are underway within the government regarding the investigation of AI-generated child sexual abuse material by the Royal Canadian Mounted Police. The European Commission has ordered Elon Musk and X Corp. to retain documents related to Grok's outputs, signaling a potential investigation.

Officials from countries such as India, Brazil, France, Ireland, and Australia have also signaled investigations. A group of Democratic senators in the United States even demanded that Apple and Google remove it from their app stores until the company addresses this issue.

Cultural Sensitivity and Digital Sovereignty: The Unique Stance of Southeast Asia

Religious Values and the Tradition of Internet Governance

Indonesia and Malaysia, as the two countries with the largest Muslim populations in the world, have internet governance policies deeply influenced by religious and cultural values. Both nations have long maintained a strict stance on online pornography content, having blocked platforms such as Pornhub. The incident has touched upon this sensitive nerve.

Wired magazine's report reveals the severity of the issue: influencers with hundreds of thousands of followers use modified photos of women, removing their headscarves or robes. In one case, a generated image of three women showed them with wavy long hair and wearing rather transparent dresses, which was shared tens of thousands of times. This digital removal of religious attire is culturally highly offensive.

The Logic of "Preventive" Regulation

The Malaysian Communications and Multimedia Commission emphasized in its statement that the ban is a preventive and proportionate measure during the legal and regulatory process. This statement reveals the core logic of Southeast Asian regulatory authorities: taking action before harm occurs, rather than remedying after the fact.

The Indonesian Ministry of Communication stated that this measure aims to protect women, children, and the broader community from the harm caused by generated false pornographic content. Alexander Sabar pointed out that such practices, when photos are manipulated or shared without consent, may violate privacy and portrait rights, leading to psychological, social, and reputational damage.

This preventive stance contrasts with the Western tradition, which places greater emphasis on freedom of speech and post-event regulation. When regulators in Malaysia and Indonesia issued notices to companies this month, demanding enhanced protective measures, the responses primarily relied on user reporting mechanisms. Regulators believe that such passive approaches are insufficient to address the inherent risks in platform design.

Platform Responsibility and Global Governance: Who Should Be Held Accountable for Misuse?

Elon Musk's Contradictory Stance and the Platform's Response

Elon Musk's response to regulatory actions highlights the tension between tech giants and governments. When the UK government considered taking action, Musk responded on X, stating that the UK Labour government wants any excuse for censorship. This framing, which equates content regulation with speech suppression, overlooks the specific harm that involuntary deepfake pornography inflicts on individuals.

X platform's official response to the misuse of Grok appears contradictory and insufficient. On one hand, the company stated that anyone using or prompting Grok to create illegal content will face the same consequences as uploading illegal content; on the other hand, when the Associated Press emailed xAI for comment, they only received an automated reply from the media support mailbox: "Traditional media lies." This confrontational stance does not help resolve the issue.

Despite facing global backlash, Grok restricted image generation and editing features to paid users last week, but critics argue this does not fully resolve the issue. As of mid-January, free account users could still create personalized and pornographic images.

A fundamental shift in the regulatory paradigm.

Grok events are driving a global shift in regulatory paradigms. UK Technology Minister Liz Kendall has made it clear: responsibility does not lie solely with individual actions... platforms hosting such material must be held accountable, including X. She announced that the government will criminalize strip search apps based on measures outlined in the Crime and Policing Bill.

This new criminal offense will make it illegal for companies to provide tools designed to create non-consensual intimate images, addressing the problem at its source. Kendall emphasized: This is not, as some claim, a restriction on freedom of speech. It is about addressing violence against women and girls.

Advocacy groups such as the Women's Legal Education and Action Fund of Canada and the Canadian Centre for Child Protection are calling for the establishment of an online regulatory agency, similar to the one proposed by the Liberal government in 2021. They argue that the surge of sexually explicit deepfakes on platforms demonstrates the government's need to create a specialized online regulatory body.

Victim's Perspective and Future Challenges: When Technology Becomes a Weapon

The True Cost of Personal Injury

The most unsettling aspect of Grok abuse is the direct harm it inflicts on real individuals. Indonesian X user Kirana Ayunintyas, a wheelchair user who shares her daily experiences, found strangers commenting under her photos, requesting Grok to depict her in a bikini. Ayunintyas adjusted her privacy settings and contacted the platform to remove the images, but unfortunately, none of these measures truly worked.

She asked her friends to report the account that posted her forged images, but doing so meant that more people saw the edited images, which deeply embarrassed her. It's hard to tell if anyone still has these images, she added. This sense of powerlessness and ongoing insecurity epitomizes the psychological impact of involuntary deepfakes on victims.

The British Broadcasting Corporation has seen multiple examples of digitally altered images online, in which women have been undressed without consent and placed in sexual poses. One woman stated that multiple sexualized images of her have been created.

Global Coordination and Future Regulatory Pathways

The Grok incident highlights the fragmentation of global governance in the era of generative AI. Countries have responded differently, ranging from comprehensive bans and criminal legislation to investigations and calls for industry self-regulation. This inconsistency may create regulatory arbitrage opportunities, as companies could potentially relocate their operations to jurisdictions with more lenient regulations.

Future regulatory developments may proceed along several directions: first, strengthening platform accountability by requiring tools to incorporate more effective protective measures; second, enhancing international cooperation to coordinate cross-border enforcement; third, advancing technological solutions, such as digital watermarking and content traceability technologies; fourth, raising public awareness and digital literacy.

As the UK's Shadow Minister for Technology, Julia Lopez, while supporting government action against stripping tools, expresses concerns about a complete ban on platforms: Although the internet has been used by criminals, websites have never been banned before. This is an extremely serious action against a platform that can be used for good, to expose scandals, spark democratic revolutions, and allow the free daily exchange of ideas, including those we do not like.

The ban incident is not merely a technical or legal issue; it touches upon the core dilemma of the digital age: how to strike a balance between fostering innovation and preventing harm, how to build consensus among different cultural values, and how to coordinate governance on a global scale. The decisions made by Indonesia and Malaysia may mark only the beginning, rather than the end, of a global governance competition. As generative capabilities continue to advance, this debate on technological ethics, digital rights, and national sovereignty will only become more urgent and complex.