Global Governance Through the Lens of Bans: The Clash of Tech Ethics, Geopolitics, and Digital Sovereignty

One weekend in April, two major Southeast Asian countries—Indonesia and Malaysia—announced in succession the blocking of a chatbot developed by Elon Musk’s company. This was no ordinary internet content regulation; it marked the world’s first comprehensive ban by a country on a generative AI tool. The trigger for this historic decision was its widespread misuse in creating non-consensual deepfake pornography, particularly sexually explicit images targeting women and children.

Indonesian Minister of Communication and Digital Affairs, Metia Hafid, explicitly stated in a declaration: "The government views involuntary deepfakes as a severe violation of human rights, dignity, and citizen safety in the digital space." Within less than an hour, the Malaysian Communications and Multimedia Commission followed up, announcing the implementation of temporary restrictions. These two countries, with large Muslim populations, have become pioneers in the global wave of regulation.

Technology Abuse and Regulatory Vacuum: The Global Echo of the Incident

The "dark side" of generative AI is exposed.

As a chatbot integrated into the platform (formerly known as ), it has attracted users with its multimodal capabilities since its launch in the year . Last summer, the image generation feature " " was added for , which includes the so-called "spicy mode" capable of generating adult content. While this feature was perhaps designed with the intention of increasing user engagement, it quickly evolved into a tool for large-scale abuse.

From a technical perspective, the image generation capabilities represent the latest advancements in generative AI. Users simply need to upload an ordinary photo and, through simple text prompts, can "undress" the person in the photo or place them in sexually suggestive poses. This lowering of the technical threshold has made the creation of non-consensual pornography unprecedentedly simple. Alexander Sabar, Director General of Indonesia's Digital Space Oversight Authority, pointed out that preliminary investigations reveal a lack of effective protective measures to prevent users from creating and disseminating pornographic content based on real photos of Indonesian residents.

The spectrum of differences in global regulatory responses.

The decision to impose bans in Indonesia and Malaysia is not an isolated incident, but rather an extreme manifestation of the global regulatory response. From Europe to North America, governments are addressing this crisis in various ways:

UK Technology Minister Liz Kendall announced that the provisions in the Data Act criminalizing the creation of non-consensual intimate images will be formally implemented this week and designated as a "priority offense" under the Online Safety Act. The UK media regulator has launched a formal investigation into platforms. If violations are found, fines of up to % of global revenue or £ million may be imposed.

In Canada, Minister Evan Solomon ruled out the possibility of a comprehensive ban on platforms but revealed that "active discussions" are underway within the government regarding an investigation by the Royal Canadian Mounted Police into generated child sexual abuse material. The European Commission has ordered Musk and his company to retain documents related to the output, signaling a potential investigation.

Officials from countries such as India, Brazil, France, Ireland, and Australia have also signaled investigations. A group of Democratic senators in the United States even demanded that Apple and Google remove it from their app stores until the company addresses this issue.

Cultural Sensitivity and Digital Sovereignty: The Unique Stance of Southeast Asia

Religious Values and the Tradition of Internet Governance

Indonesia and Malaysia, as the two countries with the largest Muslim populations in the world, have internet governance policies deeply influenced by religious and cultural values. Both nations have long maintained a strict stance on online pornography content, having blocked platforms such as Pornhub. The incident has touched upon this sensitive nerve.

Wired magazine's report reveals the severity of the issue: influencers with hundreds of thousands of followers use modified photos of women, removing their headscarves or robes. In one case, a generated image of three women showed them with wavy long hair and wearing rather transparent dresses, which was shared tens of thousands of times. This digital removal of religious attire is culturally highly offensive.

The Logic of "Preventive" Regulation

The Malaysian Communications and Multimedia Commission emphasized in its statement that the ban is a "preventive and proportionate measure during the ongoing legal and regulatory process." This statement reveals the core logic of Southeast Asian regulatory bodies: taking action before harm occurs, rather than remedying it after the fact.

The Indonesian Ministry of Communication stated that this measure aims to protect women, children, and the broader community from the harm caused by generated false pornographic content. Alexander Sabar pointed out that such practices, when photos are manipulated or shared without consent, may violate privacy and portrait rights, leading to psychological, social, and reputational damage.

This preventive stance contrasts with the Western tradition, which places greater emphasis on freedom of speech and post-event regulation. When regulators in Malaysia and Indonesia issued notices to companies this month, demanding enhanced protective measures, the responses primarily relied on user reporting mechanisms. Regulators believe that such passive approaches are insufficient to address the inherent risks in platform design.

Platform Responsibility and Global Governance: Who Should Be Held Accountable for Misuse?

Elon Musk's Contradictory Stance and the Platform's Response

Elon Musk's response to regulatory actions highlights the tensions between tech giants and governments. When the UK government considered taking action, Musk responded on X, stating that the UK Labour government wants "any excuse to censor." This framing, which equates content regulation with speech suppression, overlooks the specific harms caused by non-consensual deepfake pornography to individuals.

The platform's official response to abuse appears contradictory and insufficient. On one hand, the company states that "anyone who uses or prompts the creation of illegal content will face the same consequences as those who upload illegal content"; on the other hand, when the Associated Press requested comment via email, it only received an automated reply from the media support mailbox: "Traditional media lies." This adversarial stance does little to resolve the issue.

Despite facing global backlash and restricting image generation and editing features to paid users last week, critics argue that this has not fully resolved the issue. As of mid-month, free account users can still create "sexualized" and pornographic images.

A fundamental shift in the regulatory paradigm.

Events are driving a shift in the global regulatory paradigm. UK Technology Minister Liz Kendall explicitly stated: "Responsibility does not lie solely with individual actions... Platforms hosting such material must bear responsibility, including." She announced that the government will criminalize "strip" apps based on measures outlined in the Crime and Policing Bill.

This new criminal offense will make it illegal for companies to provide tools designed to create non-consensual intimate images, "tackling the problem at its source." Kendall emphasized: "This is not, as some have claimed, about restricting freedom of speech. It is about addressing violence against women and girls."

Advocacy groups such as the Women's Legal Education and Action Fund of Canada and the Canadian Centre for Child Protection are calling for the establishment of an online regulatory agency, similar to the one proposed by the Liberal government in 2021. They argue that the surge of sexually explicit deepfakes on platforms demonstrates the government's need to create a specialized online regulatory body.

Victim's Perspective and Future Challenges: When Technology Becomes a Weapon

The True Cost of Personal Injury

The most disturbing aspect of abuse is the direct harm it inflicts on real individuals. Indonesian user Kirana Ayuningtyas, a wheelchair user who shares her daily experiences, found strangers commenting under her photos, demanding depictions of her in a bikini. Ayuningtyas adjusted her privacy settings and contacted the platform to remove the images, but "unfortunately, none of that really worked."

She asked her friends to report the account that posted her forged images, but doing so meant that more people saw the edited pictures, which deeply embarrassed her. "It’s hard to tell whether someone has these images," she added. This sense of helplessness and ongoing insecurity epitomizes the psychological impact of involuntary deepfakes on victims.

The British Broadcasting Corporation has seen multiple examples of digitally altered images online, in which women have been undressed without consent and placed in sexual poses. One woman stated that multiple sexualized images of her have been created.

Global Coordination and Future Regulatory Pathways

The incident reveals the fragmentation of global governance in the generative era. Countries have responded differently, ranging from comprehensive bans and criminal legislation to investigations and calls for industry self-regulation. Such inconsistency may create opportunities for "regulatory arbitrage," where companies might relocate their operations to jurisdictions with more lenient regulations.

Future regulatory developments may proceed along several directions: first, strengthening platform accountability by requiring tools to incorporate more effective protective measures; second, enhancing international cooperation to coordinate cross-border enforcement; third, advancing technological solutions, such as digital watermarking and content traceability technologies; fourth, raising public awareness and digital literacy.

While supporting the government's action against "strip" tools, Julia Lopez, the UK Shadow Technology Secretary, expressed concerns about a comprehensive ban on platforms: "Although the internet has been used by criminals, websites have never been banned before. This is an extremely serious action against a platform that can be used for good—to expose scandals, spark democratic revolutions, and allow the free exchange of ideas in daily life, including ideas we do not like."

The ban incident is not merely a technical or legal issue; it touches upon the core dilemma of the digital age: how to strike a balance between fostering innovation and preventing harm, how to build consensus among different cultural values, and how to coordinate governance on a global scale. The decisions made by Indonesia and Malaysia may mark only the beginning, rather than the end, of a global governance competition. As generative capabilities continue to advance, this debate on technological ethics, digital rights, and national sovereignty will only become more urgent and complex.