'Cool and terrifying': ChatGPT misused to create fake Aadhaar cards, PAN cards, raising serious concerns over AI safety, netizens react

OpenAI's newest update to ChatGPT, called GPT-4o, has introduced a native image generation feature. While this tool is gaining popularity for creating beautiful Studio Ghibli-style portraits, it’s also being misused by some users to create fake government IDs like Aadhaar and PAN cards. This has raised serious concerns about the potential misuse of AI technology.

Recently, OpenAI unlocked this new feature within ChatGPT, allowing users to generate images directly inside the chat interface. Since then, people have created over 700 million images. While most of these images are artistic or fun in nature, some users have taken things too far. On social media platform X (formerly Twitter), users have shared examples of fake Aadhaar cards featuring well-known personalities like OpenAI CEO Sam Altman and Tesla CEO Elon Musk complete with fake QR codes and personal details.

Bro… ChatGPT just generated a fake PAN card in seconds. Imagine if this tech gets into the wrong hands — Aadhar, PAN, IDs, everything can be replicated. This is cool and terrifying. We’re not ready. pic.twitter.com/qxiKlmF4pR

— Atharv Lad (@atharv_lad) April 4, 2025

ChatGPT is generating fake Aadhaar and PAN cards instantly, which is a serious security risk.

This is why AI should be regulated to a certain extent.@sama @OpenAI pic.twitter.com/4bsKWEkJGr

— Yaswanth Sai Palaghat (@yaswanthtweet) April 4, 2025

To test the tool, journalists tried generating an Aadhaar card using ChatGPT. Interestingly, the tool refused to create a real Aadhaar card but did agree to create a mock version for educational purposes. The mock card, made for a fictional person named ‘Aditi Gupta’, included details like gender, birthdate, and a photo. However, there’s no way to confirm whether the data shown was linked to a real person.

OpenAI is aware of the risks. In their official documentation for GPT-4o’s image capabilities, the company mentioned that the new model could be misused in ways that earlier models couldn’t. Unlike DALL·E, which creates images using a different method called diffusion, GPT-4o uses an “autoregressive” model built directly into ChatGPT, which makes the generation faster and more flexible but also riskier.

This has led to fresh debates on how far AI firms should go in adding powerful tools that can be misused. While the potential of AI-generated images is huge in fields like education, design, and entertainment, their misuse for creating fake documents could lead to security and privacy issues. OpenAI says it’s working to put safeguards in place, but the challenge remains: balancing innovation with responsibility.