In recent years, there has been great enthusiasm around the potential of artificial intelligence (AI) to transform society, from improving medical diagnoses to reducing learning gaps. Tax authorities around the world have also joined this hype and focused their sights on a core problem: Uprooting tax fraud. India, too, is excited by AI’s potential to crack down on tax cheats.

However, this enthusiasm needs to be tempered by careful planning and testing. AI is unlikely to strengthen tax administrations unless they are equipped to use it and subject it to rigorous testing. A recent study from India involving an AI-based tool to prevent tax fraud highlights this.

A growing number of countries are integrating AI tools to improve tax administration. A 2023 OECD report showed that about 80% of tax administrations surveyed had implemented or were implementing AI tools for virtual assistance, predictive analytics, and transcription, among other uses. In India, tax authorities are exploring AI to improve dispute resolution and auditing.

This trend aligns with a wider push to leverage technology to improve tax collection. For instance, a randomised evaluation in Ghana found that providing electronic tablets loaded with a geospatial database to property tax collectors led to a 103% increase in collections. Another evaluation in Senegal found that a computer-assisted property valuation system was much more accurate and fair than the manual appraisals of property tax collectors.

The gains of technology are not inevitable, though. In the India study, researchers used machine learning, an offshoot of AI, to detect Goods and Services Tax (GST) fraud. While the tool was successful in detecting fraud, it did not lead to an increase in enforcement or revenues. This project offers a cautionary tale on applying AI and other technologies to tax collection.

The GST subsumed multiple central and state taxes to make collections more efficient and simple. However, the system has been undermined by rampant evasion. The estimated revenue loss in FY 2024 was equivalent to 10% of total GST collections, per the Directorate General of GST Intelligence.

Some firms have figured out creative ways to evade taxes. They may undervalue goods and services to pay less tax, wrongfully claim exemptions, or claim more input tax credits (ITC) than allowed.

The fraudulent claim of ITCs through fake firms has been especially problematic. Fake firms are entities that exist on paper purely to generate fictitious invoices and ITCs. Taxpayers can use these fake credits to offset some, if not all, of their tax liability. In August, for example, a major GST scandal was uncovered in Delhi, where 500 fake firms generated invoices worth ₹718 crore to claim GST refunds worth ₹54 crore.

To catch fake firms, a research team led by J-PAL Affiliate Aprajit Mahajan at the University of California, Berkeley created a novel machine learning tool. Machine learning technology is a branch of AI that can process large amounts of data, detect patterns, and make predictions.

Using tax data and inspection records, the researchers trained the tool to generate a ranked list of firms that had a high likelihood of being fake. The researchers used common signs of fraud, like claiming a high level of ITCs with little or no actual tax paid or sharing a registered address with multiple firms. Although conventional methods consider similar criteria, the tool incorporated a wider set of factors to detect fraud.

Based on field inspections, the ML tool was found to be highly effective — 53% of the firms flagged by the ML tool were confirmed to be fake, compared to 38% of firms flagged by conventional methods. This is impressive given how difficult it is to identify fake firms and their clients. Tax evaders can bounce money among multiple fake firms, across jurisdictions, to make it harder for authorities to catch them.

In spite of the tool’s accuracy, it did not lead to increased or faster cancellation of fake firms, or in increased enforcement against the clients of fake firms. In other words, tax collection did not improve, even though the tool was relatively accurate.

The Government of India aims to become a global leader in AI through its IndiaAI Mission. States like Telangana have also created dedicated AI research centres. This study offers important lessons for policymakers to effectively leverage this technology.

The biggest takeaway is this: Even though AI could reduce the burden of tedious tasks and free up time for other tasks better suited for humans, governments need to rigorously evaluate whether an AI tool will lead to desired outcomes. It is unwise to assume AI will inherently lead to major improvements.

In this case, the AI tool offered a more efficient and accurate way to detect fake firms, which could have enabled tax authorities to focus on taking action. But it did not provide evidence to help officials investigate and build a case against them — officials still would have had to invest a lot of time investigating each case. Spending great time and money on building an AI tool, only for it to fall into disuse, would be a tremendous waste.

More fundamentally, policymakers should first ask whether AI is the best solution to a problem. The researchers say that the ML model they developed may be detecting firms that are likely to be inspected or cancelled anyway by conventional methods. So, there may be higher and quicker returns to simpler technologies that, for instance, improve coordination between tax officials in different jurisdictions.

The appeal of AI is understandable. It is a general purpose technology that is widely accessible, highly adaptable, and constantly improving. But deploying AI for AI’s sake is unwise.

This article is authored by Guillermo Herrera Nimmagadda, policy & training manager, J-PAL South Asia.