AI Image Detectors Stumble on GPT-Image-2 Receipts and Tables
A benchmark of 8,602 text-rich images finds that five existing AI-image detectors perform very unevenly across categories, and that JPEG compression undercuts the best of them.
TL;DR
- A Sichuan University benchmark of 8,602 text-rich images, 5,616 made with GPT-Image-2, shows five existing detectors performing very unevenly, with tables and receipts hardest [^1].
- The best conventional detector's F1 score fell from 82.66 to 33.98 percent under mild JPEG compression; GPT-5.5 scored 85.72 percent accuracy but struggled on tables [^1].
Background
Image generators long gave themselves away with garbled lettering, which made fake text-rich images fairly easy to spot [^1]. Newer models such as OpenAI's GPT Image 2 render readable text and structured layouts, so their output can closely resemble real documents [^1]. Most datasets for testing AI-image detectors, the authors write, show objects with little meaningful text, while text-focused ones cover single domains such as receipts or scientific figures [^1].
What happened
Yijin Wang, Shuyi Wang, Wenhan Zhang and Yuqi Ouyang of Sichuan University's College of Computer Science posted the benchmark on arXiv on June 17 [^1]. It holds 8,602 images in six categories: commercial posters, infographics, academic posters, receipts, tables and UI screenshots. Of these, 5,616 are generated and 2,986 real [^1]. The real images come from public datasets such as SROIE for receipts and PubTabNet for tables. For the fakes, the team turned structural, semantic and style cues from those datasets into prompts for GPT-Image-2, without reusing their text or any identifying information [^1].
The team then ran five public detectors as released, with no fine-tuning: the Vision Transformer model Dima806, the SigLIP-based prithivMLmods, SuSy from HPAI-BSC, CNNSpot, and NPR, which looks for pixel patterns left by the upsampling steps inside image generators [^1]. NPR did best overall, with an F1 score of 82.66 percent, while CNNSpot flagged only 163 of the 5,616 generated images [^1]. Results swung by category: NPR's F1 ranged from 13.58 percent on tables to 99.24 percent on commercial posters, and Dima806's from 0.36 percent on receipts to 93.77 percent on commercial posters [^1]. On receipts and tables, detectors mostly missed fakes; on UI screenshots and commercial posters, they more often flagged real images as fake [^1].
The generated images are PNG files, while some real ones were collected as JPEG, and online platforms often compress and resize images further [^1]. So the team re-encoded the images for NPR. Mild JPEG compression cut its F1 score from 82.66 to 33.98 percent, stronger compression pushed it toward failure, and lossless PNG re-encoding left it close to the original; infographics nearly collapsed [^1]. In a separate test, GPT-5.5, asked whether each image was AI-generated and why, reached 85.72 percent accuracy: near-perfect on receipts, but 55.63 percent on tables [^1].
Why it matters
The weak spots sit where the stakes are. The authors note that text-rich images often hold privacy-sensitive, transactional or decision-relevant information, yet the detectors did worst on average on tables and receipts [^1]. The strongest pixel-level detector also depends on traces that ordinary JPEG compression wipes out, the kind of processing the authors say online platforms often apply [^1]. A detector that works mainly on untouched PNG files says little about an image that has been shared online. The results also show how existing tools generalize without retraining, not how well a detector trained on this data could do [^1].
GPT-5.5's explanations point to a different approach. On receipts, it often cited mismatched addresses, ZIP codes, phone numbers, product descriptions and arithmetic [^1]. The authors caution that these explanations may not reflect how the model actually decided, and tables show the limit: when a generated table had a realistic layout and plausible numbers, the model often judged it real [^1]. Their conclusion is that forensics research needs to move toward text-aware detection [^1]. The first version of the paper lists the dataset's download location only as a placeholder [^1].
Practical example
Say Tomasz runs expense approvals at a mid-sized logistics firm, where staff upload photos of receipts. To catch fakes, the firm adds an off-the-shelf AI-image detector to the upload form.
An employee makes up a dinner receipt with GPT-Image-2. Modern generators render clean, readable text, so the receipt passes a quick glance [^1]. The employee sends the file through a chat app first, which saves it as a compressed JPEG. If the firm's detector works like NPR, that step alone can be enough: under stronger JPEG compression, NPR classified fakes as real on a wide scale [^1]. A detector like Dima806 would barely have helped even on the original file, with an F1 score of 0.36 percent on receipts [^1].
Tomasz then tries a multimodal model like GPT-5.5 that reads the receipt as well as looking at it. In the study, GPT-5.5 was near-perfect on receipts and often pointed to a ZIP code that didn't fit the address, or to sums that didn't add up [^1]. The fake dinner gets caught.
The next month, someone submits a fake mileage table instead. GPT-5.5 managed 55.63 percent accuracy on tables, and realistic layouts with plausible numbers often passed as real [^1]. For Tomasz, the detector score becomes one signal among several, not the final word.