More on AI Image Processing

It was around late fall in 2023-24. I was in a small group of visitors to a steel mill of a major steelmaker in Japan and we were ushered towards one of the facilities in the steel-making line. 

Here, the hot-rolled steel coils run through the conveyer belt were scanned with a camera for any surface defects. The factory representative explained that the defects were detected using special software developed in-house, using defect images as training data. At that time, generative AI was already starting to be a popular topic worldwide, but Japan being quite “conservative” in software, there wasn't any mention of AI (人工知能 artificial intelligence in Japanese) by that representative. (At least that what I have remembered) 

“That's a neat way for defect inspection” I thought to myself, remembering that back at our factory it's done with the ol' blood-fueled retina. I wished that we someday could also build a system like that so that daily quality control tests would be more efficient. 

A lot has changed in AI throughout these years that I think someone with strong motivation and skills could attempt to develop something similar from inside their bedroom. Though image recognition is not a new technology, with the advent of AI it seems like this technology will be perfected in new future, if it isn't already. 

There is a lot of debate about whether this is a good thing or not (with con's usually regarding the use of facial recognition in surveillance), but for me, the future would introduce many more technology that was sci-fi to me 20 years ago, just like how space age was alien to person born at the start of the 20th century.

AI in Image Recognition

In my “Receipt Tracker” project ([https://github.com/muzamono/receipt-tracker](https://github.com/muzamono/receipt-tracker)), one of the main aims is to implement image processing of payment receipts to directly input the expenses data into the app. 

Testing Claude.ai image processing through the chat interface returned promising result, but I'm not ready to jump the gun by subscribing to any LLM yet. Searching for inexpensive method to test this method, I came to 2 options: Leveraging LLM's (Gemini or DeepSeek) for the whole extraction, or the combination of free OCR tool (e.g., Tesseract, or a cloud OCR API) to convert the receipt image to raw text, then send that text to DeepSeek with JSON-extraction prompt. 

I will talk about the first option for this post. 

Gemini gives free tier access to API, but trying to extract image data using hardcoded model number (gemini-2.0-flash or gemini-2.5-flash) proved fruitless with 429 RESOURCE\_EXHAUSTED error. However, further interrogation with Gemini helped me to gain a helpful intel; model = ‘gemini-flash-latest’, which will automatically route the request to whichever active Flash model has free-tier quota allocated for the key type. 

```python
response = client.models.generate_content(
 model="gemini-flash-latest",
 contents=[processed_img, prompt],
 config=types.GenerateContentConfig(
 response_mime_type="application/json",
 response_schema=ReceiptData, # Enforces structured output schema
 temperature=0.1, # Strict accuracy for financial data
 ),
 )
```

Now that returned the result below with impressive accuracy compared to the sample image used. (Image at the top of the post) 

```python
{
 "merchant": "SUSHI YA",
 "date": "2026-02-14",
 "total_amount": 54.65,
 "currency": "MYR",
 "category": "Food",
 "line_items": [
 {
 "name": "GREEN TEA HOT",
 "quantity": 2.0,
 "unit_price": 2.5,
 "total": 5.0
 },
 {
 "name": "CHICKEN KATSU DON",
 "quantity": 1.0,
 "unit_price": 15.9,
 "total": 15.9
 },
 {
 "name": "CHICKEN KARAAGE RAMEN (TORI PAITA",
 "quantity": 1.0,
 "unit_price": 22.9,
 "total": 22.9
 },
 {
 "name": "RED PLATE",
 "quantity": 1.0,
 "unit_price": 3.33,
 "total": 3.33
 }
 ],
 "extraction_confidence": "high"
}
```

Gemini is quite promising for this use case, though my sample number is really small and not robust for production level yet. 

Now, onto DeepSeek. 

Quick test of sample image with DeepSeek chat function returned promising results, but it turned out that the API models cannot receive input in image format (yet). So currently it is out of my consideration. 

All in all, I can say that Gemini API is capable of extracting data from images even in free tier and is usable in testing environment. As for implementing which method of data extraction, I may need to run further tests.

Hello, call me Zakuan

About Me

Featured Projects

Stocks Tracker ETL

Receipt Tracker

Latest Blog Posts

AI in Image Recognition

More on AI Image Processing

Photo Gallery

More on AI Image Processing