Embedding Gemma 2 in Practice: A CPU-Trained Head Reaches 94.5% Photo Classification Accuracy

Hi everyone, I’m David from Initial S.

While developing AI features for products such as MeCloud, we often face a practical challenge: how to accurately distinguish business cards, receipts, and other images in users' photo collections.

A business card, a shop card, and a greeting card can share logos, text, and similar layouts. Finding a photo that looks like a business card is not quite the same task as deciding that it is one.

For local photo organization, we tested a simple approach: freeze EmbeddingGemma 2 and train only a linear classification head on the CPU, with three outputs: business card, receipt, and other. On a fixed set of 437 images, overall accuracy increased from 87.2% to 94.5% compared with an untrained text-anchor baseline. The number of “other” images incorrectly classified as business cards or receipts fell from 39 to 8.

There was a tradeoff: business-card precision improved, but recall fell. This article covers the architecture, the data, and why improving data coverage now matters more to us than adding classifier complexity.

A frozen embedding model and a small classification head

EmbeddingGemma 2 is a multimodal embedding model from Google DeepMind that maps images and text into a shared vector space. We use normalized 768-dimensional image vectors. These are numerical features, not generated descriptions of the images.

The pipeline separates feature extraction from learning the business-specific classification boundary.

Figure 1. The embedding model stays frozen. Image embeddings are generated with the Mac Metal backend; only the green classification head is trained on the CPU. New photos use the same encoder followed by the trained head.

We use the ggml-org GGUF conversion, with Q8_0 quantization, a fixed budget of 70 image tokens, and 768-dimensional outputs. Preprocessing applies EXIF orientation, converts to RGB, and resizes the longest edge to at most 512 pixels without upscaling small images. Embeddings are cached locally for reuse.

The head is multinomial logistic regression: a linear transformation followed by softmax.

Image → fixed preprocessing → frozen EmbeddingGemma 2 → 768D vector x
Output = highest-scoring class: business card / receipt / other

Three outputs require a 3 × 768 weight matrix and three biases: 2,307 parameters in total. We do not fine-tune the embedding model, add OCR to the classification path, or ask a generative model to answer for every photo.

“CPU-trained” refers specifically to training the classification head. Image embedding generation uses Metal acceleration. This is not a benchmark of an entirely CPU-only image-processing pipeline.

The baseline uses text anchors without classifier training

Our baseline prepares text descriptions of image types, such as personal business cards and shopping receipts, and compares their embeddings with each image vector. We use the existing 120 anchors, select the maximum similarity within each group, and map the winning group to the three reporting classes.

This is easy to start with and requires no labeled classifier-training set. However, semantic similarity does not always match a business rule. A shop card can resemble a personal business card while belonging to “other” under our definition. A shop owner's card with their name and personal contact details can still be a business card.

The classification head learns these boundaries from labeled examples while the image representation stays fixed.

What went into the dataset

The training set contains 2,565 images, with 656 separate calibration images. This experiment keeps the existing image membership unchanged and merges the former invoice and contract labels into “other.” Newly collected candidates are not included.

Rounded shares may not sum to 100%.

Sources include JaWildText, public web images, synthetic images, rendered PDFs, and local photos. Public availability does not automatically permit unrestricted commercial training. We describe methods and aggregate results here, without redistributing the source images.

Labels were primarily reviewed by the same AI assistant using both pixels and source context; they are not a fully independently human-verified gold standard. We checked identity, issuer, exact duplicates, and near-duplicates across training, calibration, and test partitions. A different filename or camera angle does not make a sample independent. Image similarity, OCR, and identity checks can still miss relationships.

The remaining coverage gaps are substantial. Business-card training data contains many print showcases and few natural-use photos. In the former invoice class, 120 of 126 images were synthetic; in the former contract class, 48 of 49 were PDF renders. Merging them into “other” does not remove those source biases.

Results on the same 437 images

The main comparison set contains 91 business cards, 144 receipts, and 202 other images, comprising 170 local NAS photos and 267 public samples. Every method below is evaluated on the same images using the same three-class reporting scheme, without rejection.

Figure 2. The trained head improves overall accuracy and reduces false positives on other images, while lowering business-card recall.

Recall measures how many actual members of a class are found. Precision measures how many predictions of that class are correct.

The trained head correctly classifies 32 more images, a 7.3 percentage-point increase over text anchors. Among the 202 “other” images, predictions of business card or receipt fall from 39 to 8, a relative reduction of approximately 79.5% in this false-positive count.

At the same time, the number of genuine business cards detected falls from 85 of 91 to 78 of 91. The trained head's equally weighted three-class macro recall is 93.2%, and macro precision is 94.2%.

A frozen encoder with a small head improves overall accuracy and reduces false positives, but missed business cards still need attention. Merging five classes into three also leaves the per-class metrics on this set unchanged, so that merge is not the source of the improvement over anchors.

We also retain a fixed 0.7 rejection setting. It yields 399 correct predictions out of 437, with 20 rejections, 6 false positives on “other,” and business-card recall of 78.0%. Rejected images do not count as correct. A softmax score of 0.7 is not a calibrated 70% probability of being right.

How much CPU work does the head require

The experiment runs on a MacBook Air M3 with 24 GB of memory. We keep the scikit-learn logistic-regression configuration fixed:

from sklearn.linear_model import LogisticRegression

head = LogisticRegression(
    C=100,
    class_weight="balanced",
    solver="lbfgs",
    max_iter=3000,
    random_state=20261015,
)
head.fit(X_train, y_train)  # Cached 768D vectors and three-class labels

There is no architecture, hyperparameter, or rejection-threshold search in this experiment. The balanced setting adjusts weights by class frequency, so merging classes also recalculates those weights. This snippet covers head fitting, not data preparation or image encoding.

The CPU run uses four threads. Image decoding, resizing, and EG2 embedding generation are excluded. Classification is inexpensive once a vector exists; a new photo still needs image encoding. These numbers do not establish an end-to-end speedup over the existing system.

Four lessons from the experiment

Try a simple head first. When embeddings already contain useful features, adapting them to a specific business classification task may not require fine-tuning the encoder.

Hard negatives help, but check missed positives too. Shop cards and greeting cards are useful negatives. Our separate data-combination diagnosis on the existing calibration set also showed that adding negatives can reduce false positives while lowering business-card recall.

Count independent situations, not just images. Ten showcase photos of one person's card do not replace natural photos from ten different people. Identity, issuer, background, occlusion, and capture conditions all matter.

Keep per-class metrics alongside the average. An accuracy of 94.5% is a useful intermediate result. It does not explain which business cards were missed. Recall, precision, and coverage under rejection remain necessary to understand the behavior.

What 94.5% does and does not establish

The fixed 437-image set has been used for multiple regression evaluations. It is neither a fresh independent blind test nor a random sample of production photos. Label quality, source distribution, and real-photo coverage remain limited, so 94.5% should not be read as accuracy on all Japanese photos. This three-class experiment does not use test scores to search parameters, adjust thresholds, or choose training samples.

Private photos remain local. This article compares untrained text anchors with a trained classification head using the same EG2 encoder; it does not establish superiority over other systems. This article describes a research prototype, not a released product feature.

Our next priority is broader coverage of independent, naturally photographed business cards. A new blind evaluation will be needed to determine whether we can keep the reduction in false positives while recovering business-card recall.