Blindfolded: Building AI Systems for the Dark Web

[IMAGE: Title slide from the HackP 2025 deck — bold red background, white headline. Use as the hero/banner image.]

My AI team has spent years building a production image classifier we are not permitted to see the training data for. Not anonymised. Not redacted. Not even in aggregate. The five engineers on my team have never laid eyes on a single image the model was trained on, and they never will.

This is a post about how that constraint shapes the engineering — and why the lessons travel further than the law enforcement context they came from.

I’ve contributed to Grapnel, the darknet intelligence platform built by Kerala Police’s Cyberdome, since 2021 — initially as an independent volunteer, and more recently through Neuralcraft, which became Kerala Police’s official AI partner for Hack’P 2025. Grapnel crawls hidden web content and flags material that law enforcement needs to act on. A core function is image classification — distinguishing safe images from those depicting the sexual exploitation of children. Only specifically authorised officers are permitted to view the sensitive images, and that boundary is enforced strictly, including against my team.

What follows is the architecture we built, the mistakes we made inside it, and the patterns that I believe apply to anyone building AI in regulated or sensitive domains.

Chapter 1: A pipeline a non-engineer can run

[IMAGE: Slide 5 — the workflow diagram: Grapnel AI Team → Police Officers → Build Model → Grapnel deployment.]

The most obvious constraint shapes everything else. The people who can touch the data are not machine learning engineers. The authorised officers are deeply expert in their investigative domain; they are not, and shouldn’t need to be, PyTorch practitioners.

So the first design decision was that the training pipeline had to be operable by a non-engineer without making mistakes that would only surface in production.

We collapsed the pipeline into a single Jupyter notebook structured around two folders — safe/ and unsafe/. To develop and validate it, we needed a placeholder dataset that we could see. We deliberately chose something tonally unrelated to the actual subject matter: cats as safe, dogs as unsafe. The toy data is structurally identical to the production data — same folder shape, same pipeline, same model output — but emotionally distant from it, which matters when the same notebook is being demoed, debugged, and version-controlled in mixed contexts.

[IMAGE: Slide 6 — the dummy training flow with cat/dog images. Skip Slide 7; one is enough.]

Once the pipeline was working on cats and dogs, the officers swap the folder contents for their actual training data and run the same notebook. The output is a model file. They hand it to us. We deploy it. We never see what it was trained on.

Training happens on a GPU machine on police premises, formatted before each session and wiped afterwards. Only the model file leaves.

The principle worth taking from this chapter: when you’re building AI for a regulated or restricted environment, the interface between your engineers and the people who hold the data is itself a product. A training pipeline that only your ML team can run isn’t production-ready — it’s a prototype with a single point of failure.

Chapter 2: Debugging a model you can’t look at

High accuracy numbers are necessary but not sufficient. In our case, the production model reached F1 scores above 0.95 in internal testing — and was still wrong in ways the metrics couldn’t reveal.

We integrated Grad-CAM into the pipeline so officers could generate attribution heatmaps over any image themselves, without needing us. Grad-CAM uses gradients flowing through the network to highlight which regions of an image most influenced the prediction. It’s not perfect interpretability — it’s a partial, layer-specific signal — but for a CNN-based image classifier, it surfaces enough to debug spurious correlations.

[IMAGE: Slide 10 — Grad-CAM heatmap on the cat/dog image, showing concentration on the dog. Use as a quick illustrative example before the bedroom case.]

It paid off almost immediately.

Officers reported that certain ordinary room images were being flagged as unsafe. When they ran Grad-CAM, the heatmap centred on the bed in the image.

[IMAGE: Slides 11 and 12 paired — the bedroom image alone, then with the Grad-CAM heatmap on the bed. This is the strongest visual moment of the post; make it full-width if possible.]

The model had learned a spurious correlation. A disproportionate share of the verified unsafe training images contained beds, and the model had latched onto that as a predictive feature rather than learning the semantically meaningful signal. It wasn’t wrong on those specific images — it was right for the wrong reasons, which meant it would fail unpredictably elsewhere.

The fix was straightforward: rebalance the safe class with more images of bedrooms and furniture to break the association.

A second case was sharper. Grad-CAM on a file-explorer screenshot highlighted not any of the thumbnails inside it, but the grid lines between them. The model had learned nothing meaningful from that image at all. We removed it from the training set.

Two things worth noting honestly about this approach:

• Grad-CAM is a local explanation method with known reliability issues. It can mislead on certain architectures, and it doesn’t generalise to transformer-based models in a clean way. It is a debugging aid, not a guarantee of model trustworthiness.
• It still required officers to manually inspect heatmaps for ambiguous cases — meaning the technique scales with their available time, not with the data volume.

That said, baking interpretability into the operator workflow — rather than treating it as a post-hoc research tool — was one of the highest-leverage decisions we made.

Chapter 3: Building a large dataset from a small one

The officers started with fewer than 100 verified unsafe images. Far too few for a robust production model. Meanwhile, Grapnel had been accumulating thousands of images from its crawler since 2021, almost entirely unlabeled.

Manual labeling at that scale was not feasible — and not just for time reasons. Reviewing potential CSAM is psychologically demanding work, and the pool of authorised reviewers is small. Asking them to label tens of thousands of images was off the table.

We bootstrapped instead.

[IMAGES: Slides 16, 17, 18 used inline alongside the steps below — small inline thumbnails rather than full-width.]

1. Train a small model (ResNet-18) on the limited verified dataset, padded with generic safe images — landscapes, vehicles, objects — to give the safe class enough variety.
2. Run that small model as a pre-labeller across the full unlabeled corpus.
3. Treat its predictions as candidate labels, not ground truth.
4. Have authorised officers review only the images flagged as unsafe. The model was correct around 80% of the time at this stage — imperfect, but enough to make the human review process tractable. They confirmed or corrected each flag.
5. Use the resulting labeled dataset to train the production model — in our case, a ConvNeXt or ResNeXt-class architecture with substantially more parameters.

[IMAGE: Slide 24 — “Use this larger dataset to train a bigger model.” Use as the closing visual for this chapter.]

The counterintuitive part for many teams is that you don’t need a complete labeled dataset to start. You need enough verified data to train a useful first-pass labeller, then let that labeller do the volume work and route only the uncertain cases to human reviewers.

This pattern — small model bootstraps labels, humans correct, larger model trains on the result — is roughly active learning with a pseudo-labelling step. It’s not novel in the literature. It is, in our experience, badly underused in production teams who default to “we don’t have enough data yet” as a reason to delay shipping anything.

Chapter 4: Adapting between training cycles

Models are retrained on a cadence — for us, roughly once a month. Harmful content does not wait for the next training cycle. If the model makes a wrong call on a new image, that image could surface to users before anyone notices.

We built a human-in-the-loop correction mechanism: an authorised officer can override any prediction, immediately removing an image from circulation. But this only solves the immediate instance. The same image will reappear — resized, recompressed, recolored, re-hosted on a different URL.

[IMAGE: Slide 30 — diagram showing the same unsafe image being correctly classified once and misclassified as a variant.]

Cryptographic hashing (MD5, SHA-256) is no help here. A single pixel change produces an entirely different hash. An image already flagged as unsafe is invisible to the lookup the moment it’s reposted as a slight variant.

[IMAGE: Slide 36 — the four image variants with completely different MD5 hashes.]

Perceptual hashing is the standard solution. Algorithms like pHash and dHash generate fingerprints based on visual structure rather than byte content, so visually similar images map to similar hashes.

[IMAGE: Slide 40 — the same four variants now resolving to the same perceptual hash.]

We replaced MD5 with perceptual hashing in our lookup table. When an image gets flagged — whether by the model or by manual correction — its perceptual hash is stored. New images are hashed and checked against the table before being sent to the model. A match short-circuits the classifier entirely.

Honesty about the limits: perceptual hashing is robust to compression artifacts, minor recolouring, and resizing. It degrades on cropping, rotation beyond small angles, and aggressive manipulation. It is not a substitute for the model — it is a fast layer in front of it that catches the long tail of trivial variants that would otherwise burn officer-review time.

[IMAGE: Slide 42 — full pipeline diagram showing the hash table catching a variant without invoking the model.]

The architectural point: continuous learning pipelines solve real problems, but they are not the only way to keep a deployed model effective. A well-designed lookup layer, combined with selective human correction, can absorb a significant fraction of the operational load — without retraining, without expanding the data access surface, and without adding model inference latency to repeat cases.

What this looked like in production

Grapnel has been in active operational use by Kerala Police Cyberdome since 2021, with the image classification pipeline deployed across [N] officer workstations. The combination of bootstrapped labeling and perceptual hash deduplication reduced the manual review burden on authorised officers by roughly [X]% in our internal estimates.

What we got wrong along the way is worth naming too:

• Our first production model overfit to image resolution — a side effect of the verified unsafe dataset being uniformly small thumbnails while our safe set included high-resolution stock images. The model learned "low resolution" as a feature. We caught it in Grad-CAM review and rebalanced.
• Perceptual hashing initially produced too many false positives on near-blank or low-detail images, where many distinct images happened to hash similarly. We added a minimum-entropy filter before hashing.
• Our first attempt at the bootstrapping approach used too small an initial dataset and the pseudo-labels were noisy enough that the correction pass took longer than labeling from scratch would have. The break-even point matters; we underestimated it the first time.

Why this generalises

The constraints we navigated at Grapnel — restricted data access, expert-but-non-technical operators, sparse verified examples, the need to adapt without retraining — appear in different forms across most regulated industries. The specific case I find clients underestimate is healthcare imaging, where the clinical team holds the ground truth, anonymisation is harder than it looks, and the cost of a wrong classification is measured in patient outcomes rather than accuracy points. The engineering patterns are different in detail but structurally familiar: standardise the interface for the domain expert, bake interpretability into the operator workflow, bootstrap from what’s verifiable, and design adaptation layers that don’t require expanding data access.

Niyas Mohammed is the CEO and Co-founder of Neuralcraft, where we build AI systems for high-stakes and regulated domains. If your team is navigating constraints like the ones in this post — restricted data access, sparse labels, regulatory boundaries between engineers and domain experts — we offer paid architecture reviews. Get in touch at neuralcraft.ai/contact.