Automatic PII detection is the difference between reading forty pages line by line and checking a summary. It's also the part of redaction people trust most and understand least — so here's an honest account of what pattern detection can and cannot do.

The two kinds of sensitive data

Everything you might want removed falls into one of two categories, and they behave completely differently.

Structured data has a shape. A Social Security number is three digits, two digits, four digits. A credit card is 13–16 digits that satisfy a checksum. An email has an @ and a domain. These can be found reliably, because "reliably" means "matches a pattern that fake data rarely matches by accident."

Unstructured data has no shape. A person's name is just capitalised words. An address varies by country. A job title, a project codename, a medical condition — there is no pattern that separates these from ordinary prose.

Every honest claim about automatic redaction depends on which category you're in.

What can be detected automatically

Social Security numbers. 123-45-6789 and common variants. Detection also excludes ranges the SSA never issued, which removes a class of false positives from invoice numbers that happen to look similar.

Credit card numbers. Detected and then validated with the Luhn checksum — an arithmetic test every real card number satisfies and a random 16-digit string fails about 90% of the time. This matters more than it sounds: without it, order numbers and reference codes get redacted, and users lose trust in the tool. With it, a match is almost certainly a real card.

Email addresses. Highly reliable — the format is unambiguous.

Phone numbers. Reliable for standard formats, with the usual caveat that a ten-digit reference number can look like a US phone number. Worth a glance in the review step.

What can't be — and what to do instead

Names are the hard case. No pattern distinguishes "Michael Barnes" from "Media Services". Heuristics that match capitalised words over-redact aggressively — they'll black out your company name, the month, and the city.

The reliable path is to type the names you want removed. You know them; the detector doesn't. Supply "John Smith" and every occurrence goes, including a middle name on the same line — so "John Michael Smith" is fully removed rather than leaving "John Michael" behind. That last detail sounds pedantic until you see a document where only the surname was redacted.

Addresses, dates and identifiers vary too much to detect safely. Enter institution-specific formats — medical record numbers, matter numbers, account references — as custom terms.

Handwriting is not detected at all. OCR reads print, not cursive. A signature, an initialled margin note, a handwritten date: none of these will be found. On signed documents this is the single most common gap.

Why false positives are a real cost

It's tempting to think over-redaction is the safe error. It isn't, for two reasons.

A document with thirty spurious black bars is unusable — you've destroyed the thing you were trying to share. And a tool that cries wolf trains people to skim the review step, which is precisely when the real leak slips through.

This is why validation matters: Luhn for cards, issued-range checks for SSNs, explicit names rather than guesswork. A detector that's right almost every time gets read. One that's right half the time gets clicked past.

A workflow that holds up

Preview first. See counts by type before committing. Zero SSNs found on a document you know contains one means something is wrong — often that the page is a scan and needs OCR.
Add names and identifiers as terms. Every party, every reference number. This is where most of the real value is.
Redact, then read the output. Not the summary — the document.
Copy-paste test. Select all, copy, paste into a text editor, search for what should be gone.
Check images and metadata. A page with text can still hide a scanned insert, and document properties often carry the author's name.

Scanned documents change the calculus

On a scanned page the detector is only as good as the OCR that fed it. If OCR misreads a digit, the pattern doesn't match, no box is drawn, and nothing tells you.

We measured this on a phone photo of a passport: one widely used open-source OCR engine read zero words from the page, while a commercial service read 117 and located the name and date of birth precisely. Same image. The first would have returned the document untouched while reporting success.

So on scans, treat the review step as mandatory rather than good practice.

What to tell colleagues

Automatic detection is a first pass that finds the boring, high-volume, well-formed stuff — the SSNs and card numbers you'd otherwise hunt for manually across forty pages. It is not a substitute for someone who understands the document deciding what's sensitive.

Used that way it saves real time and catches things humans skim past. Trusted blindly, it produces confident-looking documents with one identifier left in.

Try it on a document and check the result — and if you handle financial statements or patient records, those pages cover the specifics.

How to Redact SSNs, Card Numbers and PII Automatically