Building a chi-squared Caesar cipher cracker

Here's the annoying thing about the Caesar cipher: everyone's first instinct is to just brute-force it. There are only 26 possible shifts, so you try all of them, print the results, and eyeball which one reads like English.

That works. It's also a little bit of a cop-out — you're outsourcing the "which one is English" decision to a human, which means the moment you want to automate it, you're stuck.

So the actual question isn't "how do I decode this," it's "how do I make a computer recognize English without reading it?"

Letters gossip

Turns out: letters gossip.

In any large enough sample of English text, letters don't show up with equal frequency. E shows up constantly — around 12–13% of all letters. Z, Q, and X barely show up at all. This isn't a coincidence; it's a fingerprint.

Every language has one, and it's stable enough that if you shift the entire alphabet by some amount N, that fingerprint shifts with it, intact, just rotated.

That's the whole trick. You don't need to read the decoded text to know if it's English — you just need to check whether its letter-frequency fingerprint matches English's fingerprint.

The chi-squared approach

The tool for measuring how far a letter-frequency fingerprint is from the expected English fingerprint already exists in statistics: the chi-squared statistic.

For each of the 26 possible shifts:

1. Decode the ciphertext assuming that shift.
2. Count how often each letter from A–Z appears.
3. Compare the count with standard English frequencies.
4. Calculate the chi-squared score.
5. Keep the shift with the lowest score.

for shift in 0..25:
    decoded = caesar_decode(ciphertext, shift)
    freq = letter_frequencies(decoded)
    score = chi_squared(freq, ENGLISH_FREQ)
    track best (lowest score)

Lower chi-squared means the decoded text is closer to normal English. The shift with the lowest score wins.

No brute-force eyeballing and no human in the loop — the statistics point at the answer.

Where it actually breaks

Short ciphertexts.

Chi-squared needs enough letters for the fingerprint to stabilize. Feed it a 12-character message and the fingerprint is mostly noise, so the statistic can confidently point at the wrong shift.

The next thing to fix is either to fall back to brute-force-with-dictionary-check below some length threshold, or to weight the score by a confidence interval that widens as the ciphertext gets shorter.

Also shipped alongside this

Base32 detection and single-byte XOR brute force — because in an actual CTF challenge, half the battle isn't decoding. It's figuring out what encoding you're even staring at before you can decode anything.