Skip to content

Lesson 9 of 9 · 20m

Knowing When Not to Use Regex

The formats regex can't parse, the security bugs that come from trying, and a decision checklist for regex versus parser versus pipeline.

"Some people, when confronted with a problem, think 'I know, I'll use regular expressions.' Now they have two problems." Jamie Zawinski's 1997 line gets quoted as a joke, but there's an engineering point under it. Regular expressions recognize regular languages. A lot of the data you'll be asked to process isn't one. Knowing where that boundary sits is the difference between a senior engineer's regex and a CVE.

Regex can't count nesting

A regular language can't track unbounded nesting depth: balanced parentheses, nested HTML tags, JSON objects inside arrays inside objects, block comments inside block comments. PCRE's recursion ((?R)) technically gets around this, but then you're writing a parser in the least maintainable language available. If the format nests, use a parser.

Formats that already have parsers

DataUse thisNot this
HTMLDOMParser, BeautifulSoup, html5lib<a href="(.*?)">
JSONJSON.parse, jq"name":\s*"([^"]*)"
CSVPython csv, Miller (mlr), qsvsplit(",")
URLsnew URL(), urllib.parse^https?://([^/]+)
Email addressescheck for one @, then send a confirmation linkthe 6 KB RFC 5322 regex

CSV looks regular until it isn't

const row = '42,"Smith, Jane","say ""hi"" to Bob",2025-03-14';
console.log(row.split(",").length);   // 5: the quoted comma split a field

function parseCsvLine(s) {
  const out = []; let cur = ""; let quoted = false;
  for (let i = 0; i < s.length; i++) {
    const c = s[i];
    if (quoted) {
      if (c === '"' && s[i + 1] === '"') { cur += '"'; i++; }
      else if (c === '"') quoted = false;
      else cur += c;
    } else if (c === '"') quoted = true;
    else if (c === ",") { out.push(cur); cur = ""; }
    else cur += c;
  }
  out.push(cur);
  return out;
}
console.log(parseCsvLine(row));  // ["42", "Smith, Jane", 'say "hi" to Bob', "2025-03-14"]

Even this state machine is incomplete: RFC 4180 allows newlines inside quoted fields, so "one record per line" is also false. Use a real CSV library.

The security version: URL allowlists

const allow = /^https:\/\/sudosecurity\.org/;
for (const target of ["https://sudosecurity.org.evil.test/", "https://sudosecurity.org@evil.test/"]) {
  console.log(allow.test(target), new URL(target).hostname);
}
// true evil.test   true evil.test   — both pass the regex, both go somewhere else

Regex-based redirect and SSRF allowlists are a recurring source of real vulnerabilities. Parse the URL, then compare hostname exactly against a set.

Where regex is the right call

  • Tokenizing inside a real parser. Each token is a small anchored regex, and JS's sticky y flag exists for this.
  • Extracting fields from a log format you control or have documented.
  • Validating narrow, flat formats: ^[a-z0-9-]{1,63}$ for a slug, ^[0-9a-f]{40}$ for a git SHA.

Field notes

  • Ask in order: Is there a parser for this format? Does it nest? Is the input untrusted? Will someone else maintain this? Any "yes" pushes you away from a clever regex.
  • Every production regex deserves a table-driven test with matches, non-matches, and near-misses.
  • Prefer URL, Intl, ipaddress, and datetime. They encode decades of edge cases you won't remember.