There’s a stage in every text-parsing problem where the regex stops being the right tool. You’ve got a pattern with four alternations and three lookaheads, it’s slow, it’s unreadable, and it still can’t handle nesting. StringScanner is the standard-library answer — a cursor over a string that matches anchored patterns and advances, which turns an unmaintainable regex into a readable loop.

The Model

A StringScanner holds a string and a position. Every scan attempt is anchored at the current position, and a successful scan advances the cursor past what it matched.

Example:

require "strscan"

scanner = StringScanner.new("name: Ada Lovelace")

scanner.scan(/\w+/)     # => "name"
scanner.scan(/:\s*/)    # => ": "
scanner.scan(/.+/)      # => "Ada Lovelace"
scanner.eos?            # => true

Anchoring is the key property. scan never searches forward — it either matches right here or returns nil. That makes the parse deterministic and linear, and it’s why StringScanner doesn’t suffer the catastrophic backtracking that kills a complex regex on adversarial input.

The Core Methods

check is the one that makes real parsers possible — you can look at what’s coming and decide which branch to take without committing.

A Real Parser: Log Lines

Something structured but irregular, where a single regex gets ugly.

Input:

2026-07-09T14:22:09Z [ERROR] payment.charge account_id=4821 amount=2450 retry=false msg="card declined by issuer"

Example:

require "strscan"

class LogLineParser
  Entry = Data.define(:timestamp, :level, :event, :fields, :message)

  def parse(line)
    s = StringScanner.new(line)

    timestamp = s.scan(/\d{4}-\d{2}-\d{2}T[\d:]+Z/) or raise_at(s, "timestamp")
    s.skip(/\s+/)

    s.skip(/\[/) or raise_at(s, "[")
    level = s.scan(/[A-Z]+/) or raise_at(s, "level")
    s.skip(/\]\s*/)

    event = s.scan(/[\w.]+/) or raise_at(s, "event name")
    s.skip(/\s*/)

    fields = {}
    message = nil

    until s.eos?
      key = s.scan(/[a-z_]+/) or break
      s.skip(/=/) or raise_at(s, "=")

      value = scan_value(s)
      key == "msg" ? message = value : fields[key.to_sym] = coerce(value)

      s.skip(/\s*/)
    end

    Entry.new(timestamp:, level:, event:, fields:, message:)
  end

  private

  def scan_value(s)
    if s.skip(/"/)
      value = s.scan(/(?:[^"\\]|\\.)*/)
      s.skip(/"/) or raise_at(s, "closing quote")
      value.gsub(/\\(.)/, '\1')
    else
      s.scan(/\S+/)
    end
  end

  def coerce(value)
    case value
    when /\A-?\d+\z/       then value.to_i
    when /\A-?\d+\.\d+\z/  then value.to_f
    when "true"            then true
    when "false"           then false
    else value
    end
  end

  def raise_at(s, expected)
    raise ArgumentError,
          "expected #{expected} at position #{s.pos}: #{s.rest[0, 40].inspect}"
  end
end

Example:

LogLineParser.new.parse(line)
# => #<data Entry timestamp="2026-07-09T14:22:09Z", level="ERROR",
#      event="payment.charge",
#      fields={account_id: 4821, amount: 2450, retry: false},
#      message="card declined by issuer">

Compare that to the equivalent single regex. The scanner version handles quoted strings with escapes, arbitrary field order, and unknown keys — and when it fails, raise_at tells you the exact position and what it expected. A failing regex tells you nil.

Handling Nesting

Nesting is the thing regex genuinely cannot do, and where a scanner becomes not just nicer but necessary.

Example:

require "strscan"

class BracketParser
  def parse(input)
    @s = StringScanner.new(input)
    result = parse_group
    raise "unexpected #{@s.rest[0, 20].inspect}" unless @s.eos?
    result
  end

  private

  def parse_group
    items = []

    until @s.eos? || @s.check(/\)/)
      @s.skip(/\s*/)

      if @s.skip(/\(/)
        items << parse_group
        @s.skip(/\)/) or raise "unclosed group at #{@s.pos}"
      elsif (word = @s.scan(/[\w.]+/))
        items << word
      elsif @s.check(/\)/) || @s.eos?
        break
      else
        raise "unexpected character #{@s.getch.inspect} at #{@s.pos}"
      end

      @s.skip(/\s*/)
    end

    items
  end
end

BracketParser.new.parse("a (b (c d) e) f")
# => ["a", ["b", ["c", "d"], "e"], "f"]

Recursive descent, twenty-five lines, and it reports the position of a syntax error. This is the shape of every hand-written parser you’ll ever need to build — a method per grammar rule, check to decide which branch, scan to consume.

Tokenizing With a Dispatch Table

For a lexer, a table of pattern-to-token-type reads better than a chain of elsif.

Example:

require "strscan"

TOKENS = [
  [:whitespace, /\s+/],
  [:number,     /\d+(?:\.\d+)?/],
  [:string,     /"(?:[^"\\]|\\.)*"/],
  [:operator,   /[+\-*\/=<>!]+/],
  [:lparen,     /\(/],
  [:rparen,     /\)/],
  [:identifier, /[a-zA-Z_]\w*/]
].freeze

def tokenize(source)
  s = StringScanner.new(source)
  tokens = []

  until s.eos?
    type, value = TOKENS.find { |_, pattern| (v = s.scan(pattern)) && (break [_1, v]) }
    raise "unexpected #{s.getch.inspect} at #{s.pos}" if type.nil?

    tokens << [type, value] unless type == :whitespace
  end

  tokens
end

Order in the table matters — longer or more specific patterns first, or identifier will swallow a keyword you meant to tokenize separately.

Performance Notes

StringScanner is implemented in C and is fast, but the naive usage patterns have real costs.

• Compile patterns once. A regex literal inside a loop is re-created on every iteration in some contexts. Hoist them into frozen constants.
• skip beats scan when you discard the result — it returns a length rather than allocating a matched string. On a hot tokenizer, skipping whitespace with skip instead of scan measurably reduces allocations.
• Avoid rest in a loop. It allocates a new string containing everything after the cursor, so calling it per iteration is quadratic. Use it only for error messages.
• Anchoring is the win. Because every match is anchored, patterns can’t backtrack across the whole input. A regex with nested quantifiers that takes exponential time on a crafted string takes linear time under a scanner, since each attempt is bounded at the current position.

That last point is a security property, not just a performance one. Untrusted input plus a complex regex is a ReDoS vector; the same grammar expressed as a scanner is not.

Pro-Tip: Track line and column for error messages from the start, not after someone complains. s.pos gives you a byte offset, which is useless in a message a human has to act on. Compute the position lazily only when raising: line = s.string[0, s.pos].count("\n") + 1 and col = s.pos - (s.string.rindex("\n", s.pos - 1) || -1). Doing it lazily costs nothing on the happy path — the scan never touches it — and turns expected ")" at position 4127 into line 88, col 12: expected ")", which is the difference between an error someone can fix and one they have to bisect. Add s.string.lines[line - 1] and a caret under the column and your parser produces better diagnostics than most gems.

When Not to Use It

• The format has a parser already. JSON, YAML, CSV, HTTP, URIs — use the standard library. Hand-parsing a standard format is how you get subtly wrong edge-case handling.
• A single simple regex works. line.match(/^(\w+): (.+)$/) is clearer than three scanner calls.
• The grammar is large. Past a few hundred lines of hand-written parser, a proper generator like racc (in the standard library) or parslet gives you a grammar you can read as a grammar.

StringScanner occupies the middle: too irregular for one regex, too small for a parser generator. That middle is where most real text-munging lives — log formats, DSLs, template syntax, query strings, custom config.

Conclusion

StringScanner turns parsing from pattern-matching into a loop with a cursor, which is a fundamentally more debuggable shape: you can print the position, look at what’s next with check, and produce an error that names what was expected. It handles nesting that regex cannot, it’s immune to catastrophic backtracking because every match is anchored, and it’s already installed. Reach for it the moment a regex grows a second alternation group, and invest ten minutes in line-and-column error reporting — the parser you write today gets debugged by someone reading its error messages.

FAQs

Q1: Does StringScanner handle multibyte strings correctly?
Yes, it’s encoding-aware and pos is a byte offset while patterns match characters. Mixing manual byte arithmetic with pos on UTF-8 input is where people go wrong — use charpos when you need character offsets.

Q2: What’s the difference between scan and scan_until?
scan is anchored at the cursor and fails if the pattern isn’t right there. scan_until searches forward and consumes everything up to and including the match. Use scan for parsing; scan_until for skipping to a delimiter.

Q3: Can I backtrack?
Yes — save pos before a speculative parse and restore it on failure. That’s how you implement lookahead beyond what check gives you.

Q4: Is StringScanner thread-safe?
A single instance is not — it has mutable position state. Create one per thread or per parse; they’re cheap.

Q5: How does it compare to String#split for simple cases?
split is faster and clearer when the input really is delimiter-separated with no quoting or escaping. The moment you need to handle a delimiter inside quotes, split stops working and a scanner starts.