Class: Liquid::Lexer

### Home Classes Methods

### In Files

- lib/liquid/lexer.rb

### Parent

Object

### Methods

- ::new
- #tokenize

### Class/Module Index

Quicksearch

- Liquid
- Liquid::ArgumentError
- Liquid::Assign
- Liquid::BlankFileSystem
- Liquid::Block
- Liquid::Break
- Liquid::BreakInterrupt
- Liquid::Capture
- Liquid::Case
- Liquid::Comment
- Liquid::Context
- Liquid::ContextError
- Liquid::Continue
- Liquid::ContinueInterrupt
- Liquid::Cycle
- Liquid::Decrement
- Liquid::Document
- Liquid::Drop
- Liquid::ElseCondition
- Liquid::Error
- Liquid::FileSystemError
- Liquid::FilterNotFound
- Liquid::For
- Liquid::I18n
- Liquid::I18n::TranslationError
- Liquid::If
- Liquid::Ifchanged
- Liquid::Include
- Liquid::Increment
- Liquid::Interrupt
- Liquid::Lexer
- Liquid::LocalFileSystem
- Liquid::MemoryError
- Liquid::Parser
- Liquid::Raw
- Liquid::StackLevelError
- Liquid::StandardError
- Liquid::StandardFilters
- Liquid::SyntaxError
- Liquid::TableRow
- Liquid::Tag
- Liquid::Template
- Liquid::Unless
- Liquid::Utils
- Liquid::Variable
- FalseClass
- LiquidView
- Module
- NilClass
- TrueClass

No matching classes.

# Liquid::Lexer

### Constants

COMPARISON_OPERATOR
DOUBLE_STRING_LITERAL
IDENTIFIER
NUMBER_LITERAL
SINGLE_STRING_LITERAL
SPECIALS

### Public Class Methods

new(input) click to toggle source

    # File lib/liquid/lexer.rb, line 20
    def initialize(input)
      @ss = StringScanner.new(input.rstrip)
    end

### Public Instance Methods

tokenize() click to toggle source

    # File lib/liquid/lexer.rb, line 24
    def tokenize
      @output = []

      while !@ss.eos?
        @ss.skip(/\s*/)
        tok = case
        when t = @ss.scan(COMPARISON_OPERATOR) then [:comparison, t]
        when t = @ss.scan(SINGLE_STRING_LITERAL) then [:string, t]
        when t = @ss.scan(DOUBLE_STRING_LITERAL) then [:string, t]
        when t = @ss.scan(NUMBER_LITERAL) then [:number, t]
        when t = @ss.scan(IDENTIFIER) then [:id, t]
        else
          c = @ss.getch
          if s = SPECIALS[c]
            [s,c]
          else
            raise SyntaxError, "Unexpected character #{c}"
          end
        end
        @output << tok
      end

      @output << [:end_of_string]
    end

[Validate]

Generated with the Darkfish Rdoc Generator 2.