Module: TokenStream

Module | TokenStream
In: | rdoc/tokenstream.rb

A TokenStream is a list of tokens, gathered during the parse of some entity (say a method). Entities populate these streams by being registered with the lexer. Any class can collect tokens by including TokenStream. From the outside, you use such an object by calling the start_collecting_tokens method, followed by calls to add_token and pop_token

### Methods

add_token add_tokens pop_token start_collecting_tokens token_stream

### Public Instance methods

add_token(tk)

[Source]

    # File rdoc/tokenstream.rb, line 16
      def add_token(tk)
        @token_stream << tk
      end

add_tokens(tks)

[Source]

    # File rdoc/tokenstream.rb, line 19
      def add_tokens(tks)
        tks.each  {|tk| add_token(tk)}
      end

pop_token()

[Source]

    # File rdoc/tokenstream.rb, line 22
      def pop_token
        @token_stream.pop
      end

start_collecting_tokens()

[Source]

    # File rdoc/tokenstream.rb, line 13
      def start_collecting_tokens
        @token_stream = []
      end

token_stream()

[Source]

    # File rdoc/tokenstream.rb, line 9
      def token_stream
        @token_stream
      end

[Validate]