org.extex.scanner.stream.impl32
Class TokenStreamImpl32

java.lang.Object
  org.extex.scanner.base.TokenStreamBaseImpl
      org.extex.scanner.stream.impl32.TokenStreamImpl
          org.extex.scanner.stream.impl32.TokenStreamImpl32

All Implemented Interfaces:

TokenStream

public class TokenStreamImpl32
extends TokenStreamImpl

This class contains an implementation of a token stream which is fed from a Reader.

• '^^^UnicodeName;' use the Unicode name for the character
• '^^^^ab07' a la Omega
  It use 4 hex digit or less, if a non hex digit are found.

Version:

$Revision:4725 $

Author:

Michael Niedermair

See Also:

TokenStreamImpl

Field Summary

Fields inherited from class org.extex.scanner.stream.impl32.TokenStreamImpl

CARET_LIMIT, MID_LINE, NEW_LINE, pointer, saveChar, SKIP_BLANKS, state

Constructor Summary

TokenStreamImpl32(Configuration config, TokenStreamOptions options, java.io.Reader reader, java.lang.Boolean isFile, java.lang.String theSource)
Creates a new object.

Method Summary

protected UnicodeChar

getChar(Tokenizer tokenizer) Return the next character to process.

Methods inherited from class org.extex.scanner.stream.impl32.TokenStreamImpl

atEndOfLine, endLine, getLocator, getNext, getRawChar, hex2int, isEof, isEol, refill, toString

Methods inherited from class org.extex.scanner.base.TokenStreamBaseImpl

closeFileStream, get, isFileStream, put, skipSpaces

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.extex.scanner.api.TokenStream

closeFileStream, get, isFileStream, put

Constructor Detail

TokenStreamImpl32

public TokenStreamImpl32(Configuration config,
                         TokenStreamOptions options,
                         java.io.Reader reader,
                         java.lang.Boolean isFile,
                         java.lang.String theSource)
                  throws java.io.IOException

Creates a new object.

Parameters:

config - the configuration object for this instance; This configuration is ignored in this implementation.

options - ignored here

reader - the reader

isFile - indicator for file streams

theSource - the description of the input source

Throws:

java.io.IOException - in case of an IO error

Method Detail

getChar

protected UnicodeChar getChar(Tokenizer tokenizer)
                       throws ScannerException

Return the next character to process. The pointer is advanced and points to the character returned.

This operation might involve that an additional bunch of characters is read in (with refill()).

Overrides:

getChar in class TokenStreamImpl

Parameters:

tokenizer - the classifier for characters

Returns:

the character or null if no more character is available

Throws:

ScannerException - in the rare case that an IO Exception has occurred.

See Also:

TokenStreamImpl.getChar( org.extex.scanner.api.Tokenizer)