Uses of Interface
org.extex.scanner.api.Tokenizer

Packages that use Tokenizer

org.extex.base.type.file	This package contains the basic file data types for ExTeX.
org.extex.interpreter.context	This package contains the container for data in the interpreter.
org.extex.interpreter.context.impl.extension
org.extex.interpreter.max.context	This package contains a reference implementation for the Context.
org.extex.scanner.api	This package contains the scanner for ExTeX.
org.extex.scanner.base	This package contains a basic implementation for a TokenStream.
org.extex.scanner.stream.impl32	This package contains the implementations for different TokenStream implementations.
org.extex.scanner.type.file

Uses of Tokenizer in org.extex.base.type.file

Methods in org.extex.base.type.file with parameters of type Tokenizer

Tokens

InputFile.read(TokenFactory factory, Tokenizer tokenizer) Read a line of characters into a tokens list.

Uses of Tokenizer in org.extex.interpreter.context

Subinterfaces of Tokenizer in org.extex.interpreter.context

interface	Context This interface describes the container for all data of an interpreter context.
interface	ContextInternals This interface describes the abilities of a Context together with some internal methods.

Methods in org.extex.interpreter.context that return Tokenizer

Tokenizer

Context.getTokenizer() Getter for the tokenizer.

Uses of Tokenizer in org.extex.interpreter.context.impl.extension

Subinterfaces of Tokenizer in org.extex.interpreter.context.impl.extension

interface

GroupExtension This is the implementation of a group object with ExTeX-functions.

Classes in org.extex.interpreter.context.impl.extension that implement Tokenizer

class	ContextExtensionImpl This is a reference implementation for an interpreter context with ExTeX functions.
class	GroupExtensionImpl This is a simple implementation for a group with ExTeX-functions.

Uses of Tokenizer in org.extex.interpreter.max.context

Subinterfaces of Tokenizer in org.extex.interpreter.max.context

interface

Group This is the implementation of a group object.

Classes in org.extex.interpreter.max.context that implement Tokenizer

class	ContextImpl This is a reference implementation for an interpreter context.
class	GroupImpl This is a simple implementation for a group.

Methods in org.extex.interpreter.max.context that return Tokenizer

Tokenizer

ContextImpl.getTokenizer() Getter for the tokenizer.

Uses of Tokenizer in org.extex.scanner.api

Methods in org.extex.scanner.api with parameters of type Tokenizer

Token

TokenStream.get(TokenFactory factory, Tokenizer tokenizer) Get the next token from the token stream.

Uses of Tokenizer in org.extex.scanner.base

Methods in org.extex.scanner.base with parameters of type Tokenizer

Token	TokenStreamBaseImpl.get(TokenFactory factory, Tokenizer tokenizer) Get the next token from the token stream.
protected UnicodeChar	TokenStreamImpl.getChar(Tokenizer tokenizer) Return the next character to process.
protected Token	TokenStreamImpl.getNext(TokenFactory factory, Tokenizer tokenizer) Get the next token when the stack is empty.
protected Token	TokenStreamBaseImpl.getNext(TokenFactory factory, Tokenizer tokenizer) Get the next token when the stack is empty.

Uses of Tokenizer in org.extex.scanner.stream.impl32

Methods in org.extex.scanner.stream.impl32 with parameters of type Tokenizer

protected UnicodeChar	TokenStreamImpl.getChar(Tokenizer tokenizer) Return the next character to process.
protected UnicodeChar	TokenStreamImpl32.getChar(Tokenizer tokenizer) Return the next character to process.
protected Token	TokenStreamImpl.getNext(TokenFactory factory, Tokenizer tokenizer) Get the next token when the stack is empty.

Uses of Tokenizer in org.extex.scanner.type.file

Methods in org.extex.scanner.type.file with parameters of type Tokenizer

Tokens

InFile.read(TokenFactory factory, Tokenizer tokenizer) Read a line of characters into a tokens list.