LaTeXML::Token3pm
Representation of a token, and "latexml::tokens", representing lists of tokens.
Overview
This module defines Tokens ("LaTeXML::Token", "LaTeXML::Tokens") that get created during tokenization and expansion.
A "LaTeXML::Token" represents a TeX token which is a pair of a character or string and a category code. A "LaTeXML::Tokens" is a list of tokens (and also implements the API of a LaTeXML::Mouth so that tokens can be read from a list).
Common methods
The following methods apply to all objects.
"@tokens = $object->unlist;"
|
Return a list of the tokens making up this |
|---|---|
"$string = $object->toString;"
|
Return a string representing |
Token methods
The following methods are specific to "LaTeXML::Token".
- "$string = $token->getCSName;"
-
Return the string or character part of the
$token; for the special category codes, returns the standard string (eg."T_BEGIN-"getCSName> returns "{").
- "$string = $token->getString;"
-
Return the string or character part of the
$token.
- "$code = $token->getCharcode;"
-
Return the character code of the character part of the
$token, or 256 if it is a control sequence.
- "$code = $token->getCatcode;"
-
Return the catcode of the
$token.
Tokens methods
The following methods are specific to "LaTeXML::Tokens".
- "$tokenscopy = $tokens->clone;"
-
Return a shallow copy of the
$tokens. This is useful before reading from a"LaTeXML::Tokens".
- "$token = $tokens->readToken;"
-
Returns (and remove) the next token from
$tokens. This is part of the public API of LaTeXML::Mouth so that a"LaTeXML::Tokens"can serve as a LaTeXML::Mouth.
Releases
Unchanged in 2 releases since Debian 6.0 (Squeeze).