, , ..., tags
$extor = $parser->extract(qr/^h[1-6]$/ => []);
while (my $chunk = $extor->()) {
push @headings, $chunk;
}
# all comments
$extor = $parser->extract(-COMMENT => []);
while (my $chunk = $extor->()) {
push @comments, $chunk;
}
`YAPE' MODULES
The `YAPE' hierarchy of modules is an attempt at a unified means
of parsing and extracting content. It attempts to maintain a
generic interface, to promote simplicity and reusability. The
API is powerful, yet simple. The modules do tokenization (which
can be intercepted) and build trees, so that extraction of
specific nodes is doable.
DESCRIPTION
This module is yet another parser and tree-builder for HTML
documents. It is designed to make extraction and modification of
HTML documents simplistic. The API allows for easy custom
additions to the document being parsed, and allows very specific
tag, text, and comment extraction.
USAGE
In addition to the base class, `YAPE::HTML', there is the
auxiliary class `YAPE::HTML::Element' (common to all `YAPE' base
classes) that holds the individual nodes' classes. There is
documentation for the node classes in that module's
documentation.
HTML elements and their attributes are stored internally as
lowercase strings. For clarification, that means that the tag
`' is stored as
{
TAG => 'a',
ATTR => {
href => 'FooBar.html',
}
}
This means that tags will be output in lowercase. There will be
a feature in a future version to switch output case to capital
letters.
Functions
* `YAPE::HTML::EMPTY(@tags)'
Adds to the internal hash of tags which never contain any
out-of-tag content. This hash is `%YAPE::HTML::EMPTY', and
contains the following tag names: `area', `base', `br',
`hr', `img', `input', `link', `meta', and `param'. Deletion
from this hashmust be done manually. Adding to this hash
automatically adds to the `%OPEN' hash, described next.
* `YAPE::HTML::OPEN(@tags)'
Adds to the internal hash of tags which do not require a
closing tag. This hash is `%YAPE::HTML::OPEN', and contains
the following tag names: `area', `base', `br', `dd', `dt',
`hr', `img', `input', `li', `link', `meta', `p', and
`param'. Deletion from this hash must be done manually.
There is a subtle difference between "empty" and "open"
tags. For example, the `' tag contains a few
attributes, but there is no text associated with it (nor any
other tags), and therefore, is "empty"; the `', on the
other hand,
It is strongly suggested that for ease in parsing, any tags
that you do not explicitly close have a `/' at the end of
the tag:
Here's my cat: 
Methods for `YAPE::HTML'
* `use YAPE::HTML;'
* `use YAPE::HTML qw( MyExt::Mod );'
If supplied no arguments, the module is loaded normally, and
the node classes are given the proper inheritence (from
`YAPE::HTML::Element'). If you supply a module (or list of
modules), `import' will automatically include them (if
needed) and set up *their* node classes with the proper
inheritence -- that is, it will append `YAPE::HTML' to
`@MyExt::Mod::ISA', and `YAPE::HTML::xxx' to each node
class's `@ISA' (where `xxx' is the name of the specific node
class).
It also copies the `%OPEN' and `%EMPTY' hashes, as well as
the `OPEN()' and `EMPTY()' functions, into the `MyExt::Mod'
namespace. This process is designed to save you from having
to place `@ISA' assignments all over the place.
It also copies the `%SSI' hash. This hash is not suggested
to be altered, and therefore it does not have any public
interface (you have to fiddle with it yourself). It exists
to ensure an SSI is valid.
package MyExt::Mod;
use YAPE::HTML 'MyExt::Mod';
# @MyExt::Mod::ISA = 'YAPE::HTML'
# @MyExt::Mod::text::ISA = 'YAPE::HTML::text'
# ...
# being rather strict with the tags
%OPEN = ();
%EMPTY = ();
* `my $p = YAPE::HTML->new($HTML, $strict);'
Creates a `YAPE::HTML' object, using the contents of the
`$HTML' string as its HTML to parse. The optional second
argument determines whether this parser instance will demand
strict comment parsing and require all tags to be closed
with a closing tag or a `/' at the end of the tag (`
'). Any true value (except for the special string `-
NO_STRICT') will turn strict parsing on. This is off by
default. (This could be considered a bug.)
* `my $text = $p->chunk($len);'
Returns the next `$len' characters in the input string;
`$len' defaults to 30 characters. This is useful for
figuring out why a parsing error occurs.
* `my $done = $p->done;'
Returns true if the parser is done with the input string,
and false otherwise.
* `my $errstr = $p->error;'
Returns the parser error message.
* `my $coderef = $p->extract(...);'
Returns a code reference that returns the next object that
matches the criteria given in the arguments. This is a
fundamental feature of the module, and you can extract that
from the section on "Extracting Sections".
* `my $node = $p->display(...);'
Returns a string representation of the entire content. It
calls the `parse' method in case there is more data that has
not yet been parsed. This calls the `fullstring' method on
the root nodes. Check the `YAPE::HTML::Element' docs on the
arguments to `fullstring'.
* `my $node = $p->next;'
Returns the next token, or `undef' if there is no valid
token. There will be an error message (accessible with the
`error' method) if there was a problem in the parsing.
* `my $node = $p->parse;'
Calls `next' until all the data has been parsed.
* `my $attr = $p->quote($string);'
Returns a quoted string, suitable for using as an attribute.
It turns any embedded `"' characters into `"'. This can
also be called as a raw function:
my $quoted = YAPE::HTML::quote($string);
* `my $root = $p->root;'
Returns an array reference holding the root of the tree
structure -- for documents that contain multiple top-level
tags, this will have more than one element.
* `my $state = $p->state;'
Returns the current state of the parser. It is one of the
following values: `close(TAG)', `comment', `done', `dtd',
`error', `open(TAG)', `pi', `ssi', `text', `text(script)',
or `text(xmp)'. The `open' and `close' states contain the
name of the element in parentheses (ex. `open(img)'). Tag
names, as well as the names of attributes, are converted to
lowercase. The state of `text(script)' refers to text found
inside an `
tags
$extor = $parser->extract(qr/^h[1-6]$/ => []);
while (my $chunk = $extor->()) {
push @headings, $chunk;
}
# all comments
$extor = $parser->extract(-COMMENT => []);
while (my $chunk = $extor->()) {
push @comments, $chunk;
}
`YAPE' MODULES
The `YAPE' hierarchy of modules is an attempt at a unified means
of parsing and extracting content. It attempts to maintain a
generic interface, to promote simplicity and reusability. The
API is powerful, yet simple. The modules do tokenization (which
can be intercepted) and build trees, so that extraction of
specific nodes is doable.
DESCRIPTION
This module is yet another parser and tree-builder for HTML
documents. It is designed to make extraction and modification of
HTML documents simplistic. The API allows for easy custom
additions to the document being parsed, and allows very specific
tag, text, and comment extraction.
USAGE
In addition to the base class, `YAPE::HTML', there is the
auxiliary class `YAPE::HTML::Element' (common to all `YAPE' base
classes) that holds the individual nodes' classes. There is
documentation for the node classes in that module's
documentation.
HTML elements and their attributes are stored internally as
lowercase strings. For clarification, that means that the tag
`' is stored as
{
TAG => 'a',
ATTR => {
href => 'FooBar.html',
}
}
This means that tags will be output in lowercase. There will be
a feature in a future version to switch output case to capital
letters.
Functions
* `YAPE::HTML::EMPTY(@tags)'
Adds to the internal hash of tags which never contain any
out-of-tag content. This hash is `%YAPE::HTML::EMPTY', and
contains the following tag names: `area', `base', `br',
`hr', `img', `input', `link', `meta', and `param'. Deletion
from this hashmust be done manually. Adding to this hash
automatically adds to the `%OPEN' hash, described next.
* `YAPE::HTML::OPEN(@tags)'
Adds to the internal hash of tags which do not require a
closing tag. This hash is `%YAPE::HTML::OPEN', and contains
the following tag names: `area', `base', `br', `dd', `dt',
`hr', `img', `input', `li', `link', `meta', `p', and
`param'. Deletion from this hash must be done manually.
There is a subtle difference between "empty" and "open"
tags. For example, the `' tag contains a few
attributes, but there is no text associated with it (nor any
other tags), and therefore, is "empty"; the `', on the
other hand,
It is strongly suggested that for ease in parsing, any tags
that you do not explicitly close have a `/' at the end of
the tag:
Here's my cat:
Methods for `YAPE::HTML'
* `use YAPE::HTML;'
* `use YAPE::HTML qw( MyExt::Mod );'
If supplied no arguments, the module is loaded normally, and
the node classes are given the proper inheritence (from
`YAPE::HTML::Element'). If you supply a module (or list of
modules), `import' will automatically include them (if
needed) and set up *their* node classes with the proper
inheritence -- that is, it will append `YAPE::HTML' to
`@MyExt::Mod::ISA', and `YAPE::HTML::xxx' to each node
class's `@ISA' (where `xxx' is the name of the specific node
class).
It also copies the `%OPEN' and `%EMPTY' hashes, as well as
the `OPEN()' and `EMPTY()' functions, into the `MyExt::Mod'
namespace. This process is designed to save you from having
to place `@ISA' assignments all over the place.
It also copies the `%SSI' hash. This hash is not suggested
to be altered, and therefore it does not have any public
interface (you have to fiddle with it yourself). It exists
to ensure an SSI is valid.
package MyExt::Mod;
use YAPE::HTML 'MyExt::Mod';
# @MyExt::Mod::ISA = 'YAPE::HTML'
# @MyExt::Mod::text::ISA = 'YAPE::HTML::text'
# ...
# being rather strict with the tags
%OPEN = ();
%EMPTY = ();
* `my $p = YAPE::HTML->new($HTML, $strict);'
Creates a `YAPE::HTML' object, using the contents of the
`$HTML' string as its HTML to parse. The optional second
argument determines whether this parser instance will demand
strict comment parsing and require all tags to be closed
with a closing tag or a `/' at the end of the tag (`
'). Any true value (except for the special string `-
NO_STRICT') will turn strict parsing on. This is off by
default. (This could be considered a bug.)
* `my $text = $p->chunk($len);'
Returns the next `$len' characters in the input string;
`$len' defaults to 30 characters. This is useful for
figuring out why a parsing error occurs.
* `my $done = $p->done;'
Returns true if the parser is done with the input string,
and false otherwise.
* `my $errstr = $p->error;'
Returns the parser error message.
* `my $coderef = $p->extract(...);'
Returns a code reference that returns the next object that
matches the criteria given in the arguments. This is a
fundamental feature of the module, and you can extract that
from the section on "Extracting Sections".
* `my $node = $p->display(...);'
Returns a string representation of the entire content. It
calls the `parse' method in case there is more data that has
not yet been parsed. This calls the `fullstring' method on
the root nodes. Check the `YAPE::HTML::Element' docs on the
arguments to `fullstring'.
* `my $node = $p->next;'
Returns the next token, or `undef' if there is no valid
token. There will be an error message (accessible with the
`error' method) if there was a problem in the parsing.
* `my $node = $p->parse;'
Calls `next' until all the data has been parsed.
* `my $attr = $p->quote($string);'
Returns a quoted string, suitable for using as an attribute.
It turns any embedded `"' characters into `"'. This can
also be called as a raw function:
my $quoted = YAPE::HTML::quote($string);
* `my $root = $p->root;'
Returns an array reference holding the root of the tree
structure -- for documents that contain multiple top-level
tags, this will have more than one element.
* `my $state = $p->state;'
Returns the current state of the parser. It is one of the
following values: `close(TAG)', `comment', `done', `dtd',
`error', `open(TAG)', `pi', `ssi', `text', `text(script)',
or `text(xmp)'. The `open' and `close' states contain the
name of the element in parentheses (ex. `open(img)'). Tag
names, as well as the names of attributes, are converted to
lowercase. The state of `text(script)' refers to text found
inside an `
Methods for `YAPE::HTML'
* `use YAPE::HTML;'
* `use YAPE::HTML qw( MyExt::Mod );'
If supplied no arguments, the module is loaded normally, and
the node classes are given the proper inheritence (from
`YAPE::HTML::Element'). If you supply a module (or list of
modules), `import' will automatically include them (if
needed) and set up *their* node classes with the proper
inheritence -- that is, it will append `YAPE::HTML' to
`@MyExt::Mod::ISA', and `YAPE::HTML::xxx' to each node
class's `@ISA' (where `xxx' is the name of the specific node
class).
It also copies the `%OPEN' and `%EMPTY' hashes, as well as
the `OPEN()' and `EMPTY()' functions, into the `MyExt::Mod'
namespace. This process is designed to save you from having
to place `@ISA' assignments all over the place.
It also copies the `%SSI' hash. This hash is not suggested
to be altered, and therefore it does not have any public
interface (you have to fiddle with it yourself). It exists
to ensure an SSI is valid.
package MyExt::Mod;
use YAPE::HTML 'MyExt::Mod';
# @MyExt::Mod::ISA = 'YAPE::HTML'
# @MyExt::Mod::text::ISA = 'YAPE::HTML::text'
# ...
# being rather strict with the tags
%OPEN = ();
%EMPTY = ();
* `my $p = YAPE::HTML->new($HTML, $strict);'
Creates a `YAPE::HTML' object, using the contents of the
`$HTML' string as its HTML to parse. The optional second
argument determines whether this parser instance will demand
strict comment parsing and require all tags to be closed
with a closing tag or a `/' at the end of the tag (`'). Any true value (except for the special string `- NO_STRICT') will turn strict parsing on. This is off by default. (This could be considered a bug.) * `my $text = $p->chunk($len);' Returns the next `$len' characters in the input string; `$len' defaults to 30 characters. This is useful for figuring out why a parsing error occurs. * `my $done = $p->done;' Returns true if the parser is done with the input string, and false otherwise. * `my $errstr = $p->error;' Returns the parser error message. * `my $coderef = $p->extract(...);' Returns a code reference that returns the next object that matches the criteria given in the arguments. This is a fundamental feature of the module, and you can extract that from the section on "Extracting Sections". * `my $node = $p->display(...);' Returns a string representation of the entire content. It calls the `parse' method in case there is more data that has not yet been parsed. This calls the `fullstring' method on the root nodes. Check the `YAPE::HTML::Element' docs on the arguments to `fullstring'. * `my $node = $p->next;' Returns the next token, or `undef' if there is no valid token. There will be an error message (accessible with the `error' method) if there was a problem in the parsing. * `my $node = $p->parse;' Calls `next' until all the data has been parsed. * `my $attr = $p->quote($string);' Returns a quoted string, suitable for using as an attribute. It turns any embedded `"' characters into `"'. This can also be called as a raw function: my $quoted = YAPE::HTML::quote($string); * `my $root = $p->root;' Returns an array reference holding the root of the tree structure -- for documents that contain multiple top-level tags, this will have more than one element. * `my $state = $p->state;' Returns the current state of the parser. It is one of the following values: `close(TAG)', `comment', `done', `dtd', `error', `open(TAG)', `pi', `ssi', `text', `text(script)', or `text(xmp)'. The `open' and `close' states contain the name of the element in parentheses (ex. `open(img)'). Tag names, as well as the names of attributes, are converted to lowercase. The state of `text(script)' refers to text found inside an `