Flexible rdfa parser
If you're wanting to work with an RDF::Trine::Model that can be queried with \s-1SPARQL\s0, etc:
use RDF::RDFa::Parser;
my $url = 'http://example.com/document.html';
my $options = RDF::RDFa::Parser::Config->new('xhtml', '1.1');
my $rdfa = RDF::RDFa::Parser->new_from_url($url, $options);
my $model = $rdfa->graph;
For dealing with local data:
use RDF::RDFa::Parser; my $base_url = 'http://example.com/document.html'; my $options = RDF::RDFa::Parser::Config->new('xhtml', '1.1'); my $rdfa = RDF::RDFa::Parser->new($markup, $base_url, $options); my $model = $rdfa->graph;
A simple set of operations for working with Open Graph Protocol data:
use RDF::RDFa::Parser; my $url = 'http://www.rottentomatoes.com/m/net/'; my $options = RDF::RDFa::Parser::Config->tagsoup; my $rdfa = RDF::RDFa::Parser->new_from_url($url, $options); print $rdfa->opengraph('title') . "\n"; print $rdfa->opengraph('image') . "\n";
RDF::TrineX::Parser::RDFa provides a saner interface for this module. If you are new to parsing RDFa with Perl, then that's the best place to start.
Some of the logic regarding host language and RDFa version guessing is likely to be removed from RDF::RDFa::Parser and RDF::RDFa::Parser::Config, and shifted into RDF::TrineX::Parser::RDFa instead.
This method creates a new RDF::RDFa::Parser object and returns it. The $markup variable may contain an \s-1XHTML/XML\s0 string, or a XML::LibXML::Document. If a string, the document is parsed using XML::LibXML::Parser or HTML::HTML5::Parser, depending on the configuration in $config. \s-1XML\s0 well-formedness errors will cause the function to die. $base is a \s-1URL\s0 used to resolve relative links found in the document. $config optionally holds an RDF::RDFa::Parser::Config object which determines the set of rules used to parse the RDFa. It defaults to XHTML+RDFa 1.1. Advanced usage note: $storage optionally holds an RDF::Trine::Store object. If undef, then a new temporary store is created. $url is a \s-1URL\s0 to fetch and parse, or an HTTP::Response object. $config optionally holds an RDF::RDFa::Parser::Config object which determines the set of rules used to parse the RDFa. The default is to determine the configuration by looking at the \s-1HTTP\s0 response Content-Type header; it's probably sensible to keep the default. $storage optionally holds an RDF::Trine::Store object. If undef, then a new temporary store is created. This function can also be called as \*(C`new_from_url\*(C' or \*(C`new_from_uri\*(C'. Same thing. $response is an \*(C`HTTP::Response\*(C' object. Otherwise the same as \*(C`new_from_url\*(C'.
This will return an RDF::Trine::Model containing all the RDFa data found on the page. Advanced usage note: If passed a graph \s-1URI\s0 as a parameter, will return a single named graph from within the page. This feature is only useful if you're using named graphs. Advanced usage only. Will return a hashref of all named graphs, where the graph name is a key and the value is a RDF::Trine::Model tied to a temporary storage. This method is only useful if you're using named graphs. If $property is provided, will return the value or list of values (if called in list context) for that Open Graph Protocol property. (In pure \s-1RDF\s0 terms, it returns the non-bnode objects of triples where the subject is the document base \s-1URI\s0; and the predicate is $property, with non-URI $property strings taken as having the implicit prefix 'http://ogp.me/ns#'. There is no distinction between literal and non-literal values; literal datatypes and languages are dropped.) If $property is omitted, returns a list of possible properties. Example: foreach my $property (sort $p->opengraph) { print "$property :\n"; foreach my $val (sort $p->opengraph($property)) { print " * $val\n"; } } See also: <http://opengraphprotocol.org/>. Returns the parsed XML::LibXML::Document. Returns the base \s-1URI\s0 of the document being parsed. This will usually be the same as the base \s-1URI\s0 provided to the constructor, but may differ if the document contains a <base> \s-1HTML\s0 element. Optionally it may be passed a parameter - an absolute or relative \s-1URI\s0 - in which case it returns the same \s-1URI\s0 which it was passed as a parameter, but as an absolute \s-1URI\s0, resolved relative to the document's base \s-1URI\s0. This seems like two unrelated functions, but if you consider the consequence of passing a relative \s-1URI\s0 consisting of a zero-length string, it in fact makes sense. Returns a list of errors and warnings that occurred during parsing. As per \*(C`$p->errors\*(C' but returns data as an \s-1RDF\s0 model. An alias for \*(C`graph\*(C', but does not accept a parameter. Union of the above two graphs. Advanced usage only. The document is parsed for RDFa. As of RDF::RDFa::Parser 1.09x, this is called automatically when needed; you probably don't need to touch it unless you're doing interesting things with callbacks. Calling \*(C`$p->consume(survive => 1)\*(C' will avoid crashing (e.g. when the markup provided cannot be parsed), and instead make more errors available in \*(C`$p->errors\*(C'. Advanced usage only. Set callback functions for the parser to call on certain events. These are only necessary if you want to do something especially unusual. $p->set_callbacks({ 'pretriple_resource' => sub { ... } , 'pretriple_literal' => sub { ... } , 'ontriple' => undef , 'onprefix' => \&some_function , }); Either of the two pretriple callbacks can be set to the string 'print' instead of a coderef. This enables built-in callbacks for printing Turtle to \s-1STDOUT\s0. For details of the callback functions, see the section \s-1CALLBACKS\s0. If used, \*(C`set_callbacks\*(C' must be called before \*(C`consume\*(C'. \*(C`set_callbacks\*(C' returns a reference to the parser object itself. Advanced usage only. Gets/sets a hashref of { xpath => RDF::Trine::Node } mappings. This is not touched during normal RDFa parsing, only being used by the @role and @cite features where \s-1RDF\s0 resources (i.e. URIs and blank nodes) are needed to represent \s-1XML\s0 elements themselves.
Several callback functions are provided. These may be set using the \*(C`set_callbacks\*(C' function, which takes a hashref of keys pointing to coderefs. The keys are named for the event to fire the callback on.
This is called once a triple is ready to be added to the graph. (After the pretriple callbacks.) The parameters passed to the callback function are:
A reference to the \*(C`RDF::RDFa::Parser\*(C' object
A hashref of relevant \*(C`XML::LibXML::Element\*(C' objects (subject, predicate, object, graph, current)
An RDF::Trine::Statement object.
The callback should return 1 to tell the parser to skip this triple (not add it to the graph); return 0 otherwise. The callback may modify the RDF::Trine::Statement object.
This is called when a new \s-1CURIE\s0 prefix is discovered. The parameters passed to the callback function are:
A reference to the \*(C`RDF::RDFa::Parser\*(C' object
A reference to the \*(C`XML::LibXML::Element\*(C' being parsed
The prefix (string, e.g. \*(L"foaf\*(R")
The expanded \s-1URI\s0 (string, e.g. \*(L"http://xmlns.com/foaf/0.1/\*(R")
The return value of this callback is currently ignored, but you should return 0 in case future versions of this module assign significance to the return value.
This is called when a \s-1CURIE\s0 or term has been expanded. The parameters are:
A reference to the \*(C`RDF::RDFa::Parser\*(C' object
A reference to the \*(C`XML::LibXML::Element\*(C' being parsed
The \s-1CURIE\s0 or token as a string (e.g. \*(L"foaf:name\*(R" or \*(L"Stylesheet\*(R")
The fully expanded \s-1URI\s0
The callback function must return a fully expanded \s-1URI\s0, or if it wants the \s-1CURIE\s0 to be ignored, undef.
This is called when an error occurs:
A reference to the \*(C`RDF::RDFa::Parser\*(C' object
The error level (RDF::RDFa::Parser::ERR_ERROR or RDF::RDFa::Parser::ERR_WARNING)
An error code
An error message
A hash of other information
The return value of this callback is currently ignored, but you should return 0 in case future versions of this module assign significance to the return value.
If you do not define an onerror callback, then errors will be output via \s-1STDERR\s0 and warnings will be silent. Either way, you can retrieve errors after parsing using the \*(C`errors\*(C' method.
This callback is deprecated - use ontriple instead.
This is called when a triple has been found, but before preparing the triple for adding to the model. It is only called for triples with a non-literal object value.
The parameters passed to the callback function are:
A reference to the \*(C`RDF::RDFa::Parser\*(C' object
A reference to the \*(C`XML::LibXML::Element\*(C' being parsed
Subject \s-1URI\s0 or bnode (string)
Predicate \s-1URI\s0 (string)
Object \s-1URI\s0 or bnode (string)
Graph \s-1URI\s0 or bnode (string or undef)
The callback should return 1 to tell the parser to skip this triple (not add it to the graph); return 0 otherwise.
This callback is deprecated - use ontriple instead.
This is the equivalent of pretriple_resource, but is only called for triples with a literal object value.
The parameters passed to the callback function are:
A reference to the \*(C`RDF::RDFa::Parser\*(C' object
A reference to the \*(C`XML::LibXML::Element\*(C' being parsed
Subject \s-1URI\s0 or bnode (string)
Predicate \s-1URI\s0 (string)
Object literal (string)
Datatype \s-1URI\s0 (string or undef)
Language (string or undef)
Graph \s-1URI\s0 or bnode (string or undef)
Beware: sometimes both a datatype and a language will be passed. This goes beyond the normal \s-1RDF\s0 data model.)
The callback should return 1 to tell the parser to skip this triple (not add it to the graph); return 0 otherwise.
Most features are configurable using RDF::RDFa::Parser::Config.
RDF::RDFa::Parser supports RDFa versions 1.0 and 1.1.
1.1 is currently a moving target; support is experimental.
1.1 is the default, but this can be configured using RDF::RDFa::Parser::Config.
RDF::RDFa::Parser supports various different RDFa host languages:
\s-1XHTML\s0 As per the XHTML+RDFa 1.0 and XHTML+RDFa 1.1 specifications.
\s-1HTML\s0 4 Uses an \s-1HTML5\s0 (sic) parser; uses @lang instead of @xml:lang; keeps prefixes and terms case-insensitive; recognises the @rel relations defined in the \s-1HTML\s0 4 specification. Otherwise the same as \s-1XHTML\s0.
\s-1HTML5\s0 Uses an \s-1HTML5\s0 parser; uses @lang as well as @xml:lang; keeps prefixes and terms case-insensitive; recognises the @rel relations defined in the \s-1HTML5\s0 draft specification. Otherwise the same as \s-1XHTML\s0.
\s-1XML\s0 This is implemented as per the RDFa Core 1.1 specification. There is also support for \*(L"RDFa Core 1.0\*(R", for which no specification exists, but has been reverse-engineered by applying the differences between XHTML+RDFa 1.1 and RDFa Core 1.1 to the XHTML+RDFa 1.0 specification. Embedded chunks of \s-1RDF/XML\s0 within \s-1XML\s0 are supported.
\s-1SVG\s0 For now, a synonym for \s-1XML\s0.
Atom The <feed> and <entry> elements are treated specially, setting a new subject; IANA-registered rel keywords are recognised. By passing \*(C`atom_parser=>1\*(C' as a Config option, you can also handle Atom's native semantics. (Uses XML::Atom::OWL. If this module is not installed, this option is silently ignored.) Otherwise, the same as \s-1XML\s0.
DataRSS Defines some default prefixes. Otherwise, the same as Atom.
OpenDocument \s-1XML\s0 That is, \s-1XML\s0 content formatted along the lines of 'content.xml' in OpenDocument files. Supports OpenDocument bookmarked ranges used as typed or plain object literals (though not \s-1XML\s0 literals); expects RDFa attributes in the \s-1XHTML\s0 namespace instead of in no namespace. Otherwise, the same as \s-1XML\s0.
OpenDocument That is, a \s-1ZIP\s0 file containing OpenDocument \s-1XML\s0 files. RDF::RDFa::Parser will do all the unzipping and combining for you, so you don't have to. The unregistered \*(L"jar:\*(R" \s-1URI\s0 scheme is used to refer to files within the \s-1ZIP\s0.
Though a rarely used feature, \s-1XHTML\s0 allows other \s-1XML\s0 markup languages to be directly embedded into it. In particular, chunks of \s-1RDF/XML\s0 can be included in \s-1XHTML\s0. While this is not common in \s-1XHTML\s0, it's seen quite often in \s-1SVG\s0 and other \s-1XML\s0 markup languages.
When RDF::RDFa::Parser encounters a chunk of \s-1RDF/XML\s0 in a document it's parsing (i.e. an element called '\s-1RDF\s0' with namespace 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'), there are three different courses of action it can take:
This is the behaviour that XHTML+RDFa seems to suggest is the right option. It should mostly not do any harm: triples encoded in \s-1RDF/XML\s0 will be generally ignored (though the chunk itself could theoretically end up as part of an \s-1XML\s0 literal). It will waste a bit of time though.
The parser will parse the \s-1RDF/XML\s0 properly. If named graphs are enabled, any triples will be added to a separate graph. This is the behaviour that \s-1SVG\s0 Tiny 1.2 seems to suggest is the correct thing to do.
This will skip over the \s-1RDF\s0 element entirely, and thus save you a bit of time.
You can decide which path to take by setting the 'embedded_rdfxml' Config option. For \s-1HTML\s0 and \s-1XHTML\s0, you probably want to set embedded_rdfxml to '0' (the default) or '2' (a little faster). For other \s-1XML\s0 markup languages (e.g. \s-1SVG\s0 or Atom), then you probably want to set it to '1'.
(There's also an option '3' which controls how embedded \s-1RDF/XML\s0 interacts with named graphs, but this is only really intended for internal use, parsing OpenDocument.)
The parser has support for named graphs within a single RDFa document. To switch this on, use the 'graph' Config option.
See also <http://buzzword.org.uk/2009/rdfa4/spec>.
The name of the attribute which indicates graph URIs is by default 'graph', but can be changed using the 'graph_attr' Config option. This option accepts Clark Notation to specify a namespaced attribute. By default, the attribute value is interpreted as like the 'about' attribute (i.e. CURIEs, URIs, etc), but if you set the 'graph_type' Config option to 'id', it will be treated as setting a fragment identifier (like the 'id' attribute).
The 'graph_default' Config option allows you to set the default graph URI/bnode identifier.
Once you're using named graphs, the \*(C`graphs\*(C' method becomes useful: it returns a hashref of { graph_uri => trine_model } pairs. The optional parameter to the \*(C`graph\*(C' method also becomes useful.
OpenDocument (\s-1ZIP\s0) host language support makes internal use of named graphs, so if you're parsing OpenDocument, tinker with the graph Config options at your own risk!
RDF::RDFa::Parser has a lot of different Config options to play with. Sometimes it might be useful to allow the page being parsed to control some of these options. If you switch on the 'auto_config' Config option, pages can do this.
A page can set options using a specially crafted <meta> tag:
<meta name="http://search.cpan.org/dist/RDF-RDFa-Parser/#auto_config" content="xhtml_lang=1&xml_lang=0" />
Note that the \*(C`content\*(C' attribute is an application/x-www-form-urlencoded string (which must then be HTML-escaped of course). Semicolons may be used instead of ampersands, as these tend to look nicer:
<meta name="http://search.cpan.org/dist/RDF-RDFa-Parser/#auto_config" content="xhtml_lang=1;xml_lang=0" />
It's possible to use auto config outside \s-1XHTML\s0 (e.g. in Atom or \s-1SVG\s0) using namespaces:
<xhtml:meta xmlns:xhtml="http://www.w3.org/1999/xhtml" name="http://search.cpan.org/dist/RDF-RDFa-Parser/#auto_config" content="xhtml_lang=0;xml_base=2;atom_elements=1" />
Any Config option may be given using auto config, except 'use_rtnlx', 'dom_parser', and of course 'auto_config' itself.
Support for Profiles (an experimental RDFa 1.1 feature) was added in version 1.09_00, but dropped after version 1.096, because the feature was removed from draft specs.
RDF::RDFa::Parser 0.21 passed all approved tests in the XHTML+RDFa test suite at the time of its release.
RDF::RDFa::Parser 0.22 (used in conjunction with HTML::HTML5::Parser 0.01 and HTML::HTML5::Sanity 0.01) additionally passes all approved tests in the HTML4+RDFa and HTML5+RDFa test suites at the time of its release; except test cases 0113 and 0121, which the author of this module believes mandate incorrect \s-1HTML\s0 parsing.
RDF::RDFa::Parser 1.096_01 passes all approved tests on the default graph (not the processor graph) in the RDFa 1.1 test suite for language versions 1.0 and host languages xhtml1, html4 and html5, with the following exceptions which are skipped:
0140 - wilful violation, pending proof that the test is backed up by the spec.
0198 - an \s-1XML\s0 canonicalisation test that may be dropped in the future.
0212 - wilful violation, as passing this test would require regressing on the old RDFa 1.0 test suite.
0251 to 0256 pass with RDFa 1.1 and are skipped in RDFa 1.0 because they use RDFa-1.1-specific syntax.
0256 is additionally skipped in \s-1HTML4\s0 mode, as the author believes xml:lang should be ignored in \s-1HTML\s0 versions prior to \s-1HTML5\s0.
0303 - wilful violation, as this feature is simply awful.
Please report any bugs to <http://rt.cpan.org/>.
Common gotchas:
Are you using the \s-1XML\s0 catalogue? RDF::RDFa::Parser maintains a locally cached version of the XHTML+RDFa \s-1DTD\s0. This will normally be within your Perl module directory, in a subdirectory named \*(L"auto/share/dist/RDF-RDFa-Parser/catalogue/\*(R". If this is missing, the parser should still work, but will be very slow.
RDF::TrineX::Parser::RDFa provides a saner interface for this module.
RDF::RDFa::Parser::Config.
XML::LibXML, RDF::Trine, HTML::HTML5::Parser, HTML::HTML5::Sanity, RDF::RDFa::Generator, RDF::RDFa::Linter.
<http://www.perlrdf.org/>, <http://rdfa.info>.
Toby Inkster <[email protected]>.
Kjetil Kjernsmo <[email protected]> wrote much of the stuff for building RDF::Trine models. Neubert Joachim taught me to use \s-1XML\s0 catalogues, which massively speeds up parsing of \s-1XHTML\s0 files that have DTDs.
Copyright 2008-2012 Toby Inkster
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.
\s-1THIS\s0 \s-1PACKAGE\s0 \s-1IS\s0 \s-1PROVIDED\s0 \*(L"\s-1AS\s0 \s-1IS\s0\*(R" \s-1AND\s0 \s-1WITHOUT\s0 \s-1ANY\s0 \s-1EXPRESS\s0 \s-1OR\s0 \s-1IMPLIED\s0 \s-1WARRANTIES\s0, \s-1INCLUDING\s0, \s-1WITHOUT\s0 \s-1LIMITATION\s0, \s-1THE\s0 \s-1IMPLIED\s0 \s-1WARRANTIES\s0 \s-1OF\s0 \s-1MERCHANTIBILITY\s0 \s-1AND\s0 \s-1FITNESS\s0 \s-1FOR\s0 A \s-1PARTICULAR\s0 \s-1PURPOSE\s0.