WWW::RobotRules::AnyDBM_File (3pm)

 require WWW::RobotRules::AnyDBM_File;
 require LWP::RobotUA;

 # Create a robot useragent that uses a diskcaching RobotRules
 my $rules = WWW::RobotRules::AnyDBM_File->new( 'my-robot/1.0', 'cachefile' );
 my $ua = WWW::RobotUA->new( 'my-robot/1.0', '[email protected]', $rules );

 # Then just use $ua as usual
 $res = $ua->request($req);

This is a subclass of WWW::RobotRules that uses the AnyDBM_File package to implement persistent diskcaching of robots.txt and host visit information.

The constructor (the new() method) takes an extra argument specifying the name of the \s-1DBM\s0 file to use. If the \s-1DBM\s0 file already exists, then you can specify undef as agent name as the name can be obtained from the \s-1DBM\s0 database.

WWW::RobotRules, LWP::RobotUA

Hakan Ardo <[email protected]>, Gisle Aas <[email protected]>