table of contents
| Scrappy::Queue(3pm) | User Contributed Perl Documentation | Scrappy::Queue(3pm) |
NAME¶
Scrappy::Queue - Scrappy HTTP Request Flow-Control System
VERSION¶
version 0.94112090
SYNOPSIS¶
#!/usr/bin/perl
use Scrappy::Queue;
my $queue = Scrappy::Queue->new;
$queue->add($url);
while (my $url = $queue->next) {
... $queue->add(...);
}
DESCRIPTION¶
Scrappy::Queue provides a system for saving URLs to a recordset/queue and iterating of them using the Scrappy framework.
METHODS¶
list¶
The list method return the list of URLs in the queue. This is returned in list context.
my $queue = Scrappy::Queue->new;
...
my @list = $queue->list;
add¶
The add method adds new URLs to the queue. Duplicate URLs will be ignored.
my $queue = Scrappy::Queue->new;
$queue->add($url);
clear¶
The clear method completely empties the queue and resets the cursor (loop position).
my $queue = Scrappy::Queue->new;
$queue->add(...);
$queue->add(...);
$queue->add(...);
$queue->clear;
reset¶
The reset method resets the cursor (loop position).
my $queue = Scrappy::Queue->new;
$queue->add(...);
$queue->add(...);
$queue->add(...);
while (my $url = $queue->next) {
$queue->reset if ...; # beware the infinate loop
}
$queue->reset;
current¶
The current method returns the URL in the current loop position.
my $queue = Scrappy::Queue->new;
$queue->add(...);
$queue->add(...);
$queue->add(...);
while (my $url = $queue->next) {
last if ...;
}
print 'great' if $url eq $queue->current;
next¶
The next method moves the cursor to the next loop position and returns the URL.
my $queue = Scrappy::Queue->new;
$queue->add(...);
$queue->add(...);
$queue->add(...);
while (my $url = $queue->next) {
...
}
previous¶
The previous method moves the cursor to the previous loop position and returns the URL.
my $queue = Scrappy::Queue->new;
$queue->add(...);
$queue->add(...);
$queue->add(...);
while (my $url = $queue->next) {
...
}
print $queue->previous;
first¶
The first method moves the cursor to the first loop position and returns the URL.
my $queue = Scrappy::Queue->new;
$queue->add(...);
$queue->add(...);
$queue->add(...);
print $queue->first;
last¶
The last method moves the cursor to the last loop position and returns the URL.
my $queue = Scrappy::Queue->new;
$queue->add(...);
$queue->add(...);
$queue->add(...);
print $queue->last;
index¶
The index method moves the cursor to the specified loop position and returns the URL. The loop position is a standard array index position.
my $queue = Scrappy::Queue->new;
$queue->add(...);
$queue->add(...);
$queue->add(...);
print $queue->index(1);
cursor¶
The cursor method returns the current loop position.
my $queue = Scrappy::Queue->new;
print $queue->cursor;
AUTHOR¶
Al Newkirk <awncorp@cpan.org>
COPYRIGHT AND LICENSE¶
This software is copyright (c) 2010 by awncorp.
This is free software; you can redistribute it and/or modify it under the same terms as the Perl 5 programming language system itself.
| 2022-06-17 | perl v5.34.0 |