|
||||||||||
PREV NEXT | FRAMES NO FRAMES |
AdaptiveRevisitHostQueue
s used by a
Frontier.Double
at the specified index to this list.
double
at the specified index to this list.
Double
at the end of this list.
Float
at the specified index to this list.
float
at the specified index to this list.
Float
at the end of this list.
Integer
at the specified index to this list.
int
at the specified index to this list.
Integer
at the end of this list.
Long
at the specified index to this list.
long
at the specified index to this list.
Long
at the end of this list.
String
at the specified index to this list.
String
at the end of this list.
curi
with response status and
content type.
List
with 'data' Element
values.ANVLRecord
s.int
to the buffer.
long
to the buffer.
ArchiveRecord
s.curi
.
curi
.
fetch-bandwidth
attribute.
max-length-bytes
attribute.
password
attribute.
timeout-seconds
attribute.
username
attribute.
RecyclingFastBufferedOutputStream.pos
.
WriterPoolMember
.
content digest
with the one from a previous crawl.WriterPoolProcessor.initialTasks()
when recovering a checkpoint.
ClientFTP
.
WriterPoolMember
s in pool.
DecideRule
.CrawlController
when checkpointing.
UURI
.
CandidateURI
type
giving the credential the passed name
.
offset
into
is
.
ListIterator
over the criteria set for this
refinement.
CrawlUriSWFAction
action.- CustomSWFTags(SWFActions) -
Constructor for class org.archive.crawler.extractor.CustomSWFTags
-
DecideRule.ACCEPT
,
DecideRule.REJECT
, or
DecideRule.PASS
.DecideRule
s have been set up inside
it.object
.
object
.
matcher.match(object)
returns true will be deleted from the queue.
DecidingScope
.BdbFrontier
and
QuotaEnforcer
.os
.
EndedException
.
ExternalImplDecideRule
.ExternalImplDecideRule
.ExtractorSWFActions
, which
parse URI-like strings.FetchFTP
.
MatchesFilePatternDecideRule
.ValueErrorHandler
.
CrawlURI
from the passed CandidateURI
.
record-id
generator.offset
.
offset
.
offset
.
offset
.
offset
.
System.currentTimeMillis()
at that time).
System.currentTimeMillis()
at that time).
System.currentTimeMillis()
when the crawl started).
extract.from.dirs
attribute for this
FetchFTP
and the given curi.
extract.parent
attribute for this
FetchFTP
and the given curi.
fetch-bandwidth
attribute for this
FetchFTP
and the given curi.
-Dheritrix.home
if available to us.
host
.
CrawlHost
associated with name
.
CrawlHost
associated with curi
.
URIFrontierMarker
initialized with the given
regular expression at the 'start' of the Frontier.
host
IF its in
IPV4 quads format (e.g.
max-length-bytes
attribute for this
FetchFTP
and the given curi.
File
object pointing to the order file.
ComplexType
owning the checked attribute.
parameters
associated
with this connection manager.
curi
.
classType
or a
subclass of it.
AtomicLong
s.
AtomicLong
.
AtomicLong
.
curi
.
CrawlServer
associated with name
,
creating if necessary.
CrawlServer
associated with curi
.
CrawlerSettings
for the checked attribute.
CrawlerSettings
object this refinement refers to.
key
in settings.
key
in settings.
getState()
except this method returns a
human readable name for the state instead of its constant integer value.
timeout-seconds
attribute for this
FetchFTP
and the given curi.
numberOfMatches
is reached.
CandidateURI.toString()
.
ConfigurableX509TrustManager
.DecidingFilter
and
equivalent DecideRule
.DecidingScope
.HttpRecorderGetMethod
and HttpRecorderPostMethod
.URIFrontierMarker
that has become invalid.HttpConnectionParams.isStaleCheckingEnabled()
,
HttpConnectionManager.getParams()
.
CandidateURI
with the Frontier.
Level.WARNING
).
Level.WARNING
) and default error message.
Level.WARNING
).
Level.WARNING
) and default error message.
Long
.
CrawlURI
as
requiring a prerequisite.
Queue
.RecoverableIOException
.
DecidingFilter
and
DecideRule
.int
to a String
, and pad it to
pad
spaces.
String
to pad
characters wide
by pre-pending spaces.
String
to pad
characters wide
by pre-pending padChar
.
String
with the character
encoding of the local system or the document.
DecidingFilter
and
equivalent DecideRule
.DecidingFilter
and
equivalent DecideRule
.DecidingScope
.in
.
recordId
.
int
right-aligned to the given column.
long
, right-aligned to the given column.
WriterPoolMember.copyFrom(InputStream,long,boolean)
instead
WriterPoolMember.copyFrom(InputStream,long,boolean)
instead
RecyclingFastBufferedOutputStream.DEFAULT_BUFFER_SIZE
bytes.
ListIterator
over the refinements for this
settings object.
ValueErrorHandler
.
Level.WARNING
).
name
.
name
.
ReplayCharSequence.close()
method.InputStream
to make a primitive Repositionable
stream.CandidateURI
with the Frontier.
the primary DB
, URIs indexed
by the time when they can next be processed again.
HttpConnectionParams.setStaleCheckingEnabled(boolean)
,
HttpConnectionManager.getParams()
.
CrawlURI.setContentDigest(String scheme, byte[])
parameters
for this
connection manager.
typeName
.
type
.
type
.
DecidingFilter
and
equivalent DecideRule
.DecidingScope
.HttpConnectionManager
.CandidateURI.getPathFromSeed()
) ends
with at least one, but not more than, the given number of
non-navlink ('L') hops.DecidingFilter
and
equivalent DecideRule
.CrawlURI
annotations.
ValueErrorHandler
.
DecidingFilter
and
equivalent DecideRule
.DecidingFilter
and
equivalent DecideRule
.java.util.UUID
, formatted as URNs from the UUID
namespace [See RFC4122].SettingsHandler
, only
constraints with level Level.SEVERE
will throw an
InvalidAttributeValueException
.true
if the WorkQueue implementation of this
Frontier stores its workload on disk instead of relying
on serialization mechanisms.
WARCWriter.writeRecord(String,String,String,String,URI,ANVLRecord,InputStream,long,boolean)
instead
WriterPool
.WriterPool
.warcinfo
to current file.
|
||||||||||
PREV NEXT | FRAMES NO FRAMES |