org.archive.crawler.deciderules.recrawl
Class IdenticalDigestDecideRule

java.lang.Object
  javax.management.Attribute
      org.archive.crawler.settings.Type
          org.archive.crawler.settings.ComplexType
              org.archive.crawler.settings.ModuleType
                  org.archive.crawler.deciderules.DecideRule
                      org.archive.crawler.deciderules.ConfiguredDecideRule
                          org.archive.crawler.deciderules.PredicatedDecideRule
                              org.archive.crawler.deciderules.recrawl.IdenticalDigestDecideRule

All Implemented Interfaces:

java.io.Serializable, javax.management.DynamicMBean, CoreAttributeConstants

public class IdenticalDigestDecideRule
extends PredicatedDecideRule
implements CoreAttributeConstants

Rule applies configured decision to any CrawlURIs whose prior-history content-digest matches the latest fetch.

Author:

gojomo

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.archive.crawler.settings.ComplexType

ComplexType.MBeanAttributeInfoIterator

Field Summary

Fields inherited from class org.archive.crawler.deciderules.ConfiguredDecideRule

ALLOWED_TYPES, ATTR_DECISION

Fields inherited from class org.archive.crawler.deciderules.DecideRule

ACCEPT, PASS, REJECT

Fields inherited from class org.archive.crawler.settings.ComplexType

definition, definitionMap

Fields inherited from interface org.archive.crawler.datamodel.CoreAttributeConstants

A_ANNOTATIONS, A_CONTENT_DIGEST, A_CONTENT_TYPE, A_CREDENTIAL_AVATARS_KEY, A_DELAY_FACTOR, A_DISTANCE_FROM_SEED, A_DNS_FETCH_TIME, A_DNS_SERVER_IP_LABEL, A_ETAG_HEADER, A_FETCH_BEGAN_TIME, A_FETCH_COMPLETED_TIME, A_FETCH_HISTORY, A_FORCE_RETIRE, A_FTP_CONTROL_CONVERSATION, A_FTP_FETCH_STATUS, A_HERITABLE_KEYS, A_HTML_BASE, A_HTTP_BIND_ADDRESS, A_HTTP_PROXY_HOST, A_HTTP_PROXY_PORT, A_HTTP_TRANSACTION, A_LAST_MODIFIED_HEADER, A_LOCALIZED_ERRORS, A_META_ROBOTS, A_MINIMUM_DELAY, A_MIRROR_PATH, A_PREREQUISITE_URI, A_REFERENCE_LENGTH, A_RETRY_DELAY, A_RRECORD_SET_LABEL, A_RUNTIME_EXCEPTION, A_SOURCE_TAG, A_STATUS, A_WRITTEN_TO_WARC, HEADER_TRUNC, LENGTH_TRUNC, TIMER_TRUNC, TRUNC_SUFFIX

Constructor Summary

IdenticalDigestDecideRule(java.lang.String name)
Usual constructor.

Method Summary

protected boolean	evaluate(java.lang.Object object) Evaluate whether given CrawlURI's content-digest exactly matches that of preceding fetch.
static boolean	hasIdenticalDigest(CrawlURI curi) Utility method for testing if a CrawlURI's last two history entiries (one being the most recent fetch) have identical content-digest information.

Methods inherited from class org.archive.crawler.deciderules.PredicatedDecideRule

decisionFor

Methods inherited from class org.archive.crawler.deciderules.ConfiguredDecideRule

singlePossibleNonPassDecision

Methods inherited from class org.archive.crawler.deciderules.DecideRule

getController, kickUpdate

Methods inherited from class org.archive.crawler.settings.ModuleType

addElement, listUsedFiles

Methods inherited from class org.archive.crawler.settings.ComplexType

addElementToDefinition, checkValue, earlyInitialize, getAbsoluteName, getAttribute, getAttribute, getAttribute, getAttributeInfo, getAttributeInfo, getAttributeInfoIterator, getAttributes, getDataContainerRecursive, getDataContainerRecursive, getDefaultValue, getDescription, getElementFromDefinition, getLegalValues, getLocalAttribute, getMBeanInfo, getMBeanInfo, getParent, getPreservedFields, getSettingsHandler, getUncheckedAttribute, getValue, globalSettings, invoke, isInitialized, isOverridden, iterator, removeElementFromDefinition, setAsOrder, setAttribute, setAttribute, setAttributes, setDescription, setPreservedFields, toString, unsetAttribute

Methods inherited from class org.archive.crawler.settings.Type

addConstraint, equals, getConstraints, getLegalValueType, isExpertSetting, isOverrideable, isTransient, setExpertSetting, setLegalValueType, setOverrideable, setTransient

Methods inherited from class javax.management.Attribute

getName, hashCode

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

IdenticalDigestDecideRule

public IdenticalDigestDecideRule(java.lang.String name)

Usual constructor.

Parameters:

name -

Method Detail

evaluate

protected boolean evaluate(java.lang.Object object)

Evaluate whether given CrawlURI's content-digest exactly matches that of preceding fetch.

Specified by:

evaluate in class PredicatedDecideRule

Parameters:

object - should be CrawlURI

Returns:

true if current-fetch content-digest matches previous

hasIdenticalDigest

public static boolean hasIdenticalDigest(CrawlURI curi)

Utility method for testing if a CrawlURI's last two history entiries (one being the most recent fetch) have identical content-digest information.

Parameters:

curi - CrawlURI to test

Returns:

true if last two history entries have identical digests, otherwise false