Granule Acceptance

From DocDataFlow
Revision as of 22:38, 29 December 2013 by Kris (Talk | contribs)

Jump to: navigation, search

An important mechanism in Crawler is the idea of 'granule acceptance' by adapters.

When a granule is presented to any adapter for processing, the adapter can accept or reject the granule based on a number of criteria.

Some of these criteria are part of the default infrastructure of Crawler, and are checked automatically. However, these automatic criteria can always be overruled by a particular type of adapter or adapter network.

The default criteria are only there for convenience: they will be 'the right thing' in most cases. They can be adjusted for the more uncommon cases where the acceptance criteria need to be different.

The first default criterium: granules are not normally accepted twice by the same adapter.