From DocDataFlow
Revision as of 04:58, 27 December 2013 by Kris (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Crawler is an attempt to find a good balance between complexity and flexibility.

Crawler is meant to be 'pokeable': it's OK to poke around in the configuration files: most often all that is needed to get the desired results is some educated 'poking around'.

The aim of the system's design is to allow a non-experienced user to confidently reconfigure a Crawler personality to a great extent without needing to digest massive amounts of documentation beforehand.

Crawler's configuration files, templates and formula files are all simple text files.

Users are encouraged to open them, and should to make small modifications to these files, and get the expected results. It is OK to make changes to a configuration file without fully understanding it; most often, Crawler will behave as expected and produce the expected results.

Users that desire total control or desire to build a custom personality from scratch cannot avoid having to study the Crawler documentation, and spend substantial time trying and experimenting. But unlike many other complex systems, many basic reconfigurations can be performed without much need for a manual.