A system for automatically generating a wrapper for an entire website, the wrapper characterising the structure of the website, the system having a plurality of functional elements, including at least one annotation module to classify components of a page and generate an annotated, a page classification module to identify functional and informational components of an annotated page, and an action module to identify an action to be taken to further navigate the website, wherein at least one of the annotation module, page classification module and action module is operable in response to a plurality of domain-specific rules, where a domain is understood as a conceptual domain such as real estate, used cars, or electronics.