(Preliminary content)

Shaman Visual Editor

Shaman automatically detects lists, tables and other similar patterns of repeated content. Then, it uses this information to recognize where potentially interesting data is located, and pre-populates a list of fields, each one with its own CSS selector. Shaman can analyze linked pages, detect pagination, search queries, dynamic content, HTML-embedded-JSON and much more. You can manually rename or delete fields based on your needs.

Shaman Code Editor

Eventually, you will encounter websites that require human intervention. From the code editor, you can write custom logic to extract the fields that Shaman couldn't manage to extract automatically. Changes made from the visual editor are automatically merged and synced with the ones made from the code editor.

Shaman Data Explorer

The Data Explorer lets your view the extracted data (as table, grid or list) and export it (JSON/CSV/Excel/SQLite). You can write complex queries, plot geographic data on a map. Data is loaded lazily from the website and then cached locally, but you can manually start and stop full scraping jobs.

Connector Store

Users can create and share their connectors. When creating a new connector, Shaman will let you know if a connector for that specific website has already been created.