Scraper Development workflowΒΆ
- Create a scraper git repository
- Create the scraper on Fetch
- Write a seeder and parser scripts
- Run a seeder script locally against AnswersEngine to see if it works
- Run the parser scripts locally against the global pages
- Deploy the scraper
- Run the scraper on AnswersEngine Fetch
- Check the job outputs on AnswersEngine Fetch