Getting ready...

There are several means of creating a scraper with Scrapy.  One is a programmatic pattern where we create the crawler and spider in our code.  It is also possible to configure a Scrapy project from templates or generators and then run the scraper from the command line using the scrapy command.  This book will follow the programmatic pattern as it contains the code in a single file more effectively.  This will help when we are putting together specific, targeted, recipes with Scrapy. 

This isn't necessarily a better way of running a Scrapy scraper than using the command line execution, just one that is a design decision for this book.  Ultimately this book is not about Scrapy (there are other books on just Scrapy), but more of an exposition on various things you may need to do when scraping, and in the ultimate creation of a functional scraper as a service in the cloud.