CrawlSpider, SitemapSpider, and Other Specialized Spiders

A CrawlSpider Rule has `callback='parse_product'` and `follow=False`. What happens?

• ALinks matching the rule are followed and parsed, then the spider recursively follows links in those responses
• BLinks matching the rule are followed, the response is passed to parse_product, but links inside the response are NOT followed
• CThe rule is disabled because follow is False
• DThe spider parses but does not download matching pages