I've started testing an Options Profile with Smart Scan enabled on some web apps based on a previous issue I had. I've noticed that the number of links crawled has dropped considerably. I just want to make sure that my assumption as to what's happened.
With modern web programming, you don't have to create individual web pages explicitly and can rely on templates and a database backend to pull in information from the database. Smart Scan can identify all of the assets that make up the page, including the template, images, and database data. So all that is appearing in the sitemap are the assets of the web app such as forms, and links that might have places where malicious values could be passed through.
Without Smart Scan this list continues in a folder structure for every month e.g. /posts/2015/02; /posts/2015/03; etc. :
With Smart Scan: