Always Refresh Listing Page

Syntax: choose from drop down list

It would be beneficial to refresh pages that provide links to new content more often than the content itself. This setting attempts to accomplish this, by looking for pages that have links but no content; specifically robots NOINDEX and FOLLOW rules applied. When detected, the behavior will depend on this setting:

• No - These pages receive no special schedule, and are refreshed as normal.

• Limited (default) - Listing pages are always scheduled for the Minimum Refresh Time configured for this profile.

• Yes - Listing pages are always scheduled as soon as possible, so they'll be refreshed every walk.