Details
-
Improvement
-
Status: Closed
-
Low
-
Resolution: Fixed
-
None
-
None
-
Quasar
-
Quasar Sprint 300
Description
Hi All,
Can we create an allow field in the robots.txt plugin?
According to google (https://developers.google.com/search/docs/advanced/robots/create-robots-txt) this is an allowed option, but not yet implemented in the current robots.txt plugin.
[notes from quicksort] Implementing a blog list, by having the 'allow' list included, you can disallow now. The robots.txt domain/robots.txt will be used by robots for SEO which parts of the site should be indexed or not. Especially for updating routes and mapping allow rules, for certain prefixes, for instance everything under /cars should be indexed but not under /admin.
Attachments
Issue Links
- causes
-
CMS-15430 New robots allow property missing from existing documents in client projects throws NPE
- Closed