Search engines won't know how to proceed Using the URL . It truly is unclear to them whether or not they're allowed to obtain. When directives aren't distinct to Google, they'll go With all the least restrictive directive, which in this case implies that they'd in fact obtain . By https://rankingseotoolz.com/robots-txt-generator