.Google.com confines robots.txt help to 4 industries, clarifying its own posture on in need of support ordinances.Google.com merely sustains 4 details robots.txt areas.Unsupported regulations in robots.txt will definitely be actually disregarded.Think about bookkeeping your robots.txt reports due to this improve.