[{"data":1,"prerenderedAt":11},["ShallowReactive",2],{"$f7widgrt35dxz":3},{"slug":4,"title":5,"description":6,"published":7,"updated":8,"thumb":9,"html":10},"robots-txt-fileto-exclude-all-robots-from-the-entire-site","robots.txt FileTo exclude all Robots From The Entire Site","If you don’t want to let your sites get indexed on search engines, you can disable access via robots.txt file. Create a file named robots.txt and upload...","2015-02-12","2016-06-17",null,"\u003Cp>If you don’t want to let your sites get indexed on search engines, you can disable access via robots.txt file. \u003C\u002Fp>\n\u003Cp>Create a file named robots.txt and upload it to your root directory of domain.\u003Cbr \u002F>\n\u003Ccode>\u003Cbr \u002F>\nUser-agent: *\u003Cbr \u002F>\nDisallow: \u002F\u003Cbr \u002F>\n\u003C\u002Fcode>\u003C\u002Fp>\n\u003Cp>To exclude all robots from certain directories robots.txt:\u003Cbr \u002F>\n\u003Ccode>User-agent: *\u003Cbr \u002F>\nDisallow: \u002Fcgi-bin\u002F\u003Cbr \u002F>\nDisallow: \u002Ftmp\u002F\u003Cbr \u002F>\nDisallow: \u002Fjunk\u002F\u003Cbr \u002F>\n\u003C\u002Fcode>\u003C\u002Fp>",1793901039342]