Google supports many approaches that permits you to choose away from crawling and indexing of the URLs. If you'll want to block some documents, directories, as well as your total website from Google Search, have a look at our guidebook about strategies to forestall content from showing in search