User-agent: * Disallow: /files/ Disallow: /admin/ Disallow: *.xls Disallow: *.xlsx Disallow: *.csv Disallow: *.pdf Disallow: *.doc Disallow: *.docx