regex - robot.txt syntax not understood -


i getting error on first line of robot.txt user-agent: *

my robot.txt follows:

user-agent: * disallow: /search/ disallow: /_layouts/  disallow: /blog/_layouts/ disallow: /blog/_layouts/ disallow: /reusablecontent/ disallow: /reports%20list/ disallow: /workflowtasks/ disallow: /sitecollectionimages/ disallow: /documents/forms/ disallow: /pages/forms/ disallow: /internet/

i have figured there space /reports%20list/. creating issues? not sure. appreciated.

the trouble here 3 invisible bytes @ beginning of utf-8 formatted robots.txt file, called bom (byte order mask). bom optional. yes, need file utf-8 formatted, google not optional bom in expected clear , simple robots.txt file , parsing fails.

you can read more detailed explanation here.


Comments

Post a Comment

Popular posts from this blog

blackberry 10 - how to add multiple markers on the google map just by url? -

i have android application, want convert blackberry 10 , google map needed in app,but blackberry 10 not support googlemap , blackberry developer web site supply way replace it.the sample below: @override public void oncreate(bundle savedinstancestate) { super.oncreate(savedinstancestate); setcontentview(r.layout.main); webview webview = (webview) findviewbyid(r.id.mywebview); webview.getsettings().setjavascriptenabled(true); webview.loadurl("http://maps.google.com/?ll=36.97, -122&lci=bike&z=13&t=p"); } i cannot understand it,i not know how add multiple markers on googlemap via url? can me? this worked me map shown in webview.
Read more

php - guestbook returning database data to flash -

i've been struggling problem long , unable make work, hope can me it. first, know php files correct , mysql database works fine. in flash, able submit comment database comments not display. did many tests such making returned_database flash separately , worked fine - can see old comments. long combine submitting 1 , returning 1 together, flash doesn't show comments more. put lower version flash effects on frame thinking doesn't collaborate returning flash, submitting flash works fine. since pretty complex problem hard me copy actionscript code because on different frames in flash. if has ideas fixing maybe can send file you?
Read more

delphi - Dynamic file type icon -

i want register file type in windows , make program delphi draw icon of file according contents (something custom icon format). how can this? thanks nortd , user539484 comments, searched internet "shell icon handler delphi" , found sample here , , shell+ components has component this.
Read more