MarcellusDrum@lemmy.ml to Technology@lemmy.worldEnglish · 4 days agoThis website is for humanslocalghost.devexternal-linkmessage-square61fedilinkarrow-up1479arrow-down18cross-posted to: Technology@programming.dev
arrow-up1471arrow-down1external-linkThis website is for humanslocalghost.devMarcellusDrum@lemmy.ml to Technology@lemmy.worldEnglish · 4 days agomessage-square61fedilinkcross-posted to: Technology@programming.dev
minus-square𝚝𝚛𝚔@aussie.zonelinkfedilinkEnglisharrow-up6·2 days agoIs there a more ignored file on the internet than robots.txt?
Is there a more ignored file on the internet than robots.txt?
Do dnt headers count?