Tea to Technology@lemmy.worldEnglish • edit-217 days agoAI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.diff.wikimedia.orgexternal-linkmessage-square81arrow-up1717arrow-down17
arrow-up1710arrow-down1external-linkAI crawlers cause Wikimedia(The umbrella organization of Wikipedia and a dozen or so other crowdsourced knowledge projects) Commons bandwidth demands to surge 50%.diff.wikimedia.orgTea to Technology@lemmy.worldEnglish • edit-217 days agomessage-square81
minus-square@grysbok@lemmy.sdf.orglinkfedilinkEnglish2•16 days agoBots lie about who they are, ignore robots.txt, and come from a gazillion different IPs.
Bots lie about who they are, ignore robots.txt, and come from a gazillion different IPs.
That’s what ddos protection is for.