Baidu Spider
Posted: Sat Dec 02, 2017 5:11 pm
by GingkathFox
Just wonderin', is it ok fer the BaiduSpider bot to be in the Forums?
Re: Baidu Spider
Posted: Sun Dec 03, 2017 1:08 pm
by musikbear
GingkathFox wrote: ↑Sat Dec 02, 2017 5:11 pm
is it ok fer the BaiduSpider bot to be in the Forums?
doubt 'we' could prevent it, if we would like to. Baidu 'is' Chinese google, and the bot is indexing for Chinese users..
Re: Baidu Spider
Posted: Sun Dec 03, 2017 2:52 pm
by GingkathFox
musikbear wrote: ↑Sun Dec 03, 2017 1:08 pm
GingkathFox wrote: ↑Sat Dec 02, 2017 5:11 pm
is it ok fer the BaiduSpider bot to be in the Forums?
doubt 'we' could prevent it, if we would like to. Baidu 'is' Chinese google, and the bot is indexing for Chinese users..
it seems to disregard the 'Robots.txt' though on other sites
https://www.sitepoint.com/community/t/a ... bot/4767/2