Baidu Spider
Posted: December 2nd, 2017, 5:11 pm
by GingkathFox
Just wonderin', is it ok fer the BaiduSpider bot to be in the Forums?
Re: Baidu Spider
Posted: December 3rd, 2017, 1:08 pm
by musikbear
GingkathFox wrote: ↑December 2nd, 2017, 5:11 pm
is it ok fer the BaiduSpider bot to be in the Forums?
doubt 'we' could prevent it, if we would like to. Baidu 'is' Chinese google, and the bot is indexing for Chinese users..
Re: Baidu Spider
Posted: December 3rd, 2017, 2:52 pm
by GingkathFox
musikbear wrote: ↑December 3rd, 2017, 1:08 pm
GingkathFox wrote: ↑December 2nd, 2017, 5:11 pm
is it ok fer the BaiduSpider bot to be in the Forums?
doubt 'we' could prevent it, if we would like to. Baidu 'is' Chinese google, and the bot is indexing for Chinese users..
it seems to disregard the 'Robots.txt' though on other sites
https://www.sitepoint.com/community/t/a ... bot/4767/2