The American ban on TikTok continues to loom as we speak. The matter does not seem to be getting any easier, especially after its parent company ByteDance just rolled out a new and powerful web scraping tool.
Dubbed Bytespider, the tool is designed to aggressively scrape data online. It’s getting rave reviews for overtaking other leading tech giants, thanks to its mighty speeds for data collection.
The news comes after a leading research study was published by a bot management firm about the activity of the new web scraper. As per stats published, it has a 25 times faster speed than other leading tech giants. This includes ChatGPT from OpenAI and ClaudeBot from Anthropic.
Seeing this kind of aggressive data collection method used by a company that is already causing serious controversy in the US isn’t great news. It just makes it so much harder to not back TikTok’s ban.
Another interesting point is how the tool is disregarding the current robots.txt rule in place. This advises scrapers against attacking certain websites. So why the sudden rise in web scraping capabilities?
As per a recent update, ByteDance wants to boost its TikTok’s search features. By this we mean enabling real-time keyword targeting in search results. While ByteDance is not replying to questions being thrown in its direction, it’s obvious what its intentions in this case are.
Why does all of this really matter? The answer is simple. Excessive web scraping is a trend that many tech giants embark on to get data for free. This is then used to train AI models, even though many websites forbid the actions with robots.txt protocols in place. It’s against their data privacy policy.
In the past, many tech giants like NVIDIA were slammed for scraping YouTube’s videos for AI training. This issue raised serious concerns about the rights belonging to content creators and the ethical implications it might have for using non-consensual data. Yes, data is available online but you cannot just take it without consent, advocates argue.
Another widespread debate arose last month when LinkedIn was slammed for taking users’ data and using that to train its AI models. No updates were made in the company’s terms of service, impacting users across America.
Image: DIW-Aigen
Read next: Meta Pushes For More Innovation On Facebook To Encourage Young Adults To Use The App