During my first (shitty) job as a dev outta school, they had me writing scrapers. I was actually able to subvert it pretty easily using this package that doesn’t appear to be maintained anymore https://github.com/VeNoMouS/cloudscraper
Was pretty surprised to learn that, at the time, they were only checking if JS was enabled, especially since CF is the gold standard for this sort of stuff. I’m sure this has changed?
Given that the last updates to this repo were five years ago, I’m not too sure if it’s still valid. I don’t follow Cloudflare bypasses but I am fairly certain there are more successful frameworks and services now. The landscape is evolving quickly. We are seeing a proliferation of “bot as a service”, captcha passing farms, dedicated browsers for botting, newsletters, substacks, Discord servers, you name it. Then there are the methods you don’t readily find much talk on like custom modified Chrome browsers. It’s fascinating how much effort is being funneled into this field.
Oh i can definitely see custom browsers being useful in that area. I remember the JavaScript navigator properties were always such a PITA as there was nothing you could really do to get around what they exposed
There’s a whole world of tools you can use that do that for you now. It’s easier than ever. To me it’s concerning. The level of automation, coupled with a halfway decent LLM, can give you the ability to summon hordes of fake humans to social media. I can’t help but think it’s why X and Reddit don’t use any kind of anti-bot solution.
Thanks for reading and commenting!
During my first (shitty) job as a dev outta school, they had me writing scrapers. I was actually able to subvert it pretty easily using this package that doesn’t appear to be maintained anymore https://github.com/VeNoMouS/cloudscraper
Was pretty surprised to learn that, at the time, they were only checking if JS was enabled, especially since CF is the gold standard for this sort of stuff. I’m sure this has changed?
Given that the last updates to this repo were five years ago, I’m not too sure if it’s still valid. I don’t follow Cloudflare bypasses but I am fairly certain there are more successful frameworks and services now. The landscape is evolving quickly. We are seeing a proliferation of “bot as a service”, captcha passing farms, dedicated browsers for botting, newsletters, substacks, Discord servers, you name it. Then there are the methods you don’t readily find much talk on like custom modified Chrome browsers. It’s fascinating how much effort is being funneled into this field.
Oh i can definitely see custom browsers being useful in that area. I remember the JavaScript navigator properties were always such a PITA as there was nothing you could really do to get around what they exposed
There’s a whole world of tools you can use that do that for you now. It’s easier than ever. To me it’s concerning. The level of automation, coupled with a halfway decent LLM, can give you the ability to summon hordes of fake humans to social media. I can’t help but think it’s why X and Reddit don’t use any kind of anti-bot solution.