I use to automate many tasks in the web (such as triggering news and sorting them according to my interests) and some times I hit with Cloudflare’s DDoS protection, I mean client-side, I guess they will have some real DDoS protection server-side, as the client-side is very easy to break.
They use to change their algorithms from time to time, but the basics are always the same:
- curl the protected page and you will get an invisible form which auto-sends itself after making some client-side calculations. Currently this form is called challenge-form and has three hidden fields called jschl_vc, pass and jschl_answer.
- Compute the javascript you find in the curl-ed page and send the form (the results of the calculations will be populated to the jschl_answer field). You can emulate these calculations in your favorite language (Python, AppleScript, PHP, whatever) or have a JS engine to execute the scripts. I’ve used node for this, which is available as a command-line tool you can install and execute easilly.
- If everything is OK, you will have now two cookies: __cfduid and cf_clearance. Using that cookies, you can now surf freely the website.
The details may change from time to time, as Cloudflare updates their methods, but it’s been very similar across the years. Just take a look in your regular browser/developer tools and find the magic under the hoods.
Julio Sancho's Blog 