By 2025, the digital landscape has changed: CAPTCHA is no longer the reliable gatekeeper it once was. While AI-driven bots solve CAPTCHA puzzles with near-perfect accuracy, genuine users are left frustrated and often abandon sites when challenged. Recent studies show bots now breeze through image-based and text CAPTCHAs 96–100% of the time—far outpacing real human success rates and dropping form conversions by as much as 20%. But the problem goes far deeper than any outdated puzzle.
Today, automated traffic dominates the web. I realize this personally. In 2024, it was estimated that nearly half of all online activity was generated by bots, with up to 37% classified as outright malicious. Even sites with active mitigation still report 10–20% ongoing bot activity. The reality is stark: traditional solutions like CAPTCHA and IP blacklists have become nearly powerless in the face of coordinated, rapidly evolving botnets that can mimic real users, cycle through fresh IPs, and even exploit mobile devices for large-scale attacks.
For website owners and online businesses, the impact is devastating. Bot floods can cripple server resources, slow page loads to a crawl, and ruin user experience. But the effects ripple further—Google rankings drop as page performance tanks, ad revenue evaporates as traffic quality declines, and relationships with advertising partners sour when fake visits swamp their analytics.
I experienced this crisis firsthand. It all began with an accusation from an ad agency: they claimed that 90% of my site’s traffic was fake. Their tracking code, embedded for ad delivery, revealed bot volumes that were overwhelming not only their filters but my server as well. We’re talking about over a million bot visits per day—traffic invisible to Google Analytics but catastrophic behind the scenes. What I initially believed were genuine users were, in reality, part of a relentless wave of automated hits, flooding my infrastructure and threatening the viability of my entire project.
This isn’t just a story about bad actors exploiting weaknesses—it’s about how the very architecture of the modern web is under siege. Code optimizations and server upgrades weren’t enough. The challenge became an arms race, with my site caught in the crossfire. Here’s how the bot flood unfolded, nearly destroying everything I’d built—and the steps I took to fight back.
My bot traffic story: 3-million website to a half million
It all started with an advertising agency accusing me of having 90% fake traffic, I've already said that but: they had placed a tracking code on my site to deliver ads, and the bot volume was a problem for them, too—it overwhelmed their filters and inflated server load. We’re talking about over a million bot visits per day.
At first, I was outraged. In Google Analytics, I was seeing 100,000 pure daily visits. Real users, I thought. But their concern was about traffic outside Analytics. That invisible layer of hits was wreaking havoc on server load. Back then, my project was running on Laravel 5.3 hosted on shared hosting, and I believed the performance bottleneck was the old codebase. I rewrote everything in Laravel 10 with superb optimization, including daily analysis of millions of database records.
Still, it lagged. My shared hosting couldn't handle it. Page loads crawled, and real traffic dropped—month by month, I was losing about 150,000 visits. From 3 million monthly visits, I eventually lost more than half.
I had upgraded to a powerful VPS with 16 CPU cores and 32GB of RAM, expecting this to resolve everything. But even with the improved infrastructure and recoded Laravel 10 backend, the issue persisted. In fact, things got worse—the bots became more aggressive, increasing their attack volume and frequency. It became clear that no amount of code optimization or hardware scaling could fix a problem that was fundamentally about uncontrolled, malicious bot traffic.
But that wasn’t all. Digging deeper, I realized the scale was even bigger: over 2 million website crawls per day, separate from about 1.5 million daily bot visits. And yet, the monetizable, trackable part of the site (the one that agencies cared about) was only bringing in 100,000 impressions per day. That’s where the problem escalated. I was working with an ad agency that relied on clean, human traffic. They had to find ways to filter out the bots fast, or their analytics and ad serving systems would be overwhelmed. The accusations, the infrastructure overload, and the revenue discrepancies—they were all tied to this invisible, relentless flood of bots.
My first move was to create a custom CAPTCHA, aiming to show bots a blank page while real users passed through. Unfortunately, that backfired. Malicious bots didn’t slow down—they ramped up. The CAPTCHA became a challenge they aggressively tried to overcome, doubling the load.
Next came mass blocking via .htaccess. It worked—for a few days. Then the bot networks adjusted, new IPs appeared, and .htaccess became bloated and slow. My hosting provider stepped in, helping to block entire subnets temporarily, but the problem returned weekly.
Finally, I turned to Cloudflare. This was the most impactful change. While not perfect, it allowed me to filter over 1.5 million bot requests within 24 hours. I uploaded network blocks directly into their firewall. The result? From 1.5 million bot hits, only 20 CAPTCHA challenges were triggered daily—proof that Cloudflare's edge detection worked better than anything else I tried.
To stay ahead of the bots, I built my own internal logging system. It records every single request by IP address and User-Agent string, storing them in a database. Behind the scenes, a scheduled task runs every minute to aggregate the data. When it detects suspicious activity—such as a large volume of requests coming from a single network or IP range—it triggers an automated email notification. That email includes a list of IPs and subnets ready to be added to Cloudflare for blocking.
I’m still on Cloudflare’s free plan, but even that provides enough control to implement manual firewall rules. This proactive approach allows me to detect and respond to bot floods before they overwhelm the system. At the Apache level, I originally tried using .htaccess to block traffic directly, but this method had diminishing returns. As more rules piled up, site performance degraded, making it clear that server-level blocking was not sustainable without a CDN or edge-layer support.
How to make Login System + CloudFlare Protection?
Why not rate limiting or geo-blocking? Because they don't work in my case. Most of these bots make just one request per IP—but they do it using hundreds or even thousands of IPs within the same network. That means rate limiting by IP doesn’t help much; the volume is distributed, not concentrated. As for detecting them by User-Agent? Useless. Some bots are clever enough to mimic Googlebot or other legitimate crawlers, so you can’t trust headers alone. What about geo-location filtering? Also not effective. My site is multilingual and receives traffic from many countries by design. These flood networks know that and rotate IPs from all over the world. Maybe they're scraping me because my site has valuable content—but I can't just lock it behind a registration wall. That would ruin the user experience. So I needed something smarter than the usual solutions.
Look my code, MYSQL queries and recommendations below. (Laravel 10 + MYSQL)
