The Rise of the Bots: The Good and the Bad

You may be surprised to learn that most website visitors aren’t human at all. Instead, they’re bots: programs designed to do automated tasks. They are often employed by websites, like Facebook and Google, to refresh your news feed or determine page rankings for online searches.

However, bots also have a dark side, and they can work maliciously when employed to impersonate humans to carry out DDoS attacks. Between the good and the bad ones, they account for nearly 52 percent of all web traffic.

The scary part is that bad bots outnumber the good. Of that 52 percent, harmful make up 29 percent, while good ones account for just 23 percent. Even more worrisome is that the most harmful bots — the impersonation bots used for DDoS attacks — make up 24 percent of all web traffic.

The Rise of the Bots

Neither bots nor technology is necessarily to blame for the rise in bot development. We’re struggling to deal with having access to technology because our laws, social systems, and ethical conventions are not yet prepared to deal with the new capabilities we have.

Big data, high-performance computation, and increased networking capabilities and capacity have amplified the rise of bots immensely in recent years, with spam, denial of service, and malware being top concerns among organizations. Increased network technology, especially, paved the way for bot employment, and criminals are taking full advantage of the information the web has to offer.

An army of bots can also help companies collect unimaginable amounts of data. And while big data can fuel growth and innovation, it also raises questions about ethics, privacy, and personal data. Crunching all of the data and selling it for advertisement purposes can be great for companies, but it tends to make many people uncomfortable due to the sheer amount of private information that’s hidden within the context of all the data.

Bots Are More Prevalent Than Ever Before

There’s big business in big data, which is why so many companies are in the “bot industry.” Some bots work on one-to-one systems, often used for tech support or help desk systems, while other companies use bots to gather large amounts of data for market analysis, consumer research, or business intelligence.

Then, there are other companies that resort to bot use more for internal issues, like automating their own systems to adapt to customer demand. Netflix, for instance, uses a bot called the Chaos Monkey, which runs around Netflix’s internal cloud deleting production servers at random and disrupting systems — just to ensure everything will continue to function properly in the event of a simple failure.

ADP took a similar approach because while it’s annoying to get home from work and see Netflix isn’t loading, it’s even more frustrating when you don’t get your paycheck because of a tech issue. ADP started utilizing chaos engineering to make sure engineers catch bugs before they become more serious problems.

The Most Common Bots IT Pros Face

There are two types of malicious bots that create the biggest headaches for IT professionals: DDoS bots and bots that spread malware or ransomware.

DDoS are typically employed on e-commerce websites, gaming servers, or other consumer-facing application areas that disrupt your operations, and they’re often used by hobbyists or criminal groups that make money by taking others offline.

Bots that spread malware or ransomware are used by criminals to infect computers or servers, encrypt the content, and continue to spread to other systems while simultaneously demanding payment from the server or computer owner for the release of the content back to them.

In the future, the use will very likely continue to grow — and as they become more common, IT professionals will need to learn how to manage their websites’ bot traffic efficiently and effectively. To do so, there are a few tips to keep in mind:

1. Fight Back Against the Bad Bots

When it comes to making the most of bot traffic, it’s crucial to protect yourself against bad bots. DDoS developers are continually working to develop new tactics — like the recent pulse wave attack, which generates rapid successions of attack bursts that split a botnet’s attack output. Proper IT security needs to be in place, and IT professionals need to prepare for DDoS attacks by keeping systems up-to-date and protected against malware.

2. Keep the Indexers Happy

Keep an eye on indexers by ensuring your website content is properly structured, relevant, and easy to index. SEO companies are available to help, but they’re not always the final solution. Bots benefit by properly indexing relevant content, and often the best way to get a high ranking in search engines means doing just that — creating easily accessible, relevant content.

3. Take Advantage of Smart Bots

Think about how to factor in the smart bots by considering tasks that can be automated by a bot — like parallel processing, for example, which can be offloaded to an army of bots. Tasks that are good candidates for automation are those that require minimal fields to create a record, like filing an expense report receipt, registering an email address to a mailing list, or even uploading an image of a business card.

Although not all business processes can be automated by them, automating the ones that can will allow employees to spend less time on administrative tasks and more time focusing on strategic and creative aspects of their job.

It’s fairly easy to separate the clearly good from the clearly bad when it comes to bots, but the gray area in the middle tends to be the most interesting. It’s crucial to figure out ways to deal with the impact on our privacy, bulk analysis of data, and the profiling that bots do. Current laws and ethics standards are not yet developed to deal with them, and it will be interesting to watch how this will all play out in the future.