InfoSec Bullshido – Dark Web

When considering the term “dark web”, I had trouble deciding whether to post under InfoSec Basics or InfoSec Bullshido. On the one hand, understanding the term can be quite useful for understanding how the Internet is more complex than most people realize. On the other, there are few terms being thrown around with more reckless abandon by people who either don’t fully understand them, or are trying to scare you (or both).

“Dark web” and “deep web” (and other similar terms, like “darknet”) are now used interchangeably by the vast majority of the non-InfoSec community and, I suspect, by many even in the InfoSec community.

These terms once had precise definitions given by technical experts trying to make what they considered to be important distinctions. Unfortunately, most are no longer interested in those distinctions, which have now been essentially lost through adoption into colloquial usage. I mourn for these experts, but I fear their war may be lost. Any who refuse to acknowledge the tyranny of the masses in this case risk descending into pedantry.

For those interested in precision, however (as opposed to pedantry), the “deep web” (https://en.wikipedia.org/wiki/Deep_web) generally refers simply to that part of the Internet which is not indexed by standard search engines. Sometimes referred to as the “invisible web” or “hidden web”, the primary contrast is with the “surface web”, which is accessible through a search engine.

Put another way, it’s possible to use a standard browser, such as Chrome, Firefox, Safari, or Edge to access both the surface web and the deep web using a direct URL (eg, https://www.til-technology.com) or IP address (such as https://1.1.1.1/). The deep web, by definition, is not searchable, so you would have to know the exact URL or IP address in advance, and would not be able to find it using Google (https://www.google.com), DuckDuckGo (https://www.duckduckgo.org), or another search engine.

Does the fact that it’s not searchable mean that it’s bad or dangerous?

No. Not at all. The deep web includes things like email, private pages (such as social media pages, depending on your privacy settings), the contents of many cloud services, content behind paywalls (such as media companies), and others.

Interestingly, there is a grey area between the surface web and the deep web, between content that is “easily” searchable by various means, and content which is actively prevented from being searchable. The grey area consists of sites which are not blocked, but are not currently “searchable” due to limitations in our “web crawling” technology or decisions made by search engine developers. One interesting example is archive services such as the Wayback Machine (https://web.archive.org/), which is deliberately not searched by services such as Google.

So, what about the “dark web”? That’s the bad one, right?

Well, not necessarily.

The dark web (https://en.wikipedia.org/wiki/Dark_web) consists of content which exists on “darknets” (https://en.wikipedia.org/wiki/Darknet). A darknet (https://en.wikipedia.org/wiki/Darknet) is a type of “overlay network”, which means that it is a section of the Internet which can only be accessed using specialized tools (software, system configurations, communication protocols, and the like).

Most people who have heard of the dark web generally associate it with the Tor network (https://en.wikipedia.org/wiki/Tor_(anonymity_network)), but there are a number of other services available. The name “Tor” is actually derived from the acronym for “The Onion Router”, which was developed to provide privacy protection. Leaving aside the fascinating history of the project and the network, the key idea is to protect the personal privacy of users, and their freedom to conduct confidential communications without monitoring.

While many associate the dark web with criminal activity, the intended purpose of many dark web services is anonymity. To vastly oversimplify, the basic approach taken by the Tor network is to have a network of servers which randomly “bounce” traffic around several times, through servers which encrypt traffic and only look at the immediate origin and the next destination. In theory, this means that someone trying to trace activity would have to have access to every server in the “chain” (or every layer of the onion) in order to track the connection back to its origin, and there’s no way to determine what servers will be in the chain during each connection. (I’m probably butchering that explanation, but suffice it to say that tracking someone using the Tor network is beyond the capacity of most attackers – even nation states have a difficult time of it.)

All of that said, the existence of the dark web actually highlights a number of broader issues and challenges. Should people be forced to identify themselves when they use the internet? What is the role of government, and what are the rules under which they should operate? What about private companies?

I think they key questions here are what privacy is, and whether people have a right to it.

A) What is privacy?

The concept of privacy has changed dramatically over the years. At one time, a person could acheive privacy by walking a few feet away from others. Now, in many countries, we are (or can be) under some degree of surveillance at all times. Think about traffic cameras, security cameras in businesses, home security systems, home assistants, drones, satellite images, and endless other things.

In our online lives, we are usually identified by a user id and/or an IP address, and various entities will have some degree of information about what we do. Our ISP (Internet Service Provider) and the services we are using are among those, but what about our Operating System, browser, VPN (Virtual Private Network) provider, employer, and so on? Can/should they use that information in any way they wish? Can they provide the information to anyone who asks for it? What about law enforcement? What about court orders or search warrants? And so on. A big issue for future discussion is around how laws necessarily lag behind our technology by years or decades.

B) Do we have an inherent right to privacy?

This may be the defining question of our age. In some countries, the opinion is “yes”, in others “no”, but it’s hard to even start a discussion without a clear definition of the term. See point A, rinse, and repeat.

Leaving aside these societal questions for another time (or many other times), there are a number of people who want anonymity to protect themselves while doing what they consider to be ethical or necessary things. Examples include whistle-blowers and journalists – in some cases they are knowingly breaking the law in the name of a greater good.

On the other hand, many want anonymity to protect themselves while doing things that most would agree are “criminal” or “unethical”. For these people, the dark web is a very useful and popular way to communicate and gather.

Ah ha! So, aside from a few whisle-blowers and journalists, the dark web (or deep web, or whatever) is where the bad stuff happens, right?

Well, to a degree. Unfortunately, a significant amount of the content on the dark web is criminal in nature. However, you don’t need the dark web to do bad things.

Wait, what?

Yah. Nowadays, a great many “illegal” services are freely available on the “open” Internet. And, leaving aside nation-state actors, even stopping the worst actors can be extremely difficult.

For a simple illustration, let’s consider the calls that Alice constantly gets from people wanting to clean the duct-work in Alice’s house. Alice has asked them to stop, registered with Canada’s National Do Not Call List (https://lnnte-dncl.gc.ca/), called the police, and so on, but nothing has helped.

One of the problems is that the calls are being made from outside Canada. In the best case, the operator of the service might be one of the big ones that law-enforcement is going after – this would generally mean a large task-force of law-enforcement from multiple countries. If successful, the task force might “take down” the operator... who would quickly be replaced by a dozen smaller operators, and the cycle repeats.

Another problem is that government regulations and laws differ between countries, and so do diplomatic relations and treaties between those countries. Where are the operators? Where are the victims? Where are the servers or services located? Is the activity illegal in any/all of these countries? Clearly, this is an extremely difficult challenge that a lot of very smart people are working on addressing, with varying degrees of success.

Getting back to our main point, though, many people describe the dark web as a scary place where bad people do bad things, and many vendors will try to sell products claiming to find bad stuff in the dark web since only they can protect you from said bad people and things.

The truth is more complicated, as usual.

To summarize, we can conveniently describe the Internet as being separated into three main “areas”: The surface web, which consists of most content accessible with a standard search engine; the deep web, consisting of content which is private or not being made available to search engine indexing; and the dark web, which is accessible using special tools/permissions.

The Internet can be a dangerous place. Period. And while it can certainly be argued that the dark web is more dangerous, I really don’t think we should be generating FUD (https://en.wikipedia.org/wiki/Fear,_uncertainty,_and_doubt) by preying on people’s lack of understanding of what the dark web is, or by under-playing the dangers which are right in front of us on the open Internet (ie, the surface web).

Most people can probably just ignore the dark web in their day-to-day lives, unless they are a journalist or whistle-blower concerned about their personal safety, someone trying to conceal other illicit activity, or are involved in law-enforcement or security research.

That said, I think the broader question that should be of concern to everyone is how we, as a society, define privacy and whether we should consider it a basic human right. The answer to that question will define what we consider acceptable as a society, and will influence laws, government regulation, corporate behaviour, and many other vital issues of the future.

Cheers!