TV shows like CSI: Cyber and others talk about the Dark or Deep Web. What is it?
They are actually two related but different things.
The Deep Web, aka Deep Net, Invisible Net or Hidden Web, is that part of the Internet that is not indexed by standard search engines. When you do a Google or Yahoo search, for example, you will never see anything located in the Deep Web.
The Deep Web is several orders of magnitude larger than the searchable part of the World Wide Web. We only skim the surface of total available content. This surface metaphor is why it is called the Deep Web. What is in the Deep Web?
- Websites that are not registered with any search engine. This could be deliberate as a company is building their first web site. They want to be able to view it and make sure it is working as desired, but do not want just anyone to stumble across it. It could also be an accident: the web builder forgot to register it with search engines.
- Dynamic content or pages returned in response to a query or accessed only through a form. Some process creates dynamic content web pages at the time the page is displayed in the browser, usually based on information provided in a user request or information stored about the user. One example that you often see is the current view of your shopping cart for an online purchase.
- Unlinked content, pages that are not linked to from any searchable page. Search engine web crawlers usually cannot find those pages.
- Pages that contain encoded data or special file formats that are not recognized by search engines.
- Web archives.
- Private web sites that require registration and login.
The Deep Web itself is not evil but a natural result of the development of the Internet. A significant number of web sites deliberately have content in the Deep Web to control access to sensitive or proprietary information, or as part of their ability to provide custom information tailored to specific visitors. In general, you cannot tell if you viewing something from the Deep Web.
The Dark Web is part of the Deep Net that exists on what are called darknets. A darknet overlays the public Internet and requires specific software, configurations or authorization. Dark Web sites often use non-standard communication protocols and ports.
Protocols are the rules that allow two or more network devices to communicate. There are dozens of network protocols, several you have used. TCP (Transmission Control Protocol) is the basic communications protocol used to support Internet communication. Other protocols often run over TCP, like IP (Internet Protocol), FTP (File Transfer Protocol) or HTTP (Hypertext Transfer Protocol). A port is the logical construct of one end of a communication. The first 1,024 port numbers (0 through 1,023) are defined. For example, port 80 is used for the HTTP protocol used for the World Wide Web. There are over 65,000 possible port numbers. Most firewalls block unknown ports unless individually overridden.
Primarily for security reasons, Darknets were originally implemented in the 1970s to be isolated from the ARPANET, which was the origin of the Internet. By 2002, the Dark Web was used for multiple and often illegal purposes:
- Protect information from targeted and mass surveillance.
- Protect dissidents from political reprisal.
- Support whistleblowing and news leaks.
- Support computer crime.
- Provide a market for restricted or illegal items.
- Support file sharing, often in violation of copyright laws.
You will probably never see anything from the Dark Web. Because of the special programs required to access it, it is very difficult to get to the Dark Web without meaning to.
The last word:
The Deep Web is a normal part of the World Wide Web. You are often accessing information from the Deep Web without even knowing it.
You should, however, be concerned about the Dark Web. You, your employees or your children cannot accidentally access the Dark Web. It requires specialized software, not just your favorite browser. That software is, however, available on the searchable web and often free.
For your business, the best defense is a strong network defense strategy and policy. You should limit the protocols and ports available in your internal network to only those necessary for to run your business, and audit those defenses at least once a quarter. Your security policy should require any BYOD (Bring-your-own-device) must also be similarly protected, and prohibit any employee from accessing the Dark Web from any device that is also used for company business.
As for your children, as with many subjects the best defense is conversation. Make sure that they understand the danger of the Dark Net. It is not a safe place to play.
Keep your sense of humor.