DMOZ a.k.a. The Open Directory Project (ODP)

DMOZ a.k.a. The Open Directory Project (ODP) - written by Phil Craven

Getting a website listed in DMOZ can be very frustrating. We know that being listed will probably help our Google rankings, but getting in can take a very long time. In this article I will explain why it often takes so long and why what you do when submitting your site is sometimes the cause of the delay. But first I will explain what DMOZ is and why it is worthwhile for websites to be listed in it.

DMOZ, also known as The Open Directory Project (ODP), is a large, categorized directory of websites and pages, which is staffed by volunteers. Every website and page that is added to the directory has to be manually reviewed before it is included. Being listed in the directory is free.

Not many people actually use DMOZ for searches in the same way that Yahoo! is used, so the directory itself is of little value in generating traffic. However, its data can be freely downloaded, and any website, however small, can use it. One not so small website that downloads and uses DMOZ's data is Google. In fact, Google's directory is nothing less than the downloaded DMOZ directory.

This has some significant effects for websites that are listed in DMOZ. PageRank is an integral part of Google's ranking algorithm, and higher PageRank helps towards higher rankings. The PageRank within a website is increased by pages from other sites linking to it, and the higher the PageRank of the pages that link to it, the better it is for the receiving site.

A listing in DMOZ creates two significant links into a website - one from DMOZ (Google spiders DMOZ just like any other site) and one from the Google directory. Both of these usually have decent PageRank. Then add the links from the thousands of small sites that have downloaded and use the DMOZ directory, and you can see why it is usually quite beneficial for a website to be listed in DMOZ. Simply being listed in DMOZ can take a website from a Toolbar PageRank value of 3 to 4, and even from 4 to 5.

At the time of writing, the front page at DMOZ states "57,251 editors" (volunteers who review and add websites to the directory), but this is misleading. They don't in fact have that many editors, or anywhere near that many. That number is the total number of editors that they have had since the project started. Most of them are no longer editors. Of the ones that are still editors, a significant proportion of them are not actually active or are only slightly active. So the number of editors who are actively reviewing and adding websites is relatively small.

On the other side of the equation, there is a massive backlog of sites waiting to be reviewed. Each editor can only edit in his or her own categories. Some editors have small categories with very few submissions to deal with, and they can be dealt with very quickly. Others are simply overwhelmed by the mountain of unreviewed sites, and there is little chance of getting through them in the near future.

But the huge backlog, and the relatively low number of active editors, are not the only reasons why websites seem to wait forever to get listed. Many times, the delay is the fault of the person who submitted the website. Imagine that someone submits a site to a category that is reasonably close to what the site is about, but the site really belongs in a different category. What happens? The submission waits in the unreviewed queue of the category to which it was submitted. Sooner or later its turn comes and the editor reviews it, but finds that it belongs in a different category. That editor can't edit the other category, so the submission is passed along to the other category, where it is added to the unreviewed queue. It doesn't jump the queue just because it has already waited in a different queue. Eventually its turn will come again and it will be reviewed - again.

That's the simple course of events when a site is submitted to the wrong category. In practise, though, it is often significantly different. When the first editor reviews the site, often quite a long time after it was submitted, and finds that it doesn't belong in the category, what is s/he likely to think? "If you can't be bothered finding the right category for it, neither can I". And so the site is often sent to a category that is closer to where it belongs but not necessarily to the exact one. The editor there eventually gets to it, and sends it a bit further towards the right one - maybe to the right one this time, and maybe not - and the delays mount up just because the person who submitted the site didn't take enough time to make sure that it was submitted to the right category in the first place. If the submitter can't be bothered, why should anyone else be all that bothered?

So, when submitting a site, always take time to find the right category for it. Don't be tempted to submit it to a category that is higher up the tree than it belongs, because it won't be accepted there and, doing so, could cause unnecessary, self-induced delays.

DMOZ's policy is to include sites that have unique content, which means that many sites don't qualify for inclusion. Among the sites that are likely to be rejected are those that have too much content of an affiliate nature. Some affiliate content is acceptable but when it occupies too much of a site, then the site will probably be rejected.

Another reason why a site may be rejected is because of the submission. If the Title and Description provided in the submission don't follow DMOZ's guidelines, then some editors will think, "If you can't be bothered to spend a little time on it, why should I bother rewriting it for you?", and reject the site. Personally, I find it hard to believe that editors would do that, but I've heard of it happening. So, when submitting a site, read and follow the guidelines. The description is intended to give people an objective statement of what can be found in the site, and not to promote it.

People are not informed that their site has been rejected, and there must be many people out there who think their submissions are still pending when, in fact, they've already been rejected. There's only one way to know the status of a submission and that's to get someone on the inside to tell you. The Open Directory Public Forum, which is run by some of the editors is a good place to find some. They are very helpful there in that might find out the state of play for a website's submission for you.Be careful, The Editors are not allowing people to enquire about their site's submission status in the Forum anymore so you have to get in touch with an Editor directly and not do a public post.

As I mentioned earlier, there are not many active editors when compared to the number shown on DMOZ's front page, but most of those that are active are keen. They are keen to add websites that have unique content, and keen to improve the directory; in fact, they remind me of a colony of ants busying themselves in and around their anthill (the directory), building it up, each tiny bit by each tiny bit. Contrary to what some people think, they do care about the directory and about adding new sites, but they have an uphill struggle because there aren't enough of them.

The original concept of DMOZ was excellent for its time. The DMOZ site's "About" page makes these statements about the concept, and about the reasons for the directory's creation:-

"Automated search engines are increasingly unable to turn up useful results to search queries. The small paid editorial staffs at commercial directory sites can't keep up with submissions, and the quality and comprehensiveness of their directories has suffered. Link rot is setting in and they can't keep pace with the growth of the Internet."

"The Open Directory follows in the footsteps of some of the most important editor/contributor projects of the 20th century. Just as the Oxford English Dictionary became the definitive word on words through the efforts of volunteers, the Open Directory follows in its footsteps to become the definitive catalog of the Web."

But things have changed a lot since DMOZ began in the mid 1990s. Since then, Google came along with very relevant search results, and they were kind enough to show the other engines how to produce such relevant results. That caused dramatic improvements, to the extent that top search engines have been able to provide very relevant search results for some time, and they provide a lot more of them than DMOZ is able to do.

The small paid editorial staffs at commercial directory sites still can't keep up with submissions, but their backlogs are small when compared with DMOZ's massive backlog. According to reports, there are over a million site submissions that are waiting to be reviewed, and delays of several years between submitting a site and it being reviewed are not uncommon. The backlog problem is so huge that many editors have redefined the problem so that it no longer exists. To them there is no backlog, because the submitted sites are not there to be reviewed. They are merely a low priority pool of sites that they can dip into if they want to, and some of them prefer to find sites on their own.

Link rot (dead links) has become widespread in DMOZ through the years, and they certainly can't "keep pace with the growth of the Web". There isn't a single reason for the creation of DMOZ that DMOZ itself doesn't now suffer from.

So how come such an excellent original concept ended up with a directory that has the same problems that it sought to solve, and on a much larger scale?

One reason is that the Web has grown at a much faster pace than was perhaps anticipated, and the DMOZ editors simply can't keep up. Another reason is that there are simply not enough editors who are adding sites to the directory. At the time of writing, the DMOZ front page boasts 69,412 editors, but that is the number of editors that they've had since the beginning, and most of them are no longer there. A recent report stated that there are currently about 10,000 editors who are able to edit, and that only around 3,000 of those are active in building the directory. The word "active" is used to describe editors who actually edit quite often, but as little as one edit every few months is acceptable. The word doesn't mean "busy", although some of them are.

With so few people doing anything, it isn't even possible for them to keep up with the link rot in such a huge directory, and there's the ever increasing problem of listings that link to topics other than what they were listed for. It simply isn't possible for them to maintain the directory as they would like.

The idea of becoming "the definitive catalog of the Web" was a fine one, but it turned out to be an impossible dream. The purpose of DMOZ is dead. Today's search engines produce excellent results in large quantities, and much more quickly than drilling down into a directory to find something.

So is there any value at all in the DMOZ directory? As a useful catalog of the Web, and when compared with the major search engines, the answer is no, although a few people do find it to be a useful research resource. For website owners, the links to their websites that a listing in DMOZ creates are useful for search engine ranking purposes, but even those are becoming less useful as search engines improve, and seek to block out unwanted duplicate content from their indexes.

It was a fine concept, and it looked promising for a while, but the idea of DMOZ becoming the definitive catalog of the Web is gone. Improvements in the search engines eclipsed its value, and the growth rate of the Web meant that it could never achieve its goal. It began with an excellent concept, and they gave it a good shot, but it didn't work. The continuing growth rate of the Web ensures that it can never work. It continues as a good directory of a large number of web sites, but that is all. And not many people use directories when the search engines produce such good results, and so quickly.

The Open Directory Project is the largest, most comprehensive human-edited directory of the Web. It is constructed and maintained by a vast, global community of volunteer editors.

The Republic of the Web

The web continues to grow at staggering rates. Automated search engines are increasingly unable to turn up useful results to search queries. The small paid editorial staffs at commercial directory sites can't keep up with submissions, and the quality and comprehensiveness of their directories has suffered. Link rot is setting in and they can't keep pace with the growth of the Internet.

Instead of fighting the explosive growth of the Internet, the Open Directory provides the means for the Internet to organize itself. As the Internet grows, so do the number of net-citizens. These citizens can each organize a small portion of the web and present it back to the rest of the population, culling out the bad and useless and keeping only the best content.

The Definitive Catalog of the Web

The Open Directory follows in the footsteps of some of the most important editor/contributor projects of the 20th century. Just as the Oxford English Dictionary became the definitive word on words through the efforts of a volunteers, the Open Directory follows in its footsteps to become the definitive catalog of the Web.

The Open Directory was founded in the spirit of the Open Source movement, and is the only major directory that is 100% free. There is not, nor will there ever be, a cost to submit a site to the directory, and/or to use the directory's data. The Open Directory data is made available for free to anyone who agrees to comply with our free use license.

The Internet Brain

The Open Directory is the most widely distributed data base of Web content classified by humans. Its editorial standards body of net-citizens provide the collective brain behind resource discovery on the Web. The Open Directory powers the core directory services for the Web's largest and most popular search engines and portals, including Netscape Search, AOL Search, Google, Lycos, HotBot, DirectHit, and hundreds of others.

You Can Make a Difference

Like any community, you get what you give. The Open Directory provides the opportunity for everyone to contribute.

Signing up is easy: choose a topic you know something about and join. Editing categories is a snap. We have a comprehensive set of tools for adding, deleting, and updating links in seconds. For just a few minutes of your time you can help make the Web a better place, and be recognized as an expert on your chosen topic.