Skip to content
Behind the build3 min read

Why most of this directory is not indexed

We publish thousands of pages and ask Google to ignore most of them. Here is the bar a page has to clear, why it is set there, and what the current numbers are.

Most directories in this category have one growth strategy: generate a page for every combination of category, qualifier, and audience, submit a sitemap with forty thousand URLs in it, and hope. It works for a while. Then a core update lands and the traffic does not come back, because what was built was a large number of pages rather than a large number of useful pages.

So this site does the opposite thing, and it is worth explaining exactly how, because "we care about quality" is what everyone says.

The bar

Every programmatically generated page (category hubs, use-case pages, platform pages, best-of guides, alternatives, comparisons) has to clear four thresholds before it is allowed into the index:

  • Five live listings. A shortlist of three is a list, not a guide.
  • Three hundred unique words. Unique to that page, not boilerplate repeated across a family.
  • One internal inbound link. If nothing on the site links to a page, the page was generated rather than wanted.
  • An average listing quality of 0.5. Computed from listing completeness, so a page of five thin entries fails even though it has five entries.

A page that misses any of them ships noindex, follow and says so on its own face. It stays live, stays linked, and stays readable. It is simply not submitted for ranking. follow matters as much as noindex here: the listings linked from a gated page still need their link equity, and noindex, nofollow would strand them.

Why those four, and not a quality score

Each threshold is a specific failure this category actually has.

The listing count is aimed at the cross-facet multiplication problem: "best free AI tools for students on Chrome" is four facets deep and will always have two entries. The word count is aimed at template pages, where the only thing that differs between two URLs is a noun. The inbound link is the cheapest honest test of whether a page has a reason to exist. The quality average is there because the first three can all be satisfied by five bad listings.

One composite score would have been easier to compute and impossible to argue with. Four separate thresholds give a page a specific reason for being gated, which is what makes the next part possible.

The gate is published

The current state of every gated page is at /pages: which pages are indexed, which are not, and the actual reason each one failed: "4 live listings, needs 5", not "failed the gate". The numbers are recomputed rather than asserted, so the page is a report rather than a claim.

This is uncomfortable to publish. It says, in public and continuously, how much of the site is not good enough yet. That is the point: a bar nobody can check is a marketing line, and this category has enough of those.

What it costs

Fewer indexed pages, obviously, and slower early growth than the generate- everything approach. The bet is that the pages which do clear the bar are worth more per page and survive updates that the alternative does not. Ask again in a year; the numbers will be on /pages either way.

If you want the other half of this (how ranked order is computed and why paid placement is not part of it), that formula is published in full at /how-ranking-works, including the weights.

All writing