Cataloging Strategy
How to build a complete, trustworthy software catalog in Lunar — achieving full coverage, cleaning up inconsistent existing catalog data, handling monorepos, and enforcing catalog quality with guardra
Lunar decides what to enforce, and where, from your catalog. Domains determine the reporting structure, and tags determine which collectors and policies apply to which component through on expressions. A component that is missing, misfiled, or mistagged is a component your guardrails silently skip.
That makes catalog quality the first thing to work on in a Lunar rollout. In most large organizations only a fraction of repositories ship to production, handle regulated data, or fall in scope for a compliance audit, and targeting your strictest guardrails at that fraction is what keeps them credible with the teams they apply to.
You do not need a good catalog to start. Most organizations do not have one, and the steps below are designed to be run against a messy or largely absent catalog — Lunar becomes the thing that gets you to a good one, and starts returning findings while you are still working on it.
What you are starting from
Most organizations we work with already have a catalog of some kind. It typically has three problems.
It is incomplete. Registering a service was a convention, not a requirement, and nothing enforced it. A meaningful share of repositories were never cataloged at all.
It is inconsistent. Entries were filled in by individual teams over several years, against a schema that drifted. Owners point at people who left. Tags mean different things in different divisions. Nothing has ever validated the data, because nothing has ever depended on it programatically.
It does not identify monorepo components uniquely. A monorepo's catalog entries were written for a system that keys on a service name. Lunar keys on a repository URL plus an optional subdirectory, and most existing catalogs carry no field that maps cleanly onto that.
None of this was a problem before, because the catalog was a directory that people read. Lunar is the first system to make automated, consequential decisions from it, and that is a much higher bar.
The catalog data itself usually comes in one of three shapes, and the strategy below works for all of them:
Catalog service with an API
Backstage, an internal service registry
A server you can query
Files distributed across repos
catalog-info.yaml in each repository
The repositories themselves
Central metadata repository
One repo of YAML/TOML/JSON describing every service
A single repository
Backstage is used as the running example because it is the more common, and because Lunar ships plugins for it. The same phases apply to a home-grown catalog; only the plugins differ. Where a shipped plugin does not fit your format, the AI skills for Claude Code, Codex, and Cursor are the fastest way to produce the collector, policy, or cataloger you need — they know Lunar's SDKs and conventions, so pointing one at your existing catalog schema gets you most of the way.
The shape of the operation
At the highest level this is three moves, in order:
Validate and fix the catalog data at scale
Pull the existing catalog entries in as raw data and run guardrails against them, without letting them shape your catalog yet. Developers get feedback in their pull requests on exactly what is wrong with their own entries, and the cleanup happens in parallel across the organization instead of as a central project.
The steps
1
Ingest every repository
2
Validate and fix the catalog data
3
Ingest the clean catalog
3.4
Cataloger
Monorepos
Make change detection correct in monorepos
Phase 1. Ingest every repository
1.1 Discover every repository
Start with a cataloger that enumerates your source control and creates one component per repository. This is your coverage floor, and it is what makes 100% coverage achievable on day one rather than as the outcome of a migration.
Set default_domain. Every component then lands under a known root domain, which gives you a handle that matches everything: domain tags match hierarchically, so on: ["domain:engineering"] covers engineering and every subdomain beneath it.
Repeat the cataloger once per organization if you have several. See the GitHub Org cataloger documentation for all the available settings.
At this point every repository exists as a component, everything is in one domain, and nothing is classified. That is the correct starting state. The remaining steps add more meaning.
Phase 2. Validate and fix the catalog data
2.1 Collect the existing catalog data
Write a collector that reads your existing catalog entries out of each repository and writes them into the Component JSON verbatim. Do not interpret or reshape the data here. The point is to make the raw entry visible so a policy can judge it.
Lunar ships a collector for Backstage's catalog-info.yaml; a home-grown format needs an equivalent collector of your own.
The component JSON convention is to write the raw descriptor under .catalog.native.<tool>, where the presence of the key is itself the signal: if the collector finds no entry, it writes nothing, and the absence means "not cataloged".
Use a code hook so this re-runs whenever the repository changes, which is what makes the feedback in step 2.2 immediate.
In monorepos, collect the whole tree onto the repository-level component. The subcomponents do not exist yet, so a collector scoped to a single directory would find nothing. Have the collector gather every entry in the repository and write them as a set onto the root component. That is what lets step 2.2 see all the entries at once and check them against each other. The repository-level component has no paths restriction, so it runs on any change anywhere in the monorepo.
How you gather that set depends on how the monorepo declares its components. Two layouts are common:
One entry file per component directory — services/payments/catalog-info.yaml, services/web/catalog-info.yaml, and so on. Walk the tree and record each entry together with the path of the file it came from. That path is what makes each component's identity derivable in step 2.2, so carry it through rather than discarding it after parsing.
One shared entry file listing every component — a single root catalog-info.yaml, typically a multi-document YAML or a list of entities, declaring every service in the repository. Collect the whole file and keep every entity in it. The catch is that there is no per-component file location to fall back on here: every entry shares one path, so the subdirectory has to come from a field inside each entry. If no such field exists today, this is the layout that forces you to introduce one. See Component identity below.
The two layouts often coexist in the same organization, and can coexist in the same repository. You may handle both in the collector rather than mandating one, since consolidating a monorepo's catalog files is a migration in its own right and not necessarily a prerequisite for getting started.
A central metadata repository behaves like the shared-file layout: one location, many entries, so identity has to be carried in a field. The same collector logic applies, driven from the central repo rather than from each component's own tree.
2.2 Validate structure and component identity
Now add a policy that validates the raw entries the collector gathered. This is the step that turns catalog quality into something developers see and act on, because policies report into pull requests and catalogers do not.
Component identity
Lunar identifies a component by repository URL plus an optional subdirectory. The grammar is positional:
On GitHub the first three segments are always host, org, and repo; everything after is the subdirectory. On GitLab, namespaces nest arbitrarily, so the /-/ marker is what separates the project from the subdirectory. See Components for the full rules.
Every component in a monorepo therefore needs its own distinct subdirectory path — github.com/acme/monorepo/services/payments, github.com/acme/monorepo/services/web. Deciding where that subdirectory comes from is the main piece of design work in a monorepo migration, and it is worth doing deliberately rather than discovering it later.
Where the subdirectory comes from
In Backstage, three annotations are candidates, and the one most people reach for first is the one that cannot work:
github.com/project-slugis repo-level by definition. Its documented format isowner/repo, with no path component, and the processor that auto-populates it derives it from the repository alone. Every component in a monorepo receives an identical value. This is also the defaultcomponent_id_annotationfor Lunar's Backstage catalogers, so leaving it at the default is exactly how a monorepo collapses into a single component.backstage.io/source-locationcan carry the subdirectory, but frequently does not. Backstage's built-in location processor resolves it relative to the entity's own location, so an entry inservices/payments/auto-populates tourl:https://github.com/acme/monorepo/tree/main/services/payments/. It is also documented as hand-writable, for the case where the catalog file does not sit with the source it describes. In practice, though, a great many real catalogs have this annotation pointing at the repository root anyway — hand-written once and never revisited, copied between services, or carried over from a migration. Treat it as a hint, not a fact. Validate it rather than trusting it, which is exactly what the next section is for.backstage.io/managed-by-locationis a weaker fallback. It is always present and points at the entry file, so its directory is recoverable. But Backstage documents it as many-to-one — a single location can be the source of many entities — and does not guarantee the value is even aurltype, so it cannot be the primary key in the shared-file layout.
That leaves three realistic options:
The entry file's location
The directory containing each entry file is the subdirectory
One entry file per component. Correct by construction, nothing to maintain, nothing to trust.
An existing location field
Parse source-location down to a repo-relative path
A shared entry file, or syncing from the catalog API — provided you validate the values first
A dedicated field
A new annotation holding the subdirectory verbatim
The existing fields are unreliable, or you want the value explicit and reviewable
Deriving from the entry file's location is the best option wherever it applies, precisely because it depends on no field anyone has to maintain. It is what Lunar's monorepo cataloger does.
The choice between repurposing source-location and introducing a dedicated annotation usually comes down to how much of your existing data is already correct. Measure that first — step 2.2 gives you the number — and repurpose only if the answer is "most of it". Introducing a new field costs one backfill; adopting a field that is quietly wrong for a third of your services costs you a wrong catalog that looks right.
Both annotation-based routes also need a parsing step: Lunar's API-based Backstage cataloger concatenates component_id_prefix with the raw annotation value, so it needs a field already in bare owner/repo/subdir form. A full url:https://… value cannot be pointed at directly.
The shared-entry-file layout from step 2 needs particular care here. Every entity in that file shares one location, so the auto-populated source-location is identical for all of them and points at the repository root. Auto-derivation cannot disambiguate them at all, which makes a hand-written source-location or a dedicated field mandatory in that layout rather than a preference.
Why identity collisions are dangerous
If two catalog entries resolve to the same component id, Lunar merges them into one component, and the merge is silent and lossy in a specific way:
Scalar fields —
owner,domain,branch,description— take the value of whichever cataloger ran last.Array fields —
tags,paths,ciPipelines— are concatenated.
The array behavior is the one that causes real damage. Two colliding services produce one component holding the union of both tag sets, so a service that is not production can inherit production from its collision partner and pull strict guardrails onto itself, while its partner's owner is quietly overwritten with the wrong team. Nothing errors, and the result looks plausible in the UI.
This is the single most important check to get right in a monorepo. Assert it before the data reaches your cataloger, not after.
Your policy should assert, over the full set of entries the collector found:
Every entry resolves to a distinct component id. This is the check that makes the identity decision above real, and the only one that catches a collision before it silently merges.
Whatever field you chose to carry the subdirectory is present on every entry in a monorepo. An entry that omits it falls back to the repository root and collides with every other entry that does the same.
That field is actually specific to the component, not left at the repository root. This is the check that catches a stale
source-location, and in a monorepo a root-pointing value is indistinguishable from a missing one.Each id is well-formed for your git platform — three segments plus a subdirectory on GitHub,
/-/separated on GitLab.The subdirectory corresponds to a real directory in the repository, and where you derive identity from entry-file locations, that it matches the file's own directory. This is what keeps identity stable when the repository is reorganized.
Required fields are present and typed correctly.
Start at enforcement: report-pr so teams see the problems without being blocked, then escalate to block-pr once the backlog is worked down. Group the policies under a cataloging initiative so the effort is trackable as a unit.
2.3 Require a catalog entry
Optional, but this is the step that closes the coverage gap permanently.
Because step 1.1 made every repository a component, you can now enforce that every component has a catalog entry — the mechanism your organization previously lacked.
Roll this out along the enforcement ladder: score to measure the gap without touching anyone, report-pr to surface it in pull requests, then block-pr once coverage is high enough that blocking is fair.
Phase 3. Ingest the clean catalog
3.1 Ingest the catalog into Lunar
Now add the cataloger that reads your existing catalog and maps it onto Lunar's components, domains, and tags. The source can be the same files the collector read, or the catalog service's API if the server adds value your files do not have — resolved group hierarchies, defaulted namespaces, entity relations.
Catalogers run on a hook. A scheduled full sync is the usual backbone — catalog data changes slowly, and a nightly pass is simple to reason about and self-healing. Use a repo hook instead if your source is a central metadata repository, so the catalog refreshes on merge rather than on a schedule.
You can also pair the nightly sync with a commit-triggered cataloger where your source is repo-resident files. The Backstage catalog-info.yaml cataloger ships both by default: a scheduled augment, and an augment-on-commit that re-reads the file as soon as the repo is committed to, so an edited catalog-info.yaml lands immediately rather than at the next nightly tick.
Available Backstage catalogers
Lunar ships four catalogers relevant to this step. They are designed to layer, not to compete:
Backstage REST API
Components and domains from the running server, with subdomainOf / spec.system hierarchy resolved into dotted domain paths.
catalog-info.yaml per repo
Augments each existing component from its own committed file.
Every catalog-info.yaml in a repo
Creates one component per file, keyed to that file's directory. The monorepo answer.
Running those two together is the intended monorepo setup: the first owns the repository-level component, the second adds the subcomponents. The monorepo cataloger's exclude_paths default skips the root entry, so the two never write the same component id.
If you sync from the Backstage API instead, component_id_annotation is the bridge between your catalog's identity and Lunar's — and, per Component identity above, its default of github.com/project-slug is repo-level only. Point it at a field that carries the subdirectory before running it against a monorepo.
For a home-grown catalog you write the cataloger yourself. Here is a basic example.
Mapping onto Lunar's model
Three mapping decisions are worth making deliberately:
Domains. Lunar's domains are hierarchical and dotted (engineering.payments.ledger). If your catalog expresses hierarchy by reference rather than by path — Backstage's subdomainOf and spec.system, for example — the cataloger has to resolve those references into a dotted path. Sync every level of the hierarchy, or a reference to an unsynced parent falls back to a bare name and the tree flattens.
Tags. Namespace imported tags with a prefix, which the shipped catalogers do by default (bs- for Backstage, gh- for GitHub topics). This keeps imported data distinguishable from tags you assign deliberately, and it doubles as gap detection: a component carrying gh- tags but no bs- tags is provably absent from Backstage, and on: supports NOT, so you can target exactly that population.
Precedence. Catalogers merge in declaration order, last one wins, and then lunar.yml 🚧 Coming Soon and lunar-config.yml are applied on top. Declare the broad, low-confidence source first and the specific, high-confidence source last.
lunar-config.yml being the highest-precedence layer is what gives the central platform team a way to augment and correct the catalog without waiting on the upstream system. The merge is per field, so setting owner on a component there overrides just that field — the domain, tags, and everything else keep tracking the cataloger. Use it to pin a value you know the upstream has wrong, or to add metadata the upstream cannot express, and leave the rest to sync.
3.2 Validate the ingested catalog
Optional.
Structural validity is not the same as being correct. Once the data is flowing, add policies for the semantics that only matter because Lunar now depends on them.
The checks worth having here are referential integrity — a component pointing at a domain or system that does not exist — and required tag patterns, which is how you force the classification decisions that scoping depends on. Requiring a data-classification/* tag makes every team state whether their service handles sensitive data, which is exactly the input you need before you can scope a compliance guardrail to it.
3.3 Enrich with heuristics
Optional, and the highest-leverage of the optional steps if large parts of your catalog are unclassified.
Some of what you need to know was never written down anywhere. Rather than asking every team to backfill it, derive it.
A component-cron cataloger can read a component's accumulated Component JSON and classify from evidence Lunar already collected — a deployment in a production Kubernetes namespace, a release pipeline that has actually run, an ingress reachable from the internet:
A component-repo cataloger with clone-code: true does the same from repository signals rather than collected data — the presence of a deploy workflow, a Helm chart, a Terraform production variable file.
Treat heuristics as a supplement, not a replacement. Where a heuristic and a declared value disagree, that disagreement is itself worth a check: a component that deploys to production but is declared experimental is a catalog defect, and often an interesting one.
3.4 Declare paths for monorepo components
Optional, and only relevant if you have monorepos.
The paths field tells Lunar which file changes affect which component. In a monorepo, this determines whether a component's collectors run at all when a commit lands.
A monorepo component automatically gets an implicit <subdir>/* pattern derived from its own name, and by default that is the whole of its trigger set. A component whose only pattern is services/payments/* will not re-evaluate when a shared library it depends on changes. Its checks stay green against stale data. The commit is recorded as handled for that component rather than left pending, so nothing hangs and nothing warns — the result is simply out of date.
paths is how you widen that set. The implicit subdirectory pattern is always present, so entries you add are additional triggers on top of it: in a monorepo, paths only ever expands what the component reacts to, and can never narrow it below its own subdirectory. Declare the shared code and configuration that should also trigger re-evaluation:
Matching is deliberately simple: a single trailing * is a string prefix match, and an entry without one must match a changed path exactly.
Who should own this. The people who know which shared code a component depends on are the component's owners, not the platform team. Maintaining these lists centrally in lunar-config.yml does not scale and goes stale immediately. Put the field in the catalog file the team already edits — catalog-info.yaml, lunar.yml 🚧 Coming Soon, or your internal equivalent — and wire it through the pipeline you have already built:
Step 2.1 collects the declared paths along with the rest of the entry.
Step 2.2 validates them: correct type, paths that exist, no attempt to reach outside the repository.
Step 3.1 emits them to
.components["<id>"].pathsso Lunar's change detection actually uses them.
Without that last wiring the field is inert documentation. It has to reach the Catalog JSON to have any effect.
Putting it together
See also
Catalogers and cataloger hooks — configuration reference
Components — naming rules,
paths, and CI attributionDomains — hierarchy and ownership
Catalog JSON — the merged structure and its precedence rules
Tag Matching with
on— targeting expressionsCataloger Bash SDK — the
lunar catalogcommandAI Skills — agent skills for writing catalogers, collectors, and policies
Last updated