catalogers

Catalogers

• lunar-config.yml -> catalogers

• Type: array

• Form:

  - <cataloger-object>
  - <cataloger-object>
  - ...

Catalogers are used to synchronize software catalog information, such as domains, and components, with external systems.

The catalog information resulting from catalogers is merged in the order in which the catalogers are defined. This means that if two catalogers define the same field (e.g. owner) within the same component, the last one will take precedence.

After the cataloger information is merged, the information collected from any lunar.yml files, and any component and domain information from lunar-config.yml, is merged to form the final catalog JSON, which is then used by Lunar to form the structure of domains and components internally.

Example catalogers definition:

catalogers:
  - name: GitHub repos
    runBash: |-
        gh repo list <my-org> --json | ... | \
        lunar catalog --json '.components' -
    hook:
      type: cron
      schedule: "0 2 * * *"
  - name: Backstage sync
    runBash: |-
        curl <curl-options> https://<backstage endpoint>/api/catalog/entities/by-query ... | ... | \
        lunar catalog --json '.components' -
    hook:
      type: cron
      schedule: "0 2 * * *"
  - name: DB sync
    runBash: |-
        psql ... -c 'COPY (select * FROM services WHERE ...) TO STDOUT WITH CSV HEADER' | \
        csvjson --no-header-row | ... | \
        lunar catalog --json '.components' -
    hook:
      type: cron
      schedule: "0 2 * * *"
  - name: Pick up catalog files from some central repo
    runBash: |-
      cat verticals.toml | ... | lunar catalog --json '.domains' -
      cat services.toml | ... | lunar catalog --json '.components' -
    hook:
      type: repo
      repo: github.com/foo/software-catalog
  - name: Label any repo that has a certain CI pipeline as "production"
    runBash: |-
      if grep -r --include="*.yaml" --include="*.yml" '^name: Deploy$' ./.github/workflows ; then
        lunar catalog component --tag production
      fi
    hook:
      type: component-rep
  - name: Complex operation
    mainBash: ./my-script.sh
    hook:
      type: cron
      schedule: "0 2 * * *"
  - name: Use an external cataloger
    uses: github://third-party/some-cataloger@v1

Cataloger

• lunar-config.yml -> catalogers.<cataloger-index>

• Type: object

• Forms

  • Uses form (use an external cataloger plugin):

    name: <cataloger-name>
    uses: <cataloger-repo>
    with:
        <input-name>: <input-value>
        ...

  • Run form (define a cataloger inline):

    name: <cataloger-name>
    run<language>: <script>
    hook: <hook-configuration>

  • Main form (define a cataloger inline with a main file):

    name: <cataloger-name>
    main<language>: <script>
    hook: <hook-configuration>

Catalogers are used to dynamically import information about components and domains from external systems. They are run on a schedule or in response to certain events, such as a commit to a repository.

Catalogers can either be imported (Uses form), defined as an inline command (Run form), or defined as a script with a main file (Main form). When a cataloger is defined inline (Run and Main forms), a hook must be specified to determine when the cataloger should run.

name

• lunar-config.yml -> catalogers.<cataloger-index>.name

• Type: string

• Required for Run and Main cataloger forms, Optional for Uses cataloger form

The name of the cataloger. This is used to identify the cataloger in the Lunar UI and logs.

uses

• lunar-config.yml -> catalogers.<cataloger-index>.uses

• Type: string

• Forms

  • GitHub form: github://<org>/<repo>@<version>

  • Local form: ./<path-to-cataloger>

• Required for Uses cataloger form

The uses field specifies an external (plugin) cataloger to use. The cataloger can be a third-party cataloger, or a local cataloger defined in a subdirectory.

with

• lunar-config.yml -> catalogers.<cataloger-index>.with

• Type: object

• Optional

The with field specifies the inputs to pass to the cataloger plugin. The inputs are defined in the cataloger's configuration file.

run<language>

• lunar-config.yml -> catalogers.<cataloger-index>.run<language>

• Type: string

• Required in Run cataloger form

Defines the command to execute when the cataloger is invoked. Only Bash is supported currently. So runBash is the only valid fields.

If an install.sh file is present in the same directory as lunar-config.yml or lunar-cataloger.yml, Lunar will run the install script before running the cataloger script. This script is executed only once in each environment.

runBash

• lunar-config.yml -> catalogers.<cataloger-index>.runBash

• Type: string

The runBash field specifies the bash cataloger script to run.

main<language>

• lunar-config.yml -> catalogers.<cataloger-index>.main<language>

• Type: string

• Required in Main cataloger form

Defines the main file path used to execute when the cataloger is invoked. Only Bash is supported currently. So mainBash is the only valid field.

The file path is relative to the root of the Lunar configuration repository. In the case of an external plugin definition, the path is relative to the plugin directory.

If an install.sh file is present in the same directory as lunar-config.yml or lunar-cataloger.yml, Lunar will run the install script before running the cataloger script. This script is executed only once in each environment.

mainBash

• lunar-config.yml -> catalogers.<cataloger-index>.mainBash

• Type: string

The mainBash field specifies the bash cataloger script to run.

hook

• lunar-config.yml -> catalogers.<cataloger-index>.hook

• Type: object

• Required for Run and Main cataloger forms

The hook field specifies when the cataloger should run. The hook can be a cron job, a repository event, or a component event.

For more information on how to configure hooks, see hooks.