> For the complete documentation index, see [llms.txt](https://docs-lunar.earthly.dev/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs-lunar.earthly.dev/plugin-sdks/bash-sdk/collector.md).

# Collector

The lunar collect CLI for collecting SDLC metadata into a component's JSON, and the environment available to Lunar collectors.

The Collector Bash SDK is a set of Lunar CLI subcommands that allow you to collect SDLC metadata from within a collector, or from external systems, such as your CI system directly.

## Collector environment

Earthly Lunar executes collectors in an environment set up with the following variables:

* `LUNAR_HUB_HOST`: The host of the Lunar Hub.
* `LUNAR_HUB_TOKEN`: The authentication token for the Lunar Hub.
* `LUNAR_HUB_GRPC_PORT`: The gRPC port of the Lunar Hub.
* `LUNAR_HUB_HTTP_PORT`: The HTTP port of the Lunar Hub.
* `LUNAR_HUB_INSECURE`: Whether to skip SSL verification of the Lunar Hub.
* `LUNAR_BIN_DIR`: The directory where the Lunar CLI is installed.
* `LUNAR_COLLECTOR_NAME`: The name of the collector.
* `LUNAR_COLLECTOR_CI_PIPELINE`: The CI pipeline of the component that the collector is running for (if this is a CI collector). 🚧 Coming Soon
* `LUNAR_COMPONENT_ID`: The ID of the component that the collector is running for in `github.com/.../...` format.
* `LUNAR_COMPONENT_DOMAIN`: The domain of the component.
* `LUNAR_COMPONENT_OWNER`: The owner of the component.
* `LUNAR_COMPONENT_HEAD_BRANCH`: The head branch of the PR (branch that contains the changes), if applicable.
* `LUNAR_COMPONENT_BASE_BRANCH`: The base branch of the PR, if applicable. For GitHub native stacks, this is the stack trunk (usually `main`), so diffs against this branch include unmerged lower layers. Lunar Hub matches affected component paths against the individual PR’s file list.
* `LUNAR_COMPONENT_PR`: The PR number of the component, if applicable.
* `LUNAR_COMPONENT_TAGS`: The tags of the component as a JSON array.
* `LUNAR_COMPONENT_GIT_SHA`: The Git SHA of the component that the collector is running for.
* `LUNAR_COMPONENT_META`: The component's `.meta` (an arbitrary key→value map, e.g. `pagerduty/service-id`) as a JSON object. Only set when the component has meta.
* `LUNAR_SECRET_<name>`: Any secret configured for collectors — either via `HUB_COLLECTOR_SECRETS=NAME:VALUE,NAME2:VALUE2` or at runtime using `lunar secret set <name> <value>`.

### CI collector environment

For `ci-{before,after}-job`, `ci-{before,after}-step` and `ci-{before,after}-command` hooks the following entries are also available:

* `LUNAR_CI`: The CI provider identifier (github, buildkite, ...).
* `LUNAR_CI_PIPELINE_RUN_ID`: Unique identifier for the current pipeline run. Pipeline are the top-level units of execution in a CI system. Also known as "workflow" in some CIs.
* `LUNAR_CI_PIPELINE_RUN_ATTEMPT`: Pipeline run attempt (1-based).
* `LUNAR_CI_PIPELINE_DEFINITION_REF`: URL to the pipeline source code.
* `LUNAR_CI_PIPELINE_NAME`: Name of the pipeline, if available.
* `LUNAR_CI_PIPELINE_TRIGGERED_BY_RUN_ID`: For a pipeline run started by another pipeline's run, that run's ID (e.g. in GitHub Actions, the run behind an `on: workflow_run` trigger).
* `LUNAR_CI_JOB_ID`: Job id. Jobs are units of parallel execution within a pipeline. Also known as "stages" in some CIs.
* `LUNAR_CI_JOB_NAME`: Job name, if available.

For `ci-{before,after}-step` and `ci-{before,after}-command` hooks the following entries are also available:

* `LUNAR_CI_STEP_INDEX`: Step index (1-based). Steps are the sequential units of execution within a job.
* `LUNAR_CI_STEP_ID`: Step ID (the step's `id:` in the GitHub Actions workflow), when the step has one.
* `LUNAR_CI_GITHUB_ACTION`: The action a GitHub Actions `uses:` step runs, as `<owner>/<repo>[/<subdirectory>]@<ref>` (e.g. `actions/checkout@v4`, `github/codeql-action/analyze@v3`; the ref as written, a SHA when pinned). Empty for steps that run a script, a local action, a `docker://` image, or a composite or Docker container action.

For `ci-{before,after}-command` hooks the following entries are also available:

* `LUNAR_CI_COMMAND`: Command and arguments of the hooked command, as a JSON string.
* `LUNAR_CI_COMMAND_BIN`: Binary name only (no directory).
* `LUNAR_CI_COMMAND_BIN_DIR`: Directory containing the binary.
* `LUNAR_CI_COMMAND_ARGS`: Command arguments (excluding the binary) as a JSON array.
* `LUNAR_CI_COMMAND_PID`: Process ID of the hooked command.
* `LUNAR_CI_COMMAND_PPID`: Parent process PID of the hooked command.

### `after-ci-pipeline` collector environment

An [`after-ci-pipeline`](https://docs-lunar.earthly.dev/configuration/lunar-config/collector-hooks.md#after-ci-pipeline) collector runs on a Lunar runner for each finished CI workflow run. It gets the collector environment above, with `LUNAR_COMPONENT_ID` and `LUNAR_COMPONENT_GIT_SHA` naming the component and the commit the run built, plus the run itself in:

* `LUNAR_CI_PIPELINE_RUN_ID`: The workflow run ID.
* `LUNAR_CI_PIPELINE_RUN_ATTEMPT`: The run attempt (1-based). A re-run is a new attempt.
* `LUNAR_CI_PIPELINE_NAME`: The workflow name.
* `LUNAR_CI_PIPELINE_RUN_CONCLUSION`: How the run ended (`success`, `failure`, `cancelled`, ...).

For a run another pipeline's run started, `LUNAR_COMPONENT_GIT_SHA` is the commit the chain started from ([details](https://docs-lunar.earthly.dev/configuration/lunar-config/collector-hooks.md#runs-started-by-another-pipeline)), and these are set too:

* `LUNAR_CI_PIPELINE_HEAD_SHA`: The commit the CI system recorded the run at.
* `LUNAR_CI_PIPELINE_TRIGGERED_BY_RUN_ID`: The run that started it.
* `LUNAR_CI_PIPELINE_ORIGIN_SOURCE`: How the Hub found the chain's commit: `tracer`, `run-name`, or `unresolved` (then `LUNAR_COMPONENT_GIT_SHA` is the CI system's commit).

## `lunar collect` CLI command

* Form:

  ```bash
  lunar collect [--component <name>] [--sha <commit>] [--pr <number>] \
                [--json] [--array-append] [--pretty] \
                <json-path> <value> [<json-path> <value> ...]
  ```

`lunar collect` collects SDLC metadata into a component's JSON, taking one or more `<json-path> <value>` pairs. Inside a collector the target component and commit default to the current run, so you collect by passing the paths and values to record — use `--json` for non-string values like numbers:

```bash
lunar collect --json .testing.coverage.percentage 87.4
lunar collect .testing.coverage.source.tool codecov
```

### Out-of-band collection

Collection can also happen outside of Lunar collectors — for example, in a CD pipeline, to associate release information with a previously built component's JSON. Pass `--component` and `--sha` to collect against a specific component and commit:

```bash
lunar collect --component github.com/my-org/my-repo --sha "$COMMIT_SHA" \
  .release.tag v1.2.3 \
  .release.image registry.example.com/my-app:v1.2.3
```

The Hub records the collection and re-evaluates policies for that SHA; verify the merged result with `lunar component get-json <component> --git-sha <sha>`.

With [CI job identity](https://docs-lunar.earthly.dev/docs/authorization/ci-identity.md), the write must target the job's commit or pull request; pass `--pr` for a pull request. A write from a branch the component does not track, without `--pr`, is stored but not evaluated, and only `lunar component get-json --git-sha` returns it. A write with `--pr` is evaluated for that pull request once Lunar Hub has recorded the commit in it; an ordinary pull request whose base branch no component tracks is stored but not evaluated. GitHub native stacks use the stack trunk for component matching, so an upper PR is evaluated when its trunk is tracked.

### `--component <name>`

* Type: `string`
* Optional

The component the metadata is associated with, in `github.com/org/repo` form. Defaults to `$LUNAR_COMPONENT_ID` (set automatically inside a collector); pass it to collect against a specific component.

### `--sha <commit>`

* Type: `string`
* Optional

The full 40-character commit SHA to key the collection on. Defaults to `$GITHUB_SHA`, which GitHub Actions sets automatically — in any other CI, including GitLab, pass it explicitly (for example `--sha "$CI_COMMIT_SHA"`).

### `--pr <number>`

* Type: `int`
* Optional

Optional PR number, to collect against a PR rather than a branch commit.

### `--json`

The `--json` flag is used to specify that the value should be interpreted as a JSON object, JSON array, or JSON scalar (string, number, etc). If the flag is not provided, the value is interpreted as a raw string.

Example:

```bash
lunar collect --json '.foo.bar1' 'true'              # interprets as a JSON scalar (boolean)
lunar collect '.foo.bar2' 'true'                     # interprets as a raw string
lunar collect --json '.foo.bar3' '{"baz": "value 3"}' # interprets as a JSON object
lunar collect --json '.foo.bar4' '["value 4"]'        # interprets as a JSON array
```

Will result in:

```json
{
  "foo": {
    "bar1": true,
    "bar2": "true",
    "bar3": {"baz": "value 3"},
    "bar4": ["value 4"]
  }
}
```

Note: Without `--json`, values are always treated as raw strings regardless of their content. For example, `lunar collect '.foo.bar' '{"key": "val"}'` would store the literal string `"{\"key\": \"val\"}"`, not a JSON object.

### `--array-append`

The `--array-append` flag is used to specify that the value should be appended to an array at the specified JSON path. If the path does not exist, a new array will be created. If the path exists and is not an array, the value will be replaced with a new array containing the existing value and the new value.

Since arrays are concatenated during the component JSON merge, using `--array-append` is equivalent to wrapping the value in a JSON array. The following two commands are equivalent:

```bash
lunar collect --array-append '.foo.bar' 'a value'
lunar collect --json '.foo.bar' '["a value"]'
```

Example:

```bash
lunar collect --json --array-append '.foo.bar' '{"baz": "value 1"}'
lunar collect --json --array-append '.foo.bar' '{"baz": "value 2"}'
```

Will result in:

```json
{
  "foo": {
    "bar": [
      {"baz": "value 1"},
      {"baz": "value 2"}
    ]
  }
}
```

### `--follow-processed-sha`

Write to the component's recorded processing SHA instead of the requested SHA. If commit Y references processing commit X, this flag merges your JSON into X unchanged. Without the flag, collection writes to Y.

The source must be known and available. If it cannot be resolved unambiguously, collection returns an error. Older Hubs ignore this flag and write to the requested SHA.

For example, collect deployment information at the processing commit:

```bash
lunar collect --component github.com/acme/monorepo/api \
  --sha "$GITHUB_SHA" --follow-processed-sha --json \
  .deployment.attempts.run_123 '{"status":"approved","author":"deploy-bot"}'
```

### `--pretty` | `-p`

Pretty-print the JSON output.

### `<json-path>`

* Type: `string`

The JSON path is used to specify the location in the JSON object where the value should be stored. The path is specified using dot notation, where each level of the object is separated by a dot. For example, `.foo.bar.baz` would specify the `baz` property of the `bar` object, which is a property of the `foo` object.

### `<value>`

By default, the value is interpreted as a raw string. If the `--json` flag is provided, the value is interpreted as a JSON object, JSON array, or JSON scalar (string, number, etc).

If the value is `-`, the value is read from standard input. This is useful for piping the output of a command into the `lunar collect` command.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs-lunar.earthly.dev/plugin-sdks/bash-sdk/collector.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
