Ask for the Collector ingest endpoint inside the onboarding wizard by kroepke · Pull Request #27252 · Graylog2/graylog2-server

kroepke · 2026-09-02T16:39:04Z

Motivation

On a cluster without a collectors config, every Collectors page redirected to the Settings page and forced the full settings form on the user before anything else. That was confusing: the user came to enroll a collector, not to configure lifecycle thresholds. Design exploration (clickable prototype) settled on asking for the ingest endpoint inside the onboarding wizard, right before the install command that depends on it.

Approach

FirstOnboarding: platform → fleet → IngestEndpointStrip (only when unconfigured) → install command. The enrollment token is minted after the endpoint is confirmed, because the token is signed with the key that first config save creates. Configured clusters skip the strip.
IngestEndpointStrip: slim confirm strip (hostname : port, Confirm). Once the config exists it becomes a status line mirroring the Settings page facts: input running / input exists but not running / no input (with "Create input" when permitted). Changing the port moves the existing ingest input via the inputs API (updateCollectorInputPort).
Permissions checked before the user picks anything. No config permission, or no ingest input and no permission to create one (the built-in Collectors Manager role lacks inputs:create), shows a notice instead of a wizard that cannot finish.
Server: CollectorsConfigResource#put calls CollectorIngestInputService#ensureCanCreateInput before persisting, so a request with create_input can no longer save the config and then 403 on the input.
Deployment, Fleets, Fleet detail and Instances pages redirect to the Overview (which owns the unconfigured state) instead of Settings.
useCollectorPermissions gains canCreateIngestInput / canEditIngestInput.

Server fix bundled here

InputEventListener#inputUpdated only relaunched an updated input if it was RUNNING or SETUP (#3824 kept deliberately stopped inputs stopped). A FAILED input fell into the same bucket: editing it (e.g. to fix a port conflict) left it TERMINATED until a manual start. Found while testing the port change in the strip; FAILED now relaunches like RUNNING. Same fix applies to the regular Inputs page.

Notes

Default hostname stays server-derived (http_external_uri), unchanged.
CollectorsSettings itself is untouched; its own permission gap is tracked in Collectors Settings shows the initial setup form to users who cannot create the ingest input #27250.
Built with zero new local styled components; layout uses Section, Group, Stack, Text from components/common.

Screenshots

Administrator in onboarding flow, connection settings haven't been confirmed yet
Administator confirmed the settings
Administrator edits the previously confirmed settings:
Administrator configures in-use port and sees an error
Unconfigured ingest input with a Collectors Manager role, unable to create/edit inputs

Instead of redirecting an unconfigured cluster to the Settings page, the onboarding wizard now asks for the ingest endpoint itself: platform and fleet first, then a slim "Confirm how Collectors reach this cluster" strip, and only after that the install command. Confirming performs the one-time collectors bootstrap; the enrollment token is minted afterwards because it is signed with the key that first save creates. Once the config exists the strip turns into a status line that mirrors the Settings page facts (input running, input not running, no input), and changing the port moves the existing ingest input along. Permissions are checked before the user picks anything: a user who cannot edit the config, or who cannot create the ingest input (the built-in Collectors Manager role lacks the input permissions), sees a notice instead of a wizard that cannot finish. On the server the PUT checks the input permission before persisting so a request can no longer half-succeed. The Deployment, Fleets and Instances pages redirect to the Overview instead of Settings while the cluster is unconfigured.

InputEventListener only relaunched an updated input if it was RUNNING or SETUP (from #3824, so that deliberately stopped inputs stay stopped). A FAILED input fell into the same bucket: it was removed from the registry and never started again, so fixing e.g. a port conflict by editing the input left it TERMINATED until a manual start. A failed input was never stopped on purpose, and an update is typically the fix for whatever made it fail, so treat FAILED like RUNNING here.

… onboarding steps

This reverts commit fe50e73.

linuspahl

I couldn’t find any issues with the behavior, just added a few ideas for improvements.

linuspahl · 2026-09-04T14:25:13Z

+import { mockCollectorsMutations } from '../../testing/mockMutations';
+import { mockCollectorPermissions } from '../../testing/mockPermissions';
+
+jest.mock('../../hooks');


Usually we don't mock a complete directly and the specific hooks instead.

I saw the same pattern in other tests in this PR, for example for

jest.mock('components/collectors/hooks', () => ({ useCollectorsConfig: jest.fn(), }));

linuspahl · 2026-09-04T14:27:28Z

+  type: 'org.graylog.collectors.input.CollectorIngestHttpInput',
+  content_pack: '',
+  static_fields: {},
+});


We have a similar mock for unconfigured and mockInput in other tests already. let's create a shared fixture for it and only adjust the relevant attributes in this file?

linuspahl · 2026-09-04T14:30:19Z

+
+      await waitFor(() => {
+        expect(screen.getByText(/waiting for connection/i)).toBeInTheDocument();
+      });


Usually we use await screen.findBy instead of expect(...).toBeInTheDocument(),which is async by default. See graylog2-web-interface/CONTRIBUTING.md.

linuspahl · 2026-09-04T14:35:38Z

+jest.mock('../../hooks');
+jest.mock('hooks/useInputsStates');
+jest.mock('components/collectors/hooks/useSendCollectorsTelemetry');
+jest.mock('components/inputs/InputStateBadge', () => () => <span>state badge</span>);


Usually we don't mock the badge component and just check if the related text is displayed correctly. We usually only test it's existence when it is accessible.

linuspahl · 2026-09-04T14:39:53Z

+      Confirm endpoint
+    </button>
+  </div>
+));


We try to avoid the component mocks, in this case we would mock the config API and make sure it returns the "confirmed" status.

I saw the same pattern in other tests in this PR. If possible we try to avoid something like

jest.mock('components/collectors/common', () => ({ CollectorsPageNavigation: () => <div>collectors nav</div> }));

If there is a runtime error in CollectorsPageNavigation we would not catch it.

linuspahl · 2026-09-04T14:58:56Z

+          An administrator must set up the Collector ingest endpoint before Collectors can be enrolled.
+        </Alert>
+      </BodyContainer>
+    );


One small thing that surprised me. I read the "Collectors Manager" role description. Grants full control of collectors and fleets (built-in)" and expected they are able to set up the Collector ingest endpoint.

linuspahl · 2026-09-04T15:19:12Z

+            <Text>{`Ask an administrator to create a Collector Ingest (HTTP) input on port ${config.http.port}.`}</Text>
+          )}
+        </Stack>
+      </Alert>


I was a bit overwhelmed by this alert.

I started reading the headline, saw the "Change" button and assumed I am changing the fact that "No ingest input exists", until I saw the "Create input" below.

What confused me as well, when I click "Change" I see the following section:

When I click "Confirm", I am actually creating an input - I thought I am just changing the address because there was already a path to create the input in the previous alert.

Using more descriptive names for "Change" and "Confirm" would already help, but imo we could just combine both screens in one. For example:

No ingest input exists

Collectors can enroll, but they will not be able to send data until an input is created.

Collectors send their logs to this address. It must be reachable from every Collector host: a load balancer, or this server.

linuspahl · 2026-09-04T15:27:18Z

+  if (!canCompleteSetup) {
+    return (
+      <BodyContainer>
+        <Alert bsStyle="info" title="Collectors are not set up yet">


Imo the issue here is not that "Collectors are not set up yet", it's that no Collector ingest endpoint has been configured yet.

When I read the headline for the first time I thought users with the role "Collectors manager" role no longer have the permission to create collectors.

kroepke added 2 commits September 2, 2026 18:38

Add changelog for PR 27252

cf4ad25

kroepke marked this pull request as ready for review September 4, 2026 10:22

kroepke added 3 commits September 4, 2026 12:36

remove claude-induced changelog

fe50e73

properly gate rendering ui on the permissions necessary to finish the…

4b46ed3

… onboarding steps

kroepke requested review from a team September 4, 2026 11:59

Revert "remove claude-induced changelog"

98d2551

This reverts commit fe50e73.

linuspahl self-assigned this Sep 4, 2026

linuspahl reviewed Sep 4, 2026

View reviewed changes

Merge branch 'master' into feature/collectors-onboarding-ingest-endpoint

49128dd

Navigation Menu

Uh oh!

Ask for the Collector ingest endpoint inside the onboarding wizard - #27252

Ask for the Collector ingest endpoint inside the onboarding wizard#27252
kroepke wants to merge 7 commits into
masterfrom
feature/collectors-onboarding-ingest-endpoint

kroepke commented Sep 2, 2026 •

edited

Loading

Uh oh!

linuspahl left a comment

Uh oh!

linuspahl Sep 4, 2026

Uh oh!

linuspahl Sep 4, 2026

Uh oh!

linuspahl Sep 4, 2026

Uh oh!

linuspahl Sep 4, 2026

Uh oh!

linuspahl Sep 4, 2026

Uh oh!

linuspahl Sep 4, 2026

Uh oh!

linuspahl Sep 4, 2026

Uh oh!

linuspahl Sep 4, 2026

Uh oh!

linuspahl Sep 4, 2026

Uh oh!

linuspahl Sep 4, 2026

Uh oh!

linuspahl Sep 4, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

kroepke commented Sep 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Approach

Server fix bundled here

Notes

Screenshots

Uh oh!

linuspahl left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

linuspahl Sep 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

kroepke commented Sep 2, 2026 •

edited

Loading

linuspahl Sep 4, 2026 •

edited

Loading