Add a simulated dev/test environment and simulator-based e2e CI

[vishesh92]

Fixes #4

What this does

Issue #4 asked for documentation on running kube-apiserver and
cloudstack-simulator locally, "preferring containers so automated test
workloads can be implemented later." This does both halves.

make e2e-up brings up a complete, unmocked stack: a kind cluster, the
CloudStack simulator with its advanced zone deployed, CloudStack VMs matching
the kind nodes, and the CCM built from the checkout. docs/development.md
walks through the same steps manually, so the environment can be understood and
debugged rather than just invoked.

make e2e-up
make test-e2e        # phase 1: load balancer, nodes, annotations
make e2e-vpc         # switch to a VPC in a project
make test-e2e-vpc    # phase 2: VPC / network ACL
make e2e-down

Why it is worth the files

EnsureLoadBalancer, UpdateLoadBalancer and EnsureLoadBalancerDeleted had
no tests. Only their helpers did, and those mostly pass gomock.Any() for
request parameters, so what the CCM actually sends to CloudStack was largely
unverified. The new suite is 17 tests (13 phase 1, 4 VPC phase) covering load
balancer lifecycle, node initialization, annotations and session affinity, and
the VPC/network ACL path against a real management server. It lives in
test/e2e behind the e2e build tag, so it stays out of make test and
go build ./..., and needs no new module dependencies.

Bugs this found and fixes

All CCM-side, all exposed by running the code end to end:

• Project scoping (three call sites). updateNetworkACL,
  getNetworkIDFromIPAddress, and the disassociation check in
  EnsureLoadBalancerDeleted fetched CloudStack resources without the
  configured project. On a VPC owned by a project, every LoadBalancer service
  failed with error fetching Network with ID and never received an ingress
  address, and the public IP leaked on deletion.
• Wrong error variable / field mismatch in getNetworkIDFromIPAddress: a
  failed network lookup was reported as success, and the guard checked
  Networkid while the lookup used Associatednetworkid. Either could hand the
  caller an empty network ID, which GetNetworkByID resolves as an unfiltered
  network list rather than rejecting.
• Load balancer rule leak. CloudStack does not enforce unique rule names,
  but loadBalancer.rules is keyed by name, so a duplicate displaced its twin
  in the map and survived EnsureLoadBalancerDeleted with no service left to
  reference it. Duplicates are now tracked and removed on both reconcile and
  delete, without disturbing the rule that is kept.
• Two startup/response panics hardened: getManagementServerVersion sliced
  the version string to three components without a length check (crashing the
  controller on a short version), and getPublicIPAddress guarded on the result
  count but indexed the slice.

Each fix has unit tests; the project-scoping and rule-leak fixes are also
covered end to end by the VPC phase.

CI

.github/workflows/e2e-simulator.yml runs on pull requests and pushes to
main, as a 2x2 matrix:

Kubernetes	CloudStack
v1.37.0, v1.36.4	4.22.1.0, 4.20.2.0

The CloudStack axis buys branch coverage, not just version coverage: 4.22+
updates a rule's CIDR list in place, 4.20 deletes and recreates it. A shared
build job compiles the image once and the four cells run in parallel; the
simulator and kind node images are cached between runs, so the workflow costs
about as much wall-clock as a single run. Failures upload simulator logs, CCM
logs, node and service dumps, and CloudStack-side state.

Test invocation goes through the make targets rather than being duplicated in
the workflow, so CI and a local run cannot drift apart. Not added as a required
check in .asf.yaml yet — worth letting it prove itself stable first.

Known limitation

kind starts kubelet with --provider-id=kind://..., and Kubernetes only lets a
provider ID be set once, so the CCM never assigns external-cloudstack://<uuid>
in this environment. TestNode_ProviderID detects that, logs the value it would
have assigned, and reports itself as skipped, so the gap stays visible instead
of quietly passing. Documented in docs/development.md.

Notes for reviewers

• The README's Debugging section moved to docs/development.md, and the
  stale cloudstack/simulator Docker Hub link is corrected to
  apache/cloudstack-simulator.
• .gitignore now covers the local cloud-config, cmk-config and
  kube-config files, which hold live credentials and were previously
  untracked but not ignored.
• A handful of untested behaviours are deliberately left for follow-up PRs:
  UpdateLoadBalancer node add/remove, the proxy-protocol and UDP/TCP-Proxy
  paths, spec.loadBalancerSourceRanges, and VPC + Firewall (the tier offering
  advertises NetworkACL only).

Testing

Ran the full harness locally against apache/cloudstack-simulator:

• phase 1: 12 pass, 1 informative skip (provider ID, above), ~100s
• phase 2 (VPC): 4 pass, ~30s; the project-scoping and rule-leak fixes were
  confirmed by reproducing each bug against the live simulator before the fix
• make test, golangci-lint run, go vet -tags e2e and go build ./... are
  clean

The CCM's k8s.io/* v0.24 libraries worked fine against a v1.37 API server.

🤖 Generated with Claude Code

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 66.66667% with 14 lines in your changes missing coverage. Please review.
✅ Project coverage is 50.64%. Comparing base (151452f) to head (13692b4).

Files with missing lines	Patch %	Lines
cloudstack_loadbalancer.go	63.15%	12 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #105      +/-   ##
==========================================
+ Coverage   50.05%   50.64%   +0.59%     
==========================================
  Files           4        4              
  Lines         975     1001      +26     
==========================================
+ Hits          488      507      +19     
- Misses        473      479       +6     
- Partials       14       15       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Pull request overview

Copilot reviewed 24 out of 25 changed files in this pull request and generated 4 comments.

Suppressed comments (1)

cloudstack_loadbalancer.go:487

• The netErr branch returns the wrong error variable (it returns err from the previous GetPublicIpAddressByID call). If the first call succeeded (err==nil) but GetNetworkByID fails, this currently returns a nil error and masks the failure.

		network, _, netErr := cs.client.Network.GetNetworkByID(ip.Associatednetworkid, cloudstack.WithProject(cs.projectID))
		if netErr != nil {
			klog.Errorf("Failed to fetch the network for id: %v", ip.Associatednetworkid)
			return "", err
		}

[Copilot]

Pull request overview

Copilot reviewed 24 out of 25 changed files in this pull request and generated 2 comments.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Copilot reviewed 24 out of 25 changed files in this pull request and generated 5 comments.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Copilot reviewed 26 out of 27 changed files in this pull request and generated 6 comments.

[Copilot]

Pull request overview

Copilot reviewed 26 out of 27 changed files in this pull request and generated 2 comments.

[Copilot]

Pull request overview

Copilot reviewed 26 out of 27 changed files in this pull request and generated no new comments.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Pull request overview

Copilot reviewed 26 out of 27 changed files in this pull request and generated 2 comments.