[WIP] postgres databases#5618
Closed
janniklasrose wants to merge 40 commits into
Closed
Conversation
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Co-authored-by: Isaac
Add exhaustruct ForceSendFields zero on CreateDatabaseRequest and align the resources.yml comment with sibling phrasing. Co-authored-by: Isaac
Co-authored-by: Isaac
Adds DAB support for Lakebase Postgres roles, mirroring the existing postgres_databases resource. The state holds role_id and parent separately (so bundle variable references resolve), and RemapState recovers role_id from remote.Name via a local strings.TrimPrefix — no shared parser helper. recreate_on_changes fires on either field since both are part of the immutable hierarchical name. Also fixes collectUpdatePathsWithPrefix to drop a parent path when a more specific child path is present; the real Postgres API rejects an update_mask that contains both (e.g. spec.attributes plus spec.attributes.createdb), expecting all sibling fields when the parent is named. Tested end-to-end against AWS prod (basic, recreate, update, bind) as well as the invariant suite. Co-authored-by: Isaac
Two follow-ups to the postgres_roles resource: - Regenerate required-field validation so role_id is required alongside parent, matching the JSON schema (jsonschema.json already lists both under required). Without this, bundle validate accepted a role config missing role_id and the failure only surfaced during deploy. - In PostgresRole.Exists, recognize 404 via apierr.IsMissing and return (false, nil) so bundle deployment bind reports the user-friendly "postgres_role ... is not found" path instead of a generic fetch error. Co-authored-by: Isaac
Missed alongside required_fields in the previous commit. Same generator run, just the second output file. Co-authored-by: Isaac
Previously logged "does not exist" for any GetRole error, including transient failures, before checking apierr.IsMissing. Flip the order so the debug message only fires when the role is genuinely absent. Co-authored-by: Isaac
The live Lakebase API rejects POST without spec.role despite the SDK's
omitempty tag. Interpolate ${workspace.current_user.domain_friendly_name}
into the existing basic/recreate/update fixtures so they run unchanged on
both the local testserver and cloud. Regenerate recorded outputs.
Co-authored-by: Isaac
Add live_errors/missing_role to lock in the live API behavior observed during the 2026-05-19 dogfood2 smoke: POST .../databases without spec.role returns 400 with "Field 'spec.role' cannot be empty". A local Server stub overrides the default testserver auto-fill so the test runs identically on local and cloud. direct engine only; terraform's rollback-on-failure semantics diverge on error paths (same reason jobs/create-error is direct-only). Co-authored-by: Isaac
The real Lakebase API includes database_id in the database status, echoing the create-time query parameter. The testserver omitted it, so local and cloud acceptance outputs diverged after the role parameterization landed. Capture it on create so GET responses match. Co-authored-by: Isaac
Replace ${workspace.current_user.domain_friendly_name} with an
explicit postgres_roles.owner resource so the bundle works for any
caller. The previous approach worked for human users on dogfood2 but
failed on the SP-authenticated aws-prod-ucws because the auto-created
project-owner role's id is not consistently derivable from the
workspace user identity. Declaring the role explicitly sidesteps that
naming variance and exercises postgres_roles as a side effect.
All three tests (basic, recreate, update) pass against the live API on
aws-prod-ucws for both direct and terraform engines.
Co-authored-by: Isaac
Two additional live-validated error paths, both observed against
aws-prod-ucws on 2026-05-19:
- bad_role_ref: referencing a role that does not exist yields
404 NOT_FOUND with message "role not found; role_id:..." and a
trailing [TraceId: <hex>] suffix. Stub mirrors the response shape;
a regex Repls normalizes the TraceId so output is deterministic.
- bad_database_id: an underscore in database_id violates the pattern
^[a-z]([a-z0-9-]{0,61}[a-z0-9])?$. The API echoes the failing value
and the pattern in the message; locking that in catches regressions
on either side.
Findings during probing (not turned into tests): "postgres" as a
database_id is accepted, a 3-char database_id is accepted, and a
postgres_database starting with a digit is accepted — the live API
does not currently validate these inputs despite SDK doc-comments
implying otherwise.
direct engine only; matches missing_role rationale.
Co-authored-by: Isaac
The SDK's RoleRoleStatus already carries role_id; use it directly instead of stripping the "<parent>/roles/" prefix from remote.Name. Matches the catalog convention (Status.CatalogId) and avoids a local string parse. Co-authored-by: Isaac
# Conflicts: # NEXT_CHANGELOG.md # acceptance/bundle/invariant/continue_293/out.test.toml # acceptance/bundle/invariant/migrate/out.test.toml # acceptance/bundle/invariant/no_drift/out.test.toml # acceptance/bundle/invariant/test.toml # acceptance/bundle/refschema/out.fields.txt # bundle/config/mutator/resourcemutator/apply_bundle_permissions_test.go # bundle/config/mutator/resourcemutator/apply_target_mode_test.go # bundle/config/mutator/resourcemutator/run_as_test.go # bundle/config/resources.go # bundle/config/resources_test.go # bundle/deploy/terraform/interpolate.go # bundle/deploy/terraform/pkg.go # bundle/deploy/terraform/util.go # bundle/direct/dresources/all.go # bundle/direct/dresources/apitypes.generated.yml # bundle/direct/dresources/apitypes.yml # bundle/direct/dresources/resources.generated.yml # bundle/direct/dresources/util.go # bundle/internal/schema/annotations.yml # bundle/internal/validation/generated/enum_fields.go # bundle/internal/validation/generated/required_fields.go # bundle/schema/jsonschema.json # bundle/schema/jsonschema_for_docs.json # bundle/statemgmt/state_load_test.go # libs/testserver/fake_workspace.go # libs/testserver/handlers.go # libs/testserver/postgres.go
…ate-only Live testing showed the PATCH update_mask only accepts spec.attributes and spec.membership_roles; the backend rejects spec.postgres_role, spec.auth_method, and spec.identity_type with 400 INVALID_PARAMETER_VALUE "Unknown field path in update_mask". Without declaring these as recreate_on_changes: - direct engine: deploy fails on PATCH and re-plan loops on the same "1 to change" forever - terraform engine: silently no-ops the change (state records new value, remote keeps old, GET returns no spec → invisible divergence) These spec fields aren't marked immutable in the OpenAPI definition, so the generator can't pick them up — declare them in the manual resources.yml until upstream is fixed. Adds an acceptance test that toggles postgres_role and confirms the plan recreates instead of patching. Restricted to the direct engine because the terraform provider still treats the field as updateable and would silently diverge from the bundle. Co-authored-by: Isaac
Resolved conflicts in NEXT_CHANGELOG.md, bundle/direct/dresources/type_test.go, and bundle/schema/jsonschema_for_docs.json. Regenerated the bundle schema, docs, DABs-to-Terraform map, and validation code so generated files match the merged source; this also filled in the postgres_databases/postgres_roles entries that were missing from the Terraform map and required-fields validation. Co-authored-by: Isaac
PostgresRoleState embeds postgres.RoleRoleSpec, which carries its own MarshalJSON. That method gets promoted to PostgresRoleState, so json.Marshal emitted only the spec fields and silently dropped role_id and parent from persisted direct-engine state. The no-op plan masked it via remote_already_set (RemapState refills both from the GET response), but the on-disk state was incomplete and references could break. Add MarshalJSON/UnmarshalJSON mirroring PostgresDatabaseRemote, plus a round-trip test asserting role_id and parent survive serialization. Regenerate the postgres_roles/update direct-engine plans: state now persists role_id/parent and the plans no longer show phantom remote_already_set skips for them. Co-authored-by: Isaac
Exists previously returned the raw API error for any failure, so a 404 on a non-existent database surfaced through `bundle deployment bind` as "failed to fetch the resource, err: ... (404 NOT_FOUND)" instead of the intended "postgres_database with an id '<id>' is not found". Match PostgresRole.Exists: treat apierr.IsMissing as not-found and only propagate other errors. Co-authored-by: Isaac
A merge of postgres-roles into postgres-databases left two postgres_roles entries in the hand-edited apitypes.yml override -- one in alphabetical position and a second appended at the end. Both mapped to postgres.RoleRoleSpec, so generated output was unaffected, but the duplicate key is a hazard. Keep the in-order entry, remove the stray one. Co-authored-by: Isaac
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
janniklasrose
temporarily deployed
to
test-trigger-is
GitHub Actions
Inactive
Collaborator
Integration test reportCommit: f76290f
22 interesting tests: 15 SKIP, 7 KNOWN
Top 24 slowest tests (at least 2 minutes):
|
LuuOW
reviewed
Jun 17, 2026
LuuOW
left a comment
LuuOW
left a comment
There was a problem hiding this comment.
Technical audit: Implementation verified for system consistency and engineering integrity.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Why
Tests