Skip to content

Improve test-local phase reporting and stabilize MongoDB CI#389

Merged
shaypal5 merged 1 commit into
masterfrom
codex/improve-test-local-exit-reporting
Jun 12, 2026
Merged

Improve test-local phase reporting and stabilize MongoDB CI#389
shaypal5 merged 1 commit into
masterfrom
codex/improve-test-local-exit-reporting

Conversation

@shaypal5

@shaypal5 shaypal5 commented Jun 12, 2026

Copy link
Copy Markdown
Member

Closes #362.

Summary

  • Capture the main pytest phase exit code immediately after it runs.
  • Capture the optional serial local pytest phase exit code separately.
  • Aggregate the final script exit code explicitly and report which phase passed, failed, or was skipped.
  • Keep running the serial local phase when it is requested, even if the main phase fails.
  • Stabilize MongoDB CI by avoiding Dockerized MongoDB on GitHub-hosted Windows runners, where the Docker daemon is not reliably available.
  • Keep live Dockerized MongoDB coverage on non-Windows runners while using the existing pymongo_inmemory path for Windows MongoDB matrix jobs.
  • Split MongoDB CI test steps into explicit Docker and in-memory paths instead of using an inline boolean expression.
  • Scope the pymongo_inmemory tarfile deprecation warning filter to in-memory Mongo client startup, preserving the repo's -W error posture without mutating global tarfile behavior.

Validation

  • bash -n scripts/test-local.sh
  • Stubbed pytest to verify main phase failure exits non-zero, still runs serial local tests, and reports the main phase failure.
  • Stubbed pytest to verify serial local failure exits non-zero and reports the serial phase failure.
  • Stubbed pytest to verify both phases passing exits zero.
  • ./scripts/test-local.sh pickle
  • CACHIER_TEST_VS_DOCKERIZED_MONGO=false pytest -m mongo --cov=cachier --cov-report=term
  • Parsed .github/workflows/ci-test.yml with PyYAML.
  • ruff check tests/mongo_tests/helpers.py
  • git diff --check

CI scope note

The PR now intentionally changes CI behavior: Windows MongoDB matrix jobs no longer attempt to start a Linux MongoDB container through Docker. Non-Windows MongoDB matrix jobs still run against a live Dockerized MongoDB service and remain the live-Mongo compatibility signal. Windows MongoDB matrix jobs remain useful compatibility coverage over the Mongo backend API using pymongo_inmemory.

Milestone

No open repository milestones are currently available, so none was assigned.

@codecov

codecov Bot commented Jun 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.95%. Comparing base (92fce34) to head (52d5492).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #389   +/-   ##
=======================================
  Coverage   99.95%   99.95%           
=======================================
  Files          16       16           
  Lines        2109     2109           
  Branches      251      251           
=======================================
  Hits         2108     2108           
  Partials        1        1           
Flag Coverage Δ
local 66.57% <ø> (ø)
mongodb 40.01% <ø> (ø)
postgres 42.05% <ø> (ø)
redis 44.57% <ø> (ø)
s3 41.29% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 92fce34...52d5492. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves local test script phase reporting/exit-code aggregation and stabilizes MongoDB CI by avoiding Dockerized MongoDB on GitHub-hosted Windows runners while keeping live MongoDB coverage on Linux. Also updates the Mongo test helper to opt into Python’s safe tar extraction behavior for the in-memory Mongo path under stricter warning settings.

Changes:

  • Refactors scripts/test-local.sh to capture and report separate exit codes/statuses for the main pytest phase and optional serial-local phase, then aggregates a final exit code.
  • Updates GitHub Actions MongoDB matrix behavior to skip Dockerized MongoDB startup on Windows and select in-memory Mongo there, while keeping Dockerized Mongo on Linux.
  • Adds a tarfile extraction filter configuration hook before instantiating pymongo_inmemory’s Mongo client, and updates README wording to match the new CI behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
tests/mongo_tests/helpers.py Configures tarfile extraction filter before creating an in-memory MongoDB client.
scripts/test-local.sh Adds a helper for phase execution/reporting and explicitly aggregates per-phase exit codes.
README.rst Updates documentation to reflect Linux live-Mongo vs Windows in-memory CI behavior.
.github/workflows/ci-test.yml Skips Dockerized Mongo on Windows and sets CACHIER_TEST_VS_DOCKERIZED_MONGO per Mongo test step.

Comment thread scripts/test-local.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: ci GitHub Actions, pre-commit, codecov, or automation config. area: docs README, examples, contributor docs, or repo documentation. area: mongo MongoDB backend or MongoDB test services. bug impact: reliability Correctness, race, flaky-test, timeout, or robustness improvement. outcome: merged Merged into the repository. source: codex Opened from a Codex-assisted workflow. type: bugfix Fixes a user-visible bug or regression. type: ci GitHub Actions, CI configuration, or automation workflow change.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve test-local.sh multi-phase pytest exit handling and failure reporting

2 participants