Skip to content

GH-50170: [CI][Packaging][Linux] Fix cache#50185

Merged
kou merged 1 commit into
apache:mainfrom
kou:ci-packaging-linux-optimize
Jun 16, 2026
Merged

GH-50170: [CI][Packaging][Linux] Fix cache#50185
kou merged 1 commit into
apache:mainfrom
kou:ci-packaging-linux-optimize

Conversation

@kou

@kou kou commented Jun 15, 2026

Copy link
Copy Markdown
Member

Rationale for this change

We want to reduce CI time as much as possible for developer experience and avoiding consuming shared resources.

What changes are included in this PR?

  • Use apache/infrastructure-actions/stash instead of actions/cache to use larger cache storage
  • Enable ccache
  • Increase ccache size because 500MB is small for our case
  • Fix Docker image cache
  • Disable RapidJSON's ccache detection that causes needless ccache call such as ccache ccache cc ...
  • Use Ninja instead of Make

Are these changes tested?

Yes.

CI time is reduced to 20min-60min from 60min-120min.

Are there any user-facing changes?

No.

@github-actions

Copy link
Copy Markdown

⚠️ GitHub issue #50170 has been automatically assigned in GitHub to PR creator.

@github-actions github-actions Bot added Component: C++ CI: Extra: Package: Linux Run extra Linux Packages CI awaiting committer review Awaiting committer review labels Jun 15, 2026
@kou

kou commented Jun 15, 2026

Copy link
Copy Markdown
Member Author

Before: https://github.com/kou/arrow/actions/runs/27544568196/usage
After: https://github.com/kou/arrow/actions/runs/27553127454/usage (Some slow jobs didn't have cache yet. Sorry.)

@kou kou force-pushed the ci-packaging-linux-optimize branch from 6b04d77 to 210276f Compare June 16, 2026 02:35
export CCACHE_COMPRESSLEVEL=6
export CCACHE_DIR="${PWD}/ccache"
export CCACHE_MAXSIZE=500M
export CCACHE_LOGFILE="${PWD}/ccache.log"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for debugging why ccache couldn't cache something.

@github-actions github-actions Bot added awaiting changes Awaiting changes and removed awaiting committer review Awaiting committer review labels Jun 16, 2026
@kou kou force-pushed the ci-packaging-linux-optimize branch from 210276f to 3972909 Compare June 16, 2026 04:56
@github-actions github-actions Bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Jun 16, 2026
@kou kou force-pushed the ci-packaging-linux-optimize branch 2 times, most recently from d3a366b to bf728f3 Compare June 16, 2026 07:57
@kou

kou commented Jun 16, 2026

Copy link
Copy Markdown
Member Author

@kou kou force-pushed the ci-packaging-linux-optimize branch from bf728f3 to 2b09658 Compare June 16, 2026 20:48
@kou

kou commented Jun 16, 2026

Copy link
Copy Markdown
Member Author

+1

25h -> 15h

@kou kou merged commit 39770de into apache:main Jun 16, 2026
70 of 72 checks passed
@kou kou deleted the ci-packaging-linux-optimize branch June 16, 2026 23:45
@kou kou removed the awaiting change review Awaiting change review label Jun 16, 2026
@conbench-apache-arrow

Copy link
Copy Markdown

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 39770de.

There were 6 benchmark results indicating a performance regression:

The full Conbench report has more details. It also includes information about 6 possible false positives for unstable benchmarks that are known to sometimes produce them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant