Summary

Currently it seems if master branch doesn't build for 1-2 days then the cached ESP-IDF install (1.6GB) and Zephyr workspace (3.1GB) caches expire.

Then each PR branch has to create its own redundant cache instead of falling back to the default branch cache, which is expensive and quickly blows our 10GB cache limit.

For example, there was recently an approximately 16 hour window between master branch builds of esp32 port:

❯ gh run list -b master -w "esp32 port" --json createdAt
[
  {
    "createdAt": "2025-09-24T01:43:05Z"
  },
  {
    "createdAt": "2025-09-23T05:09:37Z"
  },

These are the cached copies of esp-idf and their associated branches:

❯ gh cache list --json ref,key,createdAt --key esp-idf
[
  {
    "createdAt": "2025-09-24T02:01:57.224652Z",
    "key": "esp-idf-v5.4.2-py3.12.3",
    "ref": "refs/heads/master"
  },
  {
    "createdAt": "2025-09-24T01:46:54.024183Z",
    "key": "esp-idf-v5.4.2-py3.12.3",
    "ref": "refs/pull/18128/merge"
  },
  {
    "createdAt": "2025-09-24T00:39:09.770155Z",
    "key": "esp-idf-v5.4.2-py3.12.3",
    "ref": "refs/pull/18050/merge"
  },
  {
    "createdAt": "2025-09-24T00:14:11.887394Z",
    "key": "esp-idf-v5.4.2-py3.12.3",
    "ref": "refs/pull/15592/merge"
  },
  {
    "createdAt": "2025-09-23T08:38:24.706513Z",
    "key": "esp-idf-v5.4.2-py3.12.3",
    "ref": "refs/pull/18120/merge"
  }
]

... and it seems like about 3 hours after the first run the cached ESP-IDF entry aged out, resulting in 4 more copies being cached in branches - this uses about 8GB instead of 1.6GB and slows down the build.

I'm not sure why the cache entry aged out so quickly, GitHub is supposed to delete the least recently used cache entries and that one was only 3 hours old in theory. I guess it's either a bug, or to do with our cache being perpetually overfull at the moment (at time of writing we're using 33GB of 10GB) - so maybe when it does clean it's more of a "delete everything". Apparently they're going to start enforcing a hard limit in October, which might actually help with not triggering this pathology...

Testing

Need to merge this and see if it helps. If it doesn't:

• We could run it more frequently than once a day. If that's using too many resources then we could conditionally skip the build jobs on the scheduled runs and only run the cache hit jobs...
• We could add either a scheduled or a post-success job on the master branch which goes back and deletes any redundant cache entries on other branches. This shouldn't be necessary if GitHub's LRU cleanup worked properly, but it seems like it doesn't.