Extend branch caching logic to PR content

[rarkins]

What would you like Renovate to be able to do?

Improve performance by using intelligent cache logic to reduce work validating PR bodies.

If you have any ideas on how this should be implemented, please tell us here.

This should reuse the same data structures we have for branch caches already.

The primary goal here is to avoid the changelog lookups which take up many network requests and time.

We should work out what logic we can use to decide whether to check if a PR body needs updating. If the "fingerprint" holds, then we skip the PR body check altogether.

There is likely two parts to this fingerprinting decision:

• Has the config fingerprint changed? If so then we always validate it
• How old are the updates, and how long since we last validated it?

For example:

• If any release is <2 hours old, maybe we always validate (because release notes are often written/updated soon after release)
• If >2 <24 hours old, we check every 24 hours
• If >1d <7d old, we check daily
• Otherwise, weekly or never?

We also need to think about what to do with releases where we don't have timestamps. In this case we probably use the PR creation date as the starting point and not the release timestamp.

Is this a feature you are interested in implementing yourself?

Maybe

[viceice]

We should also include such intelligent caching to changelog retrieval to cache release notes from github releases much longer for old releases. so we can avoid the network call and used local cache.

Maybe a HEAD request can improve more to check for last modified, so we save bandwidth too.

[rarkins]

Simple initial approach for PR caching: if the PR hasn't been edited in 24 hours, and there's no config change or upgrades change, then skip checking release notes or updating the PR.

Platforms like GitHub have an "updated at" field in PRs, but we are interested in the PR body, not necessarily all of the PR. e.g. if someone approves it or adds a label, that should not trigger us to check it again.

[rarkins]

If PR config fingerprint is unchanged, and PR body hasn't been edited in >24 hours, then skip the PR update check (including changelog lookups)

[rarkins]

Ultimately we want to avoid reaching this clause unnecessarily:

renovate/lib/workers/repository/update/pr/index.ts

Lines 199 to 202 in 638302e

	if (config.fetchReleaseNotes) {
	// fetch changelogs when not already done;
	await embedChangelogs(upgrades);
	}

Currently we con't generate the pr body until further down:

renovate/lib/workers/repository/update/pr/index.ts

Lines 281 to 283 in 638302e

	const prBody = getPrBody(config, {
	debugData: updatePrDebugData(existingPr?.bodyStruct?.debugData),
	});

[RahulGautamSingh]

Simple initial approach for PR caching: if the PR hasn't been edited in 24 hours, and there's no config change or upgrades change, then skip checking release notes or updating the PR.

Platforms like GitHub have an "updated at" field in PRs, but we are interested in the PR body, not necessarily all of the PR. e.g. if someone approves it or adds a label, that should not trigger us to check it again.

Could we not store the last edited time in branchCache and use it?

[RahulGautamSingh]

renovate/lib/workers/repository/update/pr/index.ts

Lines 94 to 96 in a919d1f

	if (existingPr) {
	logger.debug('Found existing PR');
	}

I think we can add the if-check here:

  if (existingPr) {
    logger.debug('Found existing PR');
    // check pr config fingerprint
     QN. Should we use filtered config from the start or begin with whole config and filter later?
     
    // check upgrades same
    QN. Focus only on upgrades fields related to PR body
    
    // check last edited time
  }

Should we create a new PR cache for this, (IMO no)? Adding an object to the existing BranchCache will be simpler.

[RahulGautamSingh]

renovate/lib/workers/repository/update/pr/index.ts

Lines 319 to 326 in a919d1f

	} else if (!config.committedFiles && !config.rebaseRequested) {
	logger.debug(
	{
	prTitle,
	},
	'PR body changed'
	);
	}

If we only update the last-edited-time inside this if-block we won't need to worry about other PR updates.

[rarkins]

Yes add to branch cache

[RahulGautamSingh]

This is the old logic to skip an PR update.

renovate/lib/workers/repository/update/pr/index.ts

Lines 294 to 306 in 3d85b60

	// Check if existing PR needs updating
	const existingPrTitle = stripEmojis(existingPr.title);
	const existingPrBodyHash = existingPr.bodyStruct?.hash;
	const newPrTitle = stripEmojis(prTitle);
	const newPrBodyHash = hashBody(prBody);
	if (
	existingPrTitle === newPrTitle &&
	existingPrBodyHash === newPrBodyHash
	) {
	// TODO: types (#7154)
	logger.debug(`${existingPr.displayNumber!} does not need updating`);
	return { type: 'with-pr', pr: existingPr };
	}

I think it will be safe to use this to filter out the fields for the prFingerprint and wanted confirmation on the same.
The prFingerprint will consists of the prTitle and the fields from BranchConfig & BranchUpgradeConfig that are directly used to construct the prBody.

[rarkins]

I guess so?

[renovate-release]

🎉 This issue has been resolved in version 34.148.0 🎉

The release is available on:

• GitHub release
• 34.148.0

Your semantic-release bot 📦🚀