Include realized build time savings in experiment summary

[erichaagdev]

❄️ Winter Update PR Navigator ❄️

1. ⛄ Include realized build time savings in experiment summary #261
2. Include performance characteristics and support -x for Gradle experiment 1  #262
3. Always read project name from Build Scan data #263
4. Gradle init script improvements #264
5. Adopt Foojay Toolchains and Kotlin DSL plugins in build logic #265
6. Add variable to query debug mode #266
7. Modernize Java launch code #267
8. Display serialization factor in summary of all experiments #272
9. Display potential build time savings in experiment summaries #274
10. Update the contract of command line tools #283

⛄ = you are here

Summary

ℹ️ This PR only partially implements #206. This felt like a good place to stop. More PRs will be coming.

This PR introduces the following changes:

• The API command line tool now includes Effective task execution duration in the CSV response
• The Build Caching Leverage section of the experiment summary has been renamed to Performance Characteristics
• Realized build time savings is now included in the Performance Characteristics section of the experiment summary

Example

Performance Characteristics
----------------------
Realized build time savings:  1m 7.377s wall-clock time (from 1m 11.542s to 4.165s)
Avoided cacheable tasks:      292 tasks, 4m 44.680s total saved execution time
Executed cacheable tasks:       1 tasks, 0.045s total execution time
Executed non-cacheable tasks: 292 tasks, 10.686s total execution time

WARNING: Not all cacheable tasks' outputs were taken from the build cache in the second build. This reduces the savings in task execution time.

Notes

• I ensured this change was backwards compatible with the current version of the build-scan-support-tool.jar. If the new Effective task execution duration field is not included in the command line CSV response then Realized build time savings will not be displayed.
• For the display format of Realized build time savings I decided to take @runningcode's suggestion. I even tried a few display variations myself and liked his suggestion the best. Let me know if the format should be changed.

Testing

Gradle

Experiment 1

./01-validate-incremental-building.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/gradle/7.x/ge \
  -t 'build'

No visual differences.

Experiment 2

Online

./02-validate-local-build-caching-same-location.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/gradle/7.x/ge \
  -t 'build'

Offline

./02-validate-local-build-caching-same-location.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/gradle/7.x/ge \
  -t 'build' \
  -x

No visual differences.

Experiment 3

Online

./03-validate-local-build-caching-different-locations.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/gradle/7.x/ge \
  -t 'build'

Offline

./03-validate-local-build-caching-different-locations.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/gradle/7.x/ge \
  -t 'build' \
  -x

No visual differences.

Experiment 4

./04-validate-remote-build-caching-ci-ci.sh \
  -1 https://ge.solutions-team.gradle.com/s/fvwne3ht4xgjg \
  -2 https://ge.solutions-team.gradle.com/s/tjiwbi2b7oioo

Experiment 5

./05-validate-remote-build-caching-ci-local.sh \
  -1 https://ge.solutions-team.gradle.com/s/lc73bw56kdkxi

Maven

Experiment 1

Online

./01-validate-local-build-caching-same-location.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/maven/3.8.x/ge \
  -g 'package'

Offline

./01-validate-local-build-caching-same-location.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/maven/3.8.x/ge \
  -g 'package' \
  -x

No visual differences.

Experiment 2

Online

./02-validate-local-build-caching-different-locations.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/maven/3.8.x/ge \
  -g 'package'

Offline

./02-validate-local-build-caching-different-locations.sh \
  -r git@github.com:gradle/ge-solutions.git \
  -p sample-projects/maven/3.8.x/ge \
  -g 'package' \
  -x

No visual differences.

Experiment 3

./03-validate-remote-build-caching-ci-ci.sh \
  -1 https://ge.solutions-team.gradle.com/s/nhrdhjwc6nysy \
  -2 https://ge.solutions-team.gradle.com/s/v7folxiq3dkyk

Experiment 4

./04-validate-remote-build-caching-ci-local.sh \
  -1 https://ge.solutions-team.gradle.com/s/ehiqsjd2wh5qs

[runningcode]

Took a very quick look and the direction looks good.