summaryrefslogtreecommitdiff
path: root/ci
AgeCommit message (Collapse)Author
2025-03-23For test mapping config append modified path Julien Desprez
Test: presubmit Bug: 405154519 Change-Id: I4a2bc3806722be9322f0d40070d00c3fafc9faa3
2025-03-19Rework general-tests optimization Luca Farsi
Rework general-tests optimization to use the test discovery agent to determine the modules that are needed to be included in general-tests.zip. Also start reporting optimization decisions silently. Test: atest build_test_suites_test; atest optimized_targets_test Bug: 358215235 Change-Id: Iabff6729e5743805167eed87d7ef5d901b255a61
2025-03-18Merge "Keep track of which test infos use which target" into main Luca Farsi
2025-03-18Merge "[BWYN] Implement test mapping test module discovery interface in ↵ Julien Desprez
build script." into main
2025-03-13[BWYN] Implement test mapping test module discovery interface in build script. Mingjun Yang
BUG: 392457055 Test: Build, Presubmit Change-Id: Ic1195a5e4f8b32f0de4c7fe717689fe7c42bc499
2025-03-12Keep track of which test infos use which target Luca Farsi
Modify build script to keep track of which test info uses which target, which will be necessary for intra-zip test discovery later Test: test Bug: 358215235 Change-Id: I3d51d6527396fc1538d51602ad84084dbcdc7b0e
2025-02-28Cleanup python version properties Cole Faust
Now that python 2 is fully removed, there's no need for version-specific properties. Remove them. Also remove usages of embedded_launcher: true, as that is the new default value. Bug: 399954452 Test: Presubmits Change-Id: If7a2b6c9b0ea5fc40c72c87e2f5f631b3b1bb814
2025-01-14Merge "Log the build command that we are executing" into main Julien Desprez
2025-01-10Log the build command that we are executing Julien Desprez
Change-Id: Ic31b75103ff6eba9566367ac22d570f66ce7d1ee Test: presubmit Bug: None
2025-01-09Add option for test discovery info mode Luca Farsi
Add a command line option to enable test discovery info mode, where enabling it will cause test discovery to run and report results but not actually affect the build. This is useful for rolling out to new targets without immediately enabling test discovery mode. Test: atest build_test_suites_test Bug: 388833029 Change-Id: I8baf73e6353e4c3d7d6836c4f9823341a7f8b5dd
2024-12-20Report optimized targets when nothing has to be built Julien Desprez
Change-Id: I3485d70f1ebfdfcd1b8d74427f75e245eb6e7514 Test: presubmit Bug: 385339697
2024-12-19Refactor the code to avoid duplicate logic for silent and non-silent Julien Desprez
Instead fallback to returning the full list after discovery. This allows re-using the same logic even when running silent experiment Change-Id: I9e589cb67f5968aec0f719c2b0108d86b16e9023 Test: presubmit Bug: 383118767
2024-12-17patch up a bit of reporting metric logic Julien Desprez
Ensure we stop updating the optimized status for the metrics once we come of a decision. Change-Id: I14d74ea71cc79f22170d2cf10ca299f5dd2f3c1d Test: presubmit Bug: 383118767
2024-12-16Include checkbuild in list of required target Julien Desprez
We should never remove checkbuild right now when it is specified. Change-Id: I71fe5417f4f863e3b499a0f25534a374050bd064 Test: presubmit Bug: 378704055
2024-12-16Report required build target as such in metrics Julien Desprez
Those are the targets that should never be removed Change-Id: Ifcba91d044bafc9d8be717e0d8871b6b96e86439 Test: presubmit Bug: 383118767
2024-12-11Merge "Add robustness to possible empty BUILD_CONTEXT" into main Julien Desprez
2024-12-11Add robustness to possible empty BUILD_CONTEXT Julien Desprez
We find possible evidence that BUILD_CONTEXT existed but was incorrectly empty. Adding robustness for this for safety while continuing to investigate the root cause. Change-Id: I8e9149a97e8cba8291cb41e010d75102a437ad87 Test: presubmit Bug: 383383586
2024-12-10Merge "Include droid in required targets" into main Treehugger Robot
2024-12-10Include droid in required targets Julien Desprez
We never want to filter out 'droid' when it exists Test: presubmit Bug: 378704055 Change-Id: I9f8849cc825c19de99321f5687ee655ea2a0503a
2024-12-10Merge "Also report the unoptimized target in silent mode" into main Julien Desprez
2024-12-10Merge changes from topic "tree_size" into main Treehugger Robot
* changes: Add source_tree_size artifact to the android_metadata builds. Add import_build_vars to shell_utils.sh
2024-12-09Also report the unoptimized target in silent mode Julien Desprez
Change-Id: I66fff4330991376c6bdf31b98a0aa2a066f319d8 Test: presubmit Bug: 378926009
2024-12-09Add source_tree_size artifact to the android_metadata builds. Joe Onorato
Bug: 383124666 Test: treehugger Change-Id: I32150b4903438934794510a718fcd1e0ffb086d2
2024-12-07Enable real test discovery in build_test_suites Luca Farsi
Enable test discovery to actually affect build targets in build_test_suites.py (only when the flags are enabled. On failure of test discovery or if the flags are off instead fall back to the previous optimizations. Change-Id: I6937dd0c5096884095d0770bc53b6399233cefc2 Test: atest build_test_suites_test Bug: 378926009
2024-12-06Add --device-build flag to build_test_suites.py Luca Farsi
The flag will be used to differentiate device builds so we can do the appropriate BWYN logic. Also include DIST_DIR as a required env var. Bug: 378704055 Change-Id: I203645ab82d52024a713c41c6fdc806f8672f7cc Test: atest build_test_suites_test
2024-12-04Fix crash when failing to parse test discovery output Luca Farsi
Test discovery code should never fail a build as it's for info purposes only. Test: atest build_test_suites_test Bug: 382217072 Change-Id: Ib546ee6209e1c3bb9d249cb1ee1d37454449008d
2024-11-19Merge "Add device target build script wrapper" into main Luca Farsi
2024-11-18Add device target build script wrapper Luca Farsi
Add a separate wrapper to be used in device build targets, this will be helpful if experiemnts are needed just on the device target side. Test: N/A Bug: 378704055 Change-Id: If523854bc4b89451904ac70d209a0a9e0fdb8930
2024-11-18Report optimized/unoptimized targets correctly Luca Farsi
Always report every target that gets built as unoptimized, and only report targets that wouldn't get built as optimized. Previously when test discovery succeeded only targets that get built were getting reported (as optimized when those were the unoptimized ones), and targets that wouldn't get built were not being reported at all (when those would be the optimized targets). Test: Presubmit Bug: 372973116 Change-Id: I0b22b57654bba90d9eb398651b09ae0f65df0d5c
2024-11-14Begin reporting Test Discovery Agent metrics Luca Farsi
Start running the Test Discovery Agent and reportings its results via metrics. No changes to the actual build process yet. Test: atest build_test_suites_test && atest optimized_targets_test Bug: 372973116 Change-Id: I958e034985c41ede8d6a2700311a4dd6e7ac18ba
2024-11-14Record initial metrics in build_test_suites Luca Farsi
Record initial metrics for analysis time and packaging time in build_test_suites. Per-target metrics will be reported after initial test discovery agent integration. Test: atest build_test_suites_test && atest optimized_targets_test Bug: 372973116 Change-Id: Id3711ce078bfbfdc4c264a29751c475b4bd0cebd
2024-11-14Build using build_test_suites_binary Luca Farsi
Change the wrapper script for build_test_suites to instead build the binary first and then call that instead Test: Presubmit Bug: 372973116 Change-Id: Id3d09557f5d81acbb05623d33c78a2a75895190c
2024-11-14Implement metrics agent for built_test_suites Luca Farsi
Implement the metrics agent for build_test_suites. This will report metrics for optimization decisions and build/analysis times for the build_test_suites script. Test: atest build_test_suites_test Bug: 372973116 Change-Id: I6b6b726b93de6ab3fbb49a95ebadd5338859fcaf
2024-11-07Add build rule for build_test_suites Luca Farsi
Add a build rule to build build_test_suites. Test: m build_test_suites Bug: 372973116 Change-Id: I790cc723a7e4deba2309b24cdcf6a058f8a77254
2024-11-05Implement test discovery agent for test zip discovery Mingjun Yang
BUG: 371603869 Change-Id: Ia9f2e620db7625bccc848324f494e43e4374bcfe Test: Verified in presubmit, partially built test zips by referring to test discovery result in presubmit build, and presubmit test passed.
2024-10-15Add test discovery agent skeleton Mingjun Yang
Change-Id: I3dacac6d3081a815d07e461561dec380ea63782b b:371603869 Test:build
2024-10-03Add release configs artifacts to metadata build LaMont Jones
Bug: b/370544058 Test: TH, manual Change-Id: I9e7ccef1e22510932625336988903382669de0ef
2024-09-19Fix packaging outputs commands Luca Farsi
There were a few issues with the output packaging process that were found during testing of the general-tests optimization. First and foremost is that the packaging commands were trying to be created before the build ran, when the outputs don't exist. I've changed the logic to just collect the methods themselves which will then be run during build plan execution after the build has completed. A few other smaller issues include fixing the path to the soong_zip binary, incorrect execution of the soong dumpvars command, and not building the shared libs zip. Test: atest build_test_suites_test; atest optimized_targets_test Bug: 358215235 Change-Id: I8a3f54738f8bb5d871aadf7423844076c38b54a6
2024-09-13Update metadata target to the correct one Ronish Kalia
Bug: 326986729 (cherry picked from https://googleplex-android-review.googlesource.com/q/commit:3fb924d1eddfe031d799f34a2f32d39b9b671a44) Merged-In: I3da3d31abd3280f85aad30f81151502ec32fc368 Change-Id: I3da3d31abd3280f85aad30f81151502ec32fc368
2024-09-12Implement package_outputs in GeneralTestsOptimizer Luca Farsi
Implement the output packaging step in the GeneralTestsOptimizer. This step takes all the outputs built by the general-tests target (in the case where it was optimized) and packages them into the necessary zips generated by the target normally. Test: atest build_test_suites_test; atest optimized_targets_test Bug: 358215235 Change-Id: I5d27eef4e37137cc9b6e235f52f3856ba0b30460
2024-09-10Refactor package_outputs Luca Farsi
Refactor package_outputs in the TestOptimizer so it just returns a list of soong_zip commands to be run by build_test_suites. Since we already have a tested implementation for running subprocesses in build_test_suites.py there's no reason to reimplement it in optimized_targets.py. Because any packaging will ultimately use soong_zip to package its final outputs change the code to just do whatever prep it needs to and return a list of soong_zip commands. This way the code is simpler to test without requiring subprocesses and no reimplementation of subprocess running code is necessary. Test: atest build_test_suites_test; atest optimized_targets_test Bug: 358215235 Change-Id: I3025aefeeb7186f537266a72d8422211ca9835ba
2024-09-06Merge "Add script that dumps all product config varaibles." into main Treehugger Robot
2024-09-05Merge "Add new script for metadata build target" into main Joe Onorato
2024-08-30Add script that dumps all product config varaibles. Joe Onorato
Test: build/make/ci/dump_product_config Change-Id: If6b48fa69b6836ab2c06d6ebe22487001bcd6e77
2024-08-27Implement get_build_targets_impl in GeneralTestsOptimizer Luca Farsi
Implement functionality in GeneralTestsOptimizer to find what targets to build. This logic is fairly complex and involves checking for test configs that download general-tests.zip. Then the configs are checked to see if they're proper test mapping tests (if they use the 'test-mapping-test-group' option). If they are, then TEST_MAPPING modules are scanned to see if the list of changed files would cause any test mapping modules to run. The tests are then further filtered by test-mapping-test-groups used in the test configs. In case that a test uses general-tests.zip but does not specify 'test-mapping-test-group' then all bets are off and general-tests.zip is built in its entirety. package_outputs is still unimplemented so this will need to be implemented before the optimization can be enabled. Test: atest build_test_suites_test && atest optimized_targets_test Bug: 358215235 Change-Id: I6a7eebfd1b06b380799292eb2019ac17c9af5367
2024-08-26add BuildContext class and fix enabled features Luca Farsi
Add a BuildContext class to simplify parsing the build context dict, and fix the parsing of enabledBuildFeatures, which was being parsed as a list of strings when it's actually a list of dicts like: [{'name': '<feature_name>'}] Test: atest build_test_suites_test Bug: 361605425 Change-Id: I6424c444daf1582e92313c39f43207cb274aa78f
2024-08-12Change artifact matching to be more strict Luca Farsi
The current artifact matching logic will cause any test target that uses something like .*-tests.zip to be way too broad in what it causes to build. Change the logic to look for the target name in the artifact matching regex to make it so that only tests that look for a specific target to match. Test: atest build_test_suites_test Bug: 348489774 Change-Id: Ia75e38b676607f45f2b1c8fcf948045c248f1729
2024-08-08Refactor OptimizedBuildTarget and add general-tests optimization Luca Farsi
Refactor the OptimizedBuildTarget base class to more simply handle enabling/disabling build optimizations based on flags. Now all optimizers (aside from the no-op NullOptimizer will have their enabling built into the base class, and also necessitate their get_build_targets and package_outputs functions to be defined in order to not error out. Add the GeneralTestsOptimizer class as well, unimplemented for now. Test: atest build_test_suites_test Bug: 358215235 Change-Id: Ide22c64fc238d754db8d8e76484947401e54e680
2024-08-02Merge "Don't build targets if they're not used." into main Luca Farsi
2024-08-02Don't build targets if they're not used. Luca Farsi
Add functionality in build_test_suites.py to not build targets if their outputs are not used in the test configurations saved in the build context. If none of the tests reference the targets' outputs they will not be built at all. Note that the corresponding flags will need to be enabled for these optimizations to take place. Test: atest optimized_targets_test Bug: 348489774 Change-Id: I8f0ac90e75552ae80073f13229b026c7f23476a6
generated by cgit v1.2.3-59-g8ed1b (git 2.39.0) at 2025-10-19 17:34:21 +0000