summaryrefslogtreecommitdiff
path: root/android/module.go
AgeCommit message (Collapse)Author
2023-12-08Remove infrastructure to run bp2build Colin Cross
Bug: 315353489 Test: m blueprint_tests Change-Id: Idcf6377d389b94c39e4e6ff4b8efa8a9f9e78b17
2023-11-30Move test data installation to Soong Colin Cross
To generate module-info.json in Soong for b/309006256 Soong needs to know the test data paths. Moving test data installation into Soong will also help later for test suite packaging. Add ModuleContext.InstallTestData that installs the files listed in a []DataPath alongside the test. The files will also be passed to Make to allow it to continue packaging them into the test suites for now. Update the module types that are producing LOCAL_TEST_DATA entries in their Android.mk files to go through InstallTestData instead. Bug: 311428265 Test: atest --host toybox-gtests --test-timeout=120000 Change-Id: Ia8b964f86e584ea464667fd86a48d754d118bead
2023-11-15Split *ModuleContext out of module.go Colin Cross
Move EarlyModuleContext, BaseModuleContext and ModuleContext out of module.go and into early_module_context, base_module_context and module_context.go respectively. Test: builds Change-Id: I52e6eb1589d1478233c1c55d770b395a16eaa1a3
2023-11-08Use `all_apex_contributions` for source/prebuilts selection Spandan Das
This flattened singleton module explicitly lists by module name whether source or prebuilt version of a module should be used. If a module appears in this metadata module, it supersedes all other source vs prebuilts selection mechanism Implementation details - Update the module dep chain from <source> --> <prebuilt> to <source> --> <prebuilt> --> all_apex_contributions - all_apex_contributions sets a provider which is bubbled up to the source module. This requires changing `prebuilt_select` to a bottom up mutator - Update `usePrebuilt` to consult the new provider before falling back to existing source vs selection mechanisms. If (foo|prebuilt_foo) is listed in the selected `apex_contributions` modules, it will be used superseding any other selection mechanisms. - Update this depTag to IsMetaDepTag so that bootclasspath_fragment's validation ignores this new dependency. Test: Added unit tests to assert that this new mechanism supersedes `use_source_config_var` Bug: 308174768 Change-Id: I39a85639642711f3c96b6f18b94d626b55e80c66
2023-10-18Add bazel-built modules as deps on the system image Cole Faust
These bazel-built modules will be installed into the system image as part of the bazel rule, rather than going through the make staging directory. Bug: 297269187 Test: m bazel_sandwich Change-Id: I96c6e58f8e0898b2ad92cb7069745ca2059a39f8
2023-10-16Remove baseModuleContext.debug Cole Faust
It was never set to true. Test: Presubmits Change-Id: I9944d90b7e75beb4a7fe259c72bc7a82e42f593d
2023-10-05Dont write data attrs for cc lib rules Chris Parsons
The data attributes for cc library rules is dropped by their starlark macros, so serves no real purpose. Furthermore, this unused dependency proves problematic for allowlist v2, as there are many cases at HEAD where the corner-case "requires" dependency would otherwise mark a module as unconvertible. Fixes: 303307456 Test: New unit test Test: Manual verification to unblock allowlist v2 `bp2build.sh` runs in AOSP Change-Id: I6ca6104b958d2b428fc2ca5b3fa794106571acca
2023-09-22Switch bp2build mutator to bottom up Chris Parsons
This should be no-op, as the underlying mutator has not changed yet. Some other refactoring is required and done in this CL: - Delete some old, dead ApiBp2build code - Fix casting to TopDownMutator when it's not necessary This change is required to prepare for allowlist v2 work, as only BottomUp mutators can AddDependency. Bug: 285631638 Test: m nothing Test: presubmits Change-Id: I5212a5f5634cc13056195783e6df37ff8eb000da
2023-09-21Handle already existing targets of different name Chris Parsons
In other words, if, in bp2build, module "foo" would generate "foo", and "foo_two", and "foo_two" already exists in a build file, bp2build should label "foo" as being unconvertible. Fixes: 301321658 Fixes: 301312582 Bug: 285631638 Test: Unit tests Test: Verified that `m bp2build` results in bit-for-bit identical contents for out/soong/bp2build before and after this change. Change-Id: Icbbdd69fce83579ec9b172d04b2bf1f294698f70
2023-09-19Handle prebuilt vs source selection in bp2build Liz Kammer
Test: enable mainline modules build from prebuilts and build Bug: 300640274 Change-Id: Ib1d6bbca7e0ab459515d3cf6378741e8368e7327
2023-09-19Don't panic for unhandled product vars Liz Kammer
Instead, we return an error. This allows us to access some product variable information earlier when it will not be used as an attribute without panicing Test: m nothing Change-Id: Id094b2b9e1364a8d174d99b3824fa149fb235b3e
2023-09-18[bp2build] android_test bp2build. Jingwen Chen
android_test is just an android_app, but the test bits are handled purely in Starlark (tradefed_test_impl). This enables building HelloWorldTests (android_test) with Bazel. Bug: 297030120 Bug: 297029612 Test: WIP Change-Id: I6d287c94b31698ea247da48e4cd6b5a75bc71e11
2023-09-06Add target_compatible_with for compile_multilib Spandan Das
For explicit values of compile_multilib (e.g. 32 or 64), add an incompatibilty constraint for the arches that do not have the correct bitness. Test: go test ./bp2build Bug: 299135307 Change-Id: I4cd997de6804a5c6d89086d740f773936db1cab9
2023-09-01Fix target_compatible_with non-determinism for proto.include_dirs Spandan Das
A single proto providing directory can be used by multiple soong modules. Some of these can be 1. Host specific 2. Device specific 3. Both host and device Since the generated proto_library can have 1:many mapping, it should have an empty target_compatible_with. Compatiblity will be enforced at the top-level {cc|java|python}_proto_library. (This is a followup to aosp/2727054 which did not handle this correctly) Test: Added a unit test Change-Id: I09b3def70e3d043fd8ba0d1eb4ffff1910f097d1
2023-08-08Handle enabled: false via conditions_default Spandan Das
In this Android.bp file ``` my_cc_defaults { enabled: false, soong_config_variables: { my_bool_variable: { conditions_default: {enabled: false}, } } } ``` The inner enabled: false is a no-op because the top-level enabled is false. Currently, bp2build will raise an exception for this Android.bp file. However, it does not need to. `productVariableConfigEnableLabels` runs only if the top-level enabled is false. If it sees enabled: false via conditions_default, it should just ignore it since it is a no-op. Test: go test ./bp2build Bug: 210546943 Change-Id: I816f209eaf21de65ddfbc2893e5255be94bcaa11
2023-08-08Handle nil enabled values Spandan Das
If enabled does not appear inside `soong_config_vars`, we can ignore it. Bug: 210546943 Test: go test ./bp2build Change-Id: I9e4d51c3b683f262921449634f827915ce87dc8d
2023-08-08Merge changes I8ca452aa,I437fc890,If9446700 into main Spandan Das
* changes: Handle .proto files in different package for filegroups Handle .proto files that end up in a different package Allow creation of BazelTargets in a different directory
2023-08-07Allow creation of BazelTargets in a different directory Spandan Das
The current API restricts creation of targets to the directory of the visited soong module. This CL proposes adding a `Dir` property in `CommonAttributes` that can be used to create a bazel target in a specific dir. The use case for this is to dynamically create additional targets for proto_library that are adjacent to .proto files (Bazel poses a strict requirement about proto_library being in the same package as the .proto file, but Soong does not) Usage is restricted to dirs that have an existing Android.bp file. There are some places in bp2build where we use existence of Android.bp/BUILD on filesystem to curate a compatible fully qualified path (e.g. headers). If we use `CommonAttributes.Dir` to arbritraily create BUILD files, then it might render those curated labels incompatible. Test: go test ./bp2build Change-Id: If9446700457eddfb389be9d9bde39087f67daa60
2023-08-03Make bp2build-generated selects() based on product config build settings Cole Faust
...instead of based on constraint settings. Bug: 269577299 Test: m nothing and ./build/bazel/ci/bp2build.sh Change-Id: Ib9caec79c92b8fd304e46be841de5612bd1637e3
2023-07-24Handle xsd config more like other partitions Liz Kammer
This is prep work to add additional special handling for genrule generated headers as there will be similar partitioning for those headers. Test: go test soong tests Change-Id: Ib63e7e4f7554b2b7b7bc78b2825b20c05403216a
2023-06-22Merge "Use generics for DepSets" Colin Cross
2023-06-22Use generics for DepSets Colin Cross
Use Go's generics for DepSets so they don't require a type-specific wrapper and reflection. Test: depsets_test.go Change-Id: I22ba0b7d680d37d2cd05230b0f560d166c4dd20b
2023-06-16Merge "Provide reason for unconverted bp2build modules" Christopher Parsons
2023-06-16Provide reason for unconverted bp2build modules Chris Parsons
This also changes the expectation of ConvertWithBp2build. Each implementation must either create one or more Bazel target modules, or mark the module as unconvertible (with a specific reason). Manually verified no runtime hit in AOSP In AOSP, the metrics file size increases from 252K to 1.6M This changes some effective module counts in bp2build metrics: - Removes "package" modules from the module count list in metrics, as these will not be converted like regular modules. - Counts Handcrafted modules as being "unconverted", as bp2build is not responsible for them. Bug: 285631638 Test: Verified generated BUILD.bazel files are bit-for-bit identical with this change Test: Manually verified one case of each implemented reasonType Change-Id: I308dd451d8f28379b15671dae9f931bd0446f5c1
2023-06-16Fix soong config hash missing for some modules Inseob Kim
Bug: 279362051 Test: see outputs of C++ modules using soong config Change-Id: I95e41350ff17a989faf7c010a4c4ed4fcd79d66b
2023-06-01bp2build for java libraries using xsd_config srcs Spandan Das
In Soong, java libraries can use src files generated from xsd_config. In Bazel, instead of providing srcs, java_xsd_config_library will provide a jar. Update bp2build so that all src references to xsd_config modules in srcs get routed to deps and exports attributes This CL creates an interface in build/soong/android to get around a circular dependency issue. The bp2build logic needs to exist in soong-java, but soong-java does have soong-xsdc in its package path. Use the interface and type assertions to special case xsd_config references Bug: 211678537 Test: bp2build test in sibling CL in system/tools/xsdc Change-Id: Ida924bb20b1fd7eb8beeef950b070d37a9c6f3b5
2023-05-26Use hashed subdir for soong_config modules Inseob Kim
This is to differentiate soong intermediate directories for soong config modules. This will help incremental build across different devices. Test result of building panther, building cheetah, and building panther again: Before this change - build time: 02:57 - # of tasks: 31044 After this change - build time: 01:48 - # of tasks: 1694 Build time includes build.ninja generating time (which is more than 1 minute), so the overriden artifacts become far fewer. And "NINJA_ARGS='-n -d explain' m" only gave 4 "command line changed" nodes. Bug: 279362051 Test: see above Change-Id: I4891cbe823ae21628465e5c6eb26a4837ccdd202
2023-05-23Merge "Handle missing dependencies in mixed builds" Treehugger Robot
2023-05-23Handle missing dependencies in mixed builds Liz Kammer
Bazel will fail on queries with missing dependencies. Instead, we check for missing dependencies in mixed builds and we fall back to Soong when we are aware of missing dependencies in a module. Test: go test soong tests Change-Id: I4f83752704970d8b43650d5b55ff35799c7bc625
2023-05-19Parallelize singleton execution LaMont Jones
Bug: 281536768 Test: manual, presubmits Change-Id: I57fdc76ba6b277e88e196b506af87127a530fd37
2023-05-02Install sdk variants in unbundled builds and package uninstallable variants Colin Cross
This effectively undoes both If6c3ee82d588e2742c85cef7244c090c93f38b8e and I682e4f1f477f3024f7719dfaa67006ef335e0640. SDK variants are now installed again, which will fix unbundled builds of cc_test modules. The platform variants used by com.android.virt are now packagable even though they are not installable. Fix the original problem in b/194403710 by adding a flag to platform variants of modules in apexes that are not platform available, and using that to prevent install and packaging dependencies. That allows the HideFromMake flag to go back to being used for preventing install dependencies but not packaging dependencies. Test: TestPackagingWithSkipInstallDeps Test: TestFileSystemShouldInstallCoreVariantIfTargetBuildAppsIsSet Test: TestFileSystemShouldSkipApexLibraries Bug: 194403710 Bug: 268582372 Fixes: 274443025 Change-Id: If5418df3ddbb940bd631caebdf38daa81e71f40e
2023-04-25Refactor MixedBuildsEnabled and add --ensure-allowlist-integrity. MarkDacek
Currently, there is little verification around allowlisted modules actually being mixed-built. This flag would allow us to verify that a module allowlisted is mixed-built for at least one variant. Bug: 278910100 Test: m nothing --bazel-mode-staging --ensure-allowlist-integrity Test: m nothing --bazel-mode-staging --ensure-allowlist-integrity --bazel-force-enabled-modules=com.google.android.neuralnetworks (This fails, as expected) Test: build/soong/test/mixed_mode_test.sh Change-Id: Icd5976f4f44f1a8caca1e5247d986642f7995f97
2023-04-07Include license texts and kinds for VNDK snapshot Justin Yun
VNDK snapshot must include the information of the license texts and kinds. Include them when packaging the snapshot prebuilt files. This will be used to generate the VNDK snapshot files. Bug: 270625053 Bug: 277317599 Test: development/vndk/snapshot/build.sh --build-artifacts Change-Id: I0a39ff598efbb80faaf63807cdf5ce685fa074cc
2023-03-20Add documentation for BuildParams Liz Kammer
Test: n/a Change-Id: I867b3c2b7faad66a1b8c3b772214673b2c56a3b8
2023-03-10Revert "Export non-apex variants of modules to make" Iván Budnik
Revert submission 2462194-androidmk_for_apex_dependencies Reason for revert: DroidMonitor-triggered revert due to breakage https://android-build.googleplex.com/builds/quarterdeck?branch=aosp-master&target=full-eng&lkgb=9723453&lkbb=9724944&fkbb=9723508, bug b/272663333. Reverted changes: /q/submissionid:2462194-androidmk_for_apex_dependencies Bug: 272663333 Change-Id: I847fa910a8d54de19954f56fe92a80a9bcd0ee18
2023-03-09Export non-apex variants of modules to make Cole Faust
Currently, non-apex variants of modules that are in apexes are not exported to make unless they're apex_available to the platform. This means that you can't `m` those modules directly. However, there is a workaround in the apex androidmk implementation that emits make rules for the removed modules, but just redirects them to build the apex itself. We want to remove that, but one of the problems with doing so is that you can no longer `m` many modules afterwards. To fix that, unhide the apex's dependencies from make. To ensure they're not installed, call SkipInstall() on them, and update SkipInstall() to be more strict by setting `LOCAL_UNINSTALLABLE_MODULE := true`. Bug: 254205429 Test: Presubmits Change-Id: Ib094feb2c437ad50d8319c58caa997759e7ce32f
2023-03-08Use product variables from the overridden apex Cole Faust
override_apex's bp2build converter had a bug where it was looking at the product variables for the override_apex module itself instead of for the base module it is overriding. Fixes: 271424349 Test: go test Change-Id: If1e2653d3751fa908faf0ab97dfa2e943ebe98ec
2023-02-28Replace SortedStringKeys with SortedKeys Cole Faust
Now that we have generics. Bug: 193460475 Test: presubmits Change-Id: I1594fd8feb505175d5c09c03ef397e5ffd5b09cb
2023-02-17Fix symlinks from APEX to partitions Jiyong Park
Previously, the symlink optimization for APEXes assumed that the target of the symlinks are in the system partition. The assumption however doesn't hold always because the file that was added to the APEX might be with system_ext_specific: true or vendor: true. Bug: 265598720 Test: m nothing Change-Id: Ieb9a6769320c0ec697a88c0cae977e7d65288362
2022-12-19Revert "Allow adding extra tradefed options in the Android.bp file" Tahsin Loqman
This reverts commit 8ec823cba166a41eb0e9e5ff8fe679e691fec678. Reason for revert: DroidMonitor: Potential culprit for Bug b/262965953 - verifying through ABTD before revert submission. This is part of the standard investigation process, and does not mean your CL will be reverted. Change-Id: I236cc36981d8b30527ca286632727f8ca267e969
2022-12-09Allow adding extra tradefed options in the Android.bp file Cole Faust
Some tests need to add custom tradefed options, but still want to keep most of the soong autogenerated tradefed xml file. Expose a test_options: { tradefed_options: [...] } property that will allow tests to add more options to the autogenerated xml file. Fixes: 184895128 Test: go test, and verified that the ninja files did not change for aosp_arm64 Change-Id: I75f7eb002c8325ce7cdc76e12e76e16195320620
2022-11-29Add more cases to vendor-installed module Kiyoung Kim
Current InstallInVendor only checks vendor property, but there are more properties to check if the module will be installed in the vendor. This change adds two more cases (soc_specific and proprietary) to the function. Bug: 233294155 Test: N/A Change-Id: Ic3cb4c52f5097cdcc6231c0e85a62ce3ad6094b3
2022-10-24Support apex_test. Yu Liu
Bug: 249744489 Test: Manual and unit test. Change-Id: I866debf7ffee54a0f4649b5e5425a8e3cc3042a1
2022-10-12Fix some issues in bp2build converter for python_binary_host. Wei Li
1) Bp2build convert python_binary_host main attribute as LabelAttribute. Currently "main" attribute in python_binary_host is handled as string but for some modules (e.g certify_bootimg) the "main" attribute points to a file in its subpackage like "subpackage/file.py" and should be converted to "//.../subpackage:file.py". 2) Filter out duplicated labels in the merged list of "required" attributes of python_binary_host and its defaults. Test: b build //system/tools/mkbootimg:certify_bootimg Test: b build //build/make/tools/releasetools:check_target_files_signatures Bug: 253081249 Bug: 253101186 Change-Id: Ic2cb4cadec2c1348da70af9f0730da9914d3a8ca
2022-10-07Converters for contributions to systemapi and vendorapi Spandan Das
The module types in scope of this conversion are 1. cc_library and cc_library_shared (non-null llndk or stubs prop) 2. cc_library_headers (all) For (2), we need some postprocessing on the results of the parser bp2BuildParseBaseProps. This is necessary because arch and os specific API exports need to be flattened and added to the generateed API headers target along NoConfigAxis e.g. ``` The api equivalent of cc_library_headers ( name = "lifoo", deps = select({ "//build/bazel/platforms/arch:arm": ["arm_deps"], "//build/bazel/platforms/arch:arm64": ["arm64_deps"], }), ) should be cc_api_library_headers ( name = "lifoo", deps = ["arm_deps", "arm64_deps"], ) ``` For (1), we also need to generate 1:many header api targets so that arch-specific deps can propagate arch metadata to the top-level api_domain rule Test: go test ./bp2build Test: go test ./cc Change-Id: Ie40cba1ac8e89f290b3d926c190d5e93abd52859
2022-10-04Make OutputFileForModule work for AllowMissingDependencies Colin Cross
Report missing output files as missing dependencies when AllowMissingDependencies is enabled. This will fix AllowMissingDependencies builds where a dependency module is present but missing support for a specific architecture. Bug: 250918230 Test: lunch aosp_riscv64-userdebug && m ALLOW_MISSING_DEPENDENCIES=true nothing Test: TestOutputFileForModule Change-Id: I95e96e3e7cb3df7bf678ed628b45baf659addbad
2022-09-30Merge "typo in comment" Usta (Tsering) Shrestha
2022-09-27typo in comment Usta Shrestha
Bug: N/A Test: N/A Change-Id: Ibfcaf4deba2181d48a6428d05f715fac447d74c6
2022-09-27Convert Soong `licenses` attribute to Bazel's `applicable_licenses` attribute Sasha Smundak
Add it to the `CommonAttributes` struct and handle it in fillCommonBp2BuildModuleAttrs Bug: 190817312 Test: treehugger Change-Id: I7b21056e680384d4046372ee844512029721026e
2022-09-23Remove apexer.required workaround for aapt2. Jingwen Chen
Add a filter for cyclic references in required module names in bp2build. Test: CI Bug: 247985196 Bug: 225291837 Fixes: 234748998 Change-Id: Id1cfd0b2e5f217cdb01218ec3bbe11d5070cc9df