summaryrefslogtreecommitdiff
path: root/python
AgeCommit message (Collapse)Author
2023-11-15Add proto for Test ownership metadata. Aditya Choudhary
This Cl adds a new rule to Soong to generate test spec metadata. Also, this CL adds a provider in various test module to provide test spec related data to the Soong rule. Will add providers and test code to other Module in the future changes. Provider added for the following test modules in this change: android_robolectric_test, android_test, bootclasspath_fragment_test, java_test, java_test_host, python_test, python_test_host, sh_test,and sh_test_host. Bug: 296873595 Change-Id: I5f89f72d5874bb7838ae357efdb8c6ca208e18a7
2023-09-20Have ConvertWBp2build use Bp2buildMutatorContext Chris Parsons
This no-op refactoring facilitates some upcoming functional changes for "bp2build allowlist v2". The work requires that the bp2build conversion mutator be changed from a TopDown mutator to a BottomUp mutator. Refactoring all bp2build-related methods so that they use Bp2buildMutatorContext makes it easier to make this functional change without touching tens of files and multiple projects. Bug: 285631638 Test: m bp2build Change-Id: I3d1ef3064146e959c6f0dc315350fc9764bf2bd2
2023-08-31Merge "Revert "Temporarily disable python precompilation for py3.11"" into main Dan Willemsen
2023-08-28Merge changes from topic "java-proto-include-dir" into main Treehugger Robot
* changes: Handle proto.include_dirs for java Translate python_libray.pkg_path to proto.import_prefix
2023-08-24Revert "Temporarily disable python precompilation for py3.11" Dan Willemsen
This reverts commit b1094d602bbd1fc6d4d60ace63ff82a92b479b23. Reason for revert: prebuilts updated in aosp/2726579 Bug: 278602456 Change-Id: I750b0f7e4dd94329a4455499f869a1823e660d77
2023-08-24Merge changes from topic "proto-include-dir-py" into main Spandan Das
* changes: Handle proto.include_dirs in bp2build for py Add proto.local_include_dirs support in bp2build Add manual tag to proto_library at the root package
2023-08-24Handle proto.include_dirs in bp2build for py Spandan Das
Followup to aosp/2711114 which added proto.include_dirs support for cc Test: go test ./bp2build Test: Added an integration test in build/bazel/examples Bug: 285140726 Change-Id: Ic78dacbe46d9ea67e9441948852a3f94dabbb35e
2023-08-24Translate python_libray.pkg_path to proto.import_prefix Spandan Das
If a python_library uses a pkg_path foo/bar, then the proto srcs in that libray need to import the dep .proto as foo/bar/proto.proto. This behavior is restricted to python modules. To implement this is in bp2build, this CL creates a new interface with a single method `PkgPath`. Only python module structs implement this interface, and this method is only available during bp2build Test: Added a bp2build unit test Test: TH Change-Id: If8d207c0b321f75337a053795826b283a5eaaf46
2023-08-24Merge "Temporarily disable python precompilation for py3.11" into main James Farrell
2023-08-22Temporarily disable python precompilation for py3.11 Dan Willemsen
The precompilation logic assumes that the source tree matches the python prebuilts, which is not necessarily true. This very much breaks during major version upgrades, so just disable precompilation temporarily while we generate new prebuilts. Bug: 278602456 Test: treehugger Change-Id: Idb206bb1d971e8c0fa556c75419623c3bc457898
2023-08-22Skip py2-cmd tests on Mac Dan Willemsen
Mac no longer builds python2 Test: prebuilt/build-tools/build-prebuilts.sh on Mac Change-Id: I5857ba7ec023021e660e86df3573e4b6e3d1902b
2023-08-17Merge "Prepare soong for python 3.11" into main Dan Willemsen
2023-08-16Prepare soong for python 3.11 Dan Willemsen
Due to upstream changes in python path calculations, I'm simplifying the python launcher to work more like a standard python distribution. This is effectively changing the stdlib path from `internal/stdlib` to `internal/python3.10` (or `3.11`, etc). This allows us to specify the zip file as PYTHONHOME, set PYTHONPLATLIBDIR to `internal` and use the default detection after that. That does mean during upgrades that the stdlib pkg path will change, so move the source vs prebuilt calculation from the Android.bp into Soong to choose which stdlib module to pick up (with the corresponding `pkg_path`) Bug: 278602456 Test: treehugger with python3.10 Test: a python3.11 source + 3.10 prebuilt build Test: a python3.11 source+prebuilt build Change-Id: I8b02e7b22a1f1d1e02819ae1a31a99cdc985542c
2023-08-15Use relative path instead of base for python proto Trevor Radcliffe
We encountered a duplicate sources bug in python proto generation because we're only using the base name. This CL makes it use the entire path to avoid this. Fixes: 295333879 Test: Presubmits Change-Id: I4ab821508cd5979bd595e327c1ccd3d68110fba6
2023-08-08Merge changes I8ca452aa,I437fc890,If9446700 into main Spandan Das
* changes: Handle .proto files in different package for filegroups Handle .proto files that end up in a different package Allow creation of BazelTargets in a different directory
2023-08-07Handle .proto files that end up in a different package Spandan Das
Bazel poses a strict requirement that .proto files and proto_library must be in the same package. This CL handles this automatically by creating the proto_library in a separate dir/package if necessary Implementation details - Partition the `srcs` by package. `srcs` has been computed using `transformSubpackagePath`, so the information about packages is available at this point - Create a proto_library in each package by using `CommonAttributes.Dir`. Collect all these additional libraries and put them in `info.Proto_libraries` so that they get added as deps of (cc|python|...)_proto_library - Add an import_prefix to the proto_library in subpackages relative to the current directory. This relies on the assumption that every src is beneath the current directory (Soong will complain if a path in Android.bp contains ../) filegroup module type uses a separate code-path to create proto_library. This will be handled in the next CL in stack. Test: bp2build unit tests Test: TH Test: Built the failing internal module mentioned in b/292583584#comment1 Bug: 292583584 Change-Id: I437fc89092321b26c5f0511387cde9e84084d6f9
2023-08-01Convert merge_annotation_zips_test to b test yike
Test: b test //frameworks/base/api:merge_annotation_zips_test Bug: 293225210 Change-Id: Ia53b4168894b5c70c843ca315d963fa905ef582f
2023-05-19Add data_device_bins_both properties for python_test_host Zi Wang
Test: presubmits and manual verification Bug: 274930471 Change-Id: Iafc85526afdb8264526e8a5a33319fa33a23c66b
2023-05-18Enable host cross python test builds Colin Cross
Ic37c8db918873ddf324c86b12b5412952b0f2be2 converted python_binary_host and python_library_host from HostSupportedNoCross to HostSupported, but left python_test_host. Do the same for python_test_host. Fixes dependencies between java_test_host modules that create host cross variants and python_test_host modules that were missing host cross variants. Bug: 282918027 Test: builds Change-Id: Iae687aa3aa0f0b005f2dd27469f631145247a008
2023-05-04Delete Python 2 embedded launcher test. Krzysztof Kosiński
There are no remaining Python 2 binaries using an embedded launcher in Android. Bug: 245854393 Test: m par_test py2-cmd py3-cmd && build/soong/python/tests/runtest.sh Change-Id: I241bbaa417060b51b4d2883011ccb43e22ace4c8
2023-04-13bp2build support for python_test(_host) Cole Faust
There was a request for using b with python tests. bp2build python tests exactly the same way as python binaries so that they can be used with `b`. Bug: None Test: go test Change-Id: Id68a6a73572745a4885b3e5bb1b8452e36baa982
2023-03-22stub_template_host redirect SIGINT and SIGTERM to subprocess Qiao Yang
Test: manual Change-Id: I561f581f7413dc18b2d449f4ef338db3b6b9ca47
2023-03-20Merge "Add python template for mobly tests." Ziwei Zhang
2023-03-09Make using python 2 an error Cole Faust
Except if BUILD_BROKEN_USES_SOONG_PYTHON2_MODULES is set, and except for some core py2 modules that can't be removed until python2 is fully gone. Bug: 203436762 Test: m nothing Change-Id: I62ccb6f5687eab1e79c372ffc234a90ca5b566ac
2023-03-08Add python template for mobly tests. Ziwei Zhang
Mobly is a Python-based test framework that specializes in supporting test cases that require multiple devices. There are some Mobly based Android tests, but these tests cannot be added to xTS since they are not using TF. Currently TF provides a 2-device Mobly test runner. This change is a short-term solution to help teams (eg. bluetooth) to add their multi-device Mobly tests to xTS without writing TF configs. For long-term, device infra team and xTS infra team will provide a solution for more complex multi-device tests in xTS and deprecate this TF template. Test: manually check the generated mobly xml config Bug: 272239365 Change-Id: I252cc774094eebbd9264243d30023ace4839c2cd
2023-02-28Replace SortedStringKeys with SortedKeys Cole Faust
Now that we have generics. Bug: 193460475 Test: presubmits Change-Id: I1594fd8feb505175d5c09c03ef397e5ffd5b09cb
2023-02-13Revert "Dynamically assemble the list of python launcher shared libs for ↵ Cole Faust
precompiling" This reverts commit aa5b377ee956f2e7ff2e0fb6630f636ecfe15b84. Reason for revert: this broke musl builds by removing the install dependency from python binaries to the launcher shared libraries: https://android-build.googleplex.com/builds/submitted/9600042/unit_tests_x86_64_musl/latest/view/logs/build_error.log Change-Id: I4e13f333f287831ba13724f13ab2be789a92b476
2023-02-02Dynamically assemble the list of python launcher shared libs for precompiling Cole Faust
Still fall back to the hardcoded list for what to put in LOCAL_SHARED_LIBRARIES, because I couldn't figure out a good way to get those modules at the moment. Bug: 259718110 Test: Presubmits Change-Id: Ia992ffea8819d8c67280b08bc45505f77af09de0
2023-01-27Precompile python sources Cole Faust
This signifigantly improves the startup time of soong-built python binaries. For example, running `m apexer && time out/host/linux-x86/bin/apexer` gives 0.734s before this cl, and 0.094s after. Fixes: 259718110 Test: Presubmits Change-Id: Ib19e83e2c60c39a849525be117279c318de3afa7
2023-01-25Refactor python rules Cole Faust
The goal of this cl is to simplify the python rules, mostly by removing the "decorator" pattern that they currently use, and instead making separate module types for libraries, binaries, and tests that inherit from each other. Bug: 259718110 Test: Verified ninja files are unchanged (they only change in the list of soong sources because I added/deleted files) Change-Id: I1e836e2cc4782c7818f91db7df7895de3b8db7ca
2023-01-03Allow adding extra tradefed options in the Android.bp file Cole Faust
Some tests need to add custom tradefed options, but still want to keep most of the soong autogenerated tradefed xml file. Expose a test_options: { tradefed_options: [...] } property that will allow tests to add more options to the autogenerated xml file. Fixes: 184895128 Test: go test, and verified that the ninja files did not change for aosp_arm64 Change-Id: I50d4ad139322e9e207202f1e1a50f5bbb424aa6f
2022-12-19Revert "Allow adding extra tradefed options in the Android.bp file" Tahsin Loqman
This reverts commit 8ec823cba166a41eb0e9e5ff8fe679e691fec678. Reason for revert: DroidMonitor: Potential culprit for Bug b/262965953 - verifying through ABTD before revert submission. This is part of the standard investigation process, and does not mean your CL will be reverted. Change-Id: I236cc36981d8b30527ca286632727f8ca267e969
2022-12-09Allow adding extra tradefed options in the Android.bp file Cole Faust
Some tests need to add custom tradefed options, but still want to keep most of the soong autogenerated tradefed xml file. Expose a test_options: { tradefed_options: [...] } property that will allow tests to add more options to the autogenerated xml file. Fixes: 184895128 Test: go test, and verified that the ninja files did not change for aosp_arm64 Change-Id: I75f7eb002c8325ce7cdc76e12e76e16195320620
2022-10-21Remove flags that enable the new python path behavior Cole Faust
The new behavior has been enabled by default, and these flags aren't necessary anymore. Fixes: 245583294 Test: m py_dont_import_folder_of_entrypoint_test && /ssd/aosp-master/out/host/linux-x86/testcases/py_dont_import_folder_of_entrypoint_test/x86_64/py_dont_import_folder_of_entrypoint_test Change-Id: I5b6f98da51791bc5d28662ef799a10c1bb6a35a0
2022-10-13Merge "Fix some issues in bp2build converter for python_binary_host." Wei Li
2022-10-12Fix some issues in bp2build converter for python_binary_host. Wei Li
1) Bp2build convert python_binary_host main attribute as LabelAttribute. Currently "main" attribute in python_binary_host is handled as string but for some modules (e.g certify_bootimg) the "main" attribute points to a file in its subpackage like "subpackage/file.py" and should be converted to "//.../subpackage:file.py". 2) Filter out duplicated labels in the merged list of "required" attributes of python_binary_host and its defaults. Test: b build //system/tools/mkbootimg:certify_bootimg Test: b build //build/make/tools/releasetools:check_target_files_signatures Bug: 253081249 Bug: 253101186 Change-Id: Ic2cb4cadec2c1348da70af9f0730da9914d3a8ca
2022-10-11Merge "Revert^2 "Enable new python path behavior by default"" Cole Faust
2022-10-11Revert^2 "Enable new python path behavior by default" Cole Faust
This reverts commit 4d5cc709fa16f5dc78f690369e946fcf66156312. Reason for revert: Acloud issue was fixed in aosp/2243150, make sure the host unit tests are run on this cl via go/abtd before submitting Change-Id: Ie148430d331a01a9877adaf3cb88f1c57af136a0 Bug: 245583294 Bug: 247578564 Bug: 251688241
2022-10-07Merge "Revert "Enable new python path behavior by default"" Lais Andrade
2022-10-07Revert "Enable new python path behavior by default" Lais Andrade
This reverts commit 5fe655d1c71ee079dd46e93c96b54aae39a10d16. Reason for revert: DroidMonitor: Potential culprit for Bug b/251688241 - verifying through ABTD before revert submission. This is part of the standard investigation process, and does not mean your CL will be reverted. Change-Id: I1ea4df27557fc891d8f0156ec5681f9b0c601c98
2022-10-07Merge "Enable new python path behavior by default" Treehugger Robot
2022-10-05Enable new python path behavior by default Cole Faust
Now that usages have been updated, enable the new behavior by default. If everything goes well with this, we can then remove the flags. Bug: 245583294 Bug: 247578564 Test: Presubmits Change-Id: Idac8d2dc0f96ae750dba4a99b32525676d0d6f25
2022-09-29[LSC] Add LOCAL_LICENSE_KINDS to build/soong Bob Badour
Added SPDX-license-identifier-Apache-2.0 to: python/tests/dont_import_folder_of_entrypoint/Android.bp python/tests/top_level_dirs/Android.bp Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Change-Id: I7a1ea160c685361318f9a8543e10b83eeb9dcb7f
2022-09-26Add binary properties to python_defaults and add modern_python_path_defaults Cole Faust
So that we can more easily enable these new flags on several modules. It seems you can still apply a python_defaults to a python_library even with this change, so I didn't break it out into a separate python_binary_defaults. Bug: 245583294 Test: m py_dont_import_folder_of_entrypoint_test && out/host/linux-x86/testcases/py_dont_import_folder_of_entrypoint_test/x86_64/py_dont_import_folder_of_entrypoint_test Change-Id: Iecf9c1845df4630bafdea9957eb2450c15f8596b
2022-09-26Add flag to not add directory of entrypoint to sys.path Cole Faust
The python interpreter will by default add the directory of the entrypoint script to the beginning of sys.path. This can be disabled in python 3.11+ (which is not released yet) using the PYTHON_SAFE_PATH environment variable or the -P flag. As a workaround to have this behavior in older python versions, we can make an __soong_entrypoint_redirector__.py file at the root of the zip file that is the entrypoint, and then that file will redirect to the real entrypoint. This brings non-embedded-launcher python modules closer to the embedded launcher version. The embedded launcher binaries already act like this because they start at an __main__.py file at the root of the zip file. Bug: 245583294 Test: m py_dont_import_folder_of_entrypoint_test && out/host/linux-x86/nativetest64/py_dont_import_folder_of_entrypoint_test/py_dont_import_folder_of_entrypoint_test Change-Id: I39aaf04fb19c3ba7f5c9d98220872d6d08abf736
2022-09-26Merge "Add flag to not add top-level modules to PYTHONPATH" Cole Faust
2022-09-23Add flag to not add top-level modules to PYTHONPATH Cole Faust
stub_template_host.txt added all the top-level modules to the PYTHONPATH, which isn't correct, and caused absl.logging to override the built-in logging module. Removing this also makes it more consistent with python binaries built with embedded_launcher: true. embedded_launcher: true binaries don't add the top-level modules. Fixes: 245583294 Test: m py_dont_add_top_level_dirs_test && out/host/linux-x86/testcases/py_dont_add_top_level_dirs_test/x86_64/py_dont_add_top_level_dirs_test Change-Id: Id3069565d2b2c4b2bda0ff5301e757a7b4201751
2022-09-22[LSC] Add LOCAL_LICENSE_KINDS to build/soong Bob Badour
Added SPDX-license-identifier-Apache-2.0 to: python/tests/proto_pkg_path/Android.bp Bug: 68860345 Bug: 151177513 Bug: 151953481 Test: m all Change-Id: I17e9945a265d0ede3e291ed6f5d410dde035fbe5
2022-09-20Make protobufs respect pkg_path properly Cole Faust
Currently, python protobuf sources are generated as if pkg_path didn't exist, but then are moved into the pkg_path directory after being generated. This means they're generated with import statements in them that don't include the pkg_path. These import statements won't work at all when pkg_path is at least 2 levels deep, but currently erroneously work with a 1 level deep pkg_path because we mistakenly add the top-level modules in a soong-built python zip to the PYTHONPATH. We want to remove those modules from the PYTHONPATH, so the generated protobuf source files have to use the correct imports. Since there are existing cases of code that needs to be updated, guard this new behavior behind a flag, protos_respect_pkg_path. We will set this to true on modules individually as we update them, and then eventually change the default to true and remove this flag. Bug: 247578564 Test: m py_proto_pkg_path_test && out/host/linux-x86/nativetest64/py_proto_pkg_path_test/py_proto_pkg_path_test Change-Id: I3695cf5521837da087592f2ad5350201035b7b0e
2022-09-09Don't close file descriptors when starting python program Cole Faust
Python programs run via a wrapper script. That wrapper script can be started with open file descriptors, for example by using the <(echo foo) syntax in bash. The shouldn't be closed when starting the real script. Bug: 203436762 Test: Presubmits Change-Id: Ib20450f713b15c8a5527d1e433bfd9635cead85f
generated by cgit v1.2.3-59-g8ed1b (git 2.39.0) at 2025-12-15 23:39:20 +0000