summaryrefslogtreecommitdiff
path: root/python/python.go
AgeCommit message (Collapse)Author
2023-12-14Convert SetProvider to generic providers API Colin Cross
Convert all of the callers of SetProvider to use the type-safe android.SetProvider API. Bug: 316410648 Test: builds Change-Id: If58f4b5355264ddab2045bc3591a4eac19cd58fc
2023-12-08Remove ConvertWithBp2build implementations Colin Cross
Remove the ConvertWithBp2build implementations from all the module types, along with the related code. Bug: 315353489 Test: m blueprint_tests Change-Id: I212672286686a318893bc7348ddd5a5ec51e77a7
2023-12-05Add source file provider for genrule/srcs, python libraries and rust libraries. Aditya Choudhary
Change-Id: I2d7d4684a10c15aeecc27b8db800ab27a807d2e2
2023-08-31Merge "Revert "Temporarily disable python precompilation for py3.11"" into main Dan Willemsen
2023-08-28Merge changes from topic "java-proto-include-dir" into main Treehugger Robot
* changes: Handle proto.include_dirs for java Translate python_libray.pkg_path to proto.import_prefix
2023-08-24Revert "Temporarily disable python precompilation for py3.11" Dan Willemsen
This reverts commit b1094d602bbd1fc6d4d60ace63ff82a92b479b23. Reason for revert: prebuilts updated in aosp/2726579 Bug: 278602456 Change-Id: I750b0f7e4dd94329a4455499f869a1823e660d77
2023-08-24Translate python_libray.pkg_path to proto.import_prefix Spandan Das
If a python_library uses a pkg_path foo/bar, then the proto srcs in that libray need to import the dep .proto as foo/bar/proto.proto. This behavior is restricted to python modules. To implement this is in bp2build, this CL creates a new interface with a single method `PkgPath`. Only python module structs implement this interface, and this method is only available during bp2build Test: Added a bp2build unit test Test: TH Change-Id: If8d207c0b321f75337a053795826b283a5eaaf46
2023-08-22Temporarily disable python precompilation for py3.11 Dan Willemsen
The precompilation logic assumes that the source tree matches the python prebuilts, which is not necessarily true. This very much breaks during major version upgrades, so just disable precompilation temporarily while we generate new prebuilts. Bug: 278602456 Test: treehugger Change-Id: Idb206bb1d971e8c0fa556c75419623c3bc457898
2023-08-16Prepare soong for python 3.11 Dan Willemsen
Due to upstream changes in python path calculations, I'm simplifying the python launcher to work more like a standard python distribution. This is effectively changing the stdlib path from `internal/stdlib` to `internal/python3.10` (or `3.11`, etc). This allows us to specify the zip file as PYTHONHOME, set PYTHONPLATLIBDIR to `internal` and use the default detection after that. That does mean during upgrades that the stdlib pkg path will change, so move the source vs prebuilt calculation from the Android.bp into Soong to choose which stdlib module to pick up (with the corresponding `pkg_path`) Bug: 278602456 Test: treehugger with python3.10 Test: a python3.11 source + 3.10 prebuilt build Test: a python3.11 source+prebuilt build Change-Id: I8b02e7b22a1f1d1e02819ae1a31a99cdc985542c
2023-05-04Delete Python 2 embedded launcher test. Krzysztof Kosiński
There are no remaining Python 2 binaries using an embedded launcher in Android. Bug: 245854393 Test: m par_test py2-cmd py3-cmd && build/soong/python/tests/runtest.sh Change-Id: I241bbaa417060b51b4d2883011ccb43e22ace4c8
2023-04-13bp2build support for python_test(_host) Cole Faust
There was a request for using b with python tests. bp2build python tests exactly the same way as python binaries so that they can be used with `b`. Bug: None Test: go test Change-Id: Id68a6a73572745a4885b3e5bb1b8452e36baa982
2023-03-09Make using python 2 an error Cole Faust
Except if BUILD_BROKEN_USES_SOONG_PYTHON2_MODULES is set, and except for some core py2 modules that can't be removed until python2 is fully gone. Bug: 203436762 Test: m nothing Change-Id: I62ccb6f5687eab1e79c372ffc234a90ca5b566ac
2023-02-28Replace SortedStringKeys with SortedKeys Cole Faust
Now that we have generics. Bug: 193460475 Test: presubmits Change-Id: I1594fd8feb505175d5c09c03ef397e5ffd5b09cb
2023-02-13Revert "Dynamically assemble the list of python launcher shared libs for ↵ Cole Faust
precompiling" This reverts commit aa5b377ee956f2e7ff2e0fb6630f636ecfe15b84. Reason for revert: this broke musl builds by removing the install dependency from python binaries to the launcher shared libraries: https://android-build.googleplex.com/builds/submitted/9600042/unit_tests_x86_64_musl/latest/view/logs/build_error.log Change-Id: I4e13f333f287831ba13724f13ab2be789a92b476
2023-02-02Dynamically assemble the list of python launcher shared libs for precompiling Cole Faust
Still fall back to the hardcoded list for what to put in LOCAL_SHARED_LIBRARIES, because I couldn't figure out a good way to get those modules at the moment. Bug: 259718110 Test: Presubmits Change-Id: Ia992ffea8819d8c67280b08bc45505f77af09de0
2023-01-27Precompile python sources Cole Faust
This signifigantly improves the startup time of soong-built python binaries. For example, running `m apexer && time out/host/linux-x86/bin/apexer` gives 0.734s before this cl, and 0.094s after. Fixes: 259718110 Test: Presubmits Change-Id: Ib19e83e2c60c39a849525be117279c318de3afa7
2023-01-25Refactor python rules Cole Faust
The goal of this cl is to simplify the python rules, mostly by removing the "decorator" pattern that they currently use, and instead making separate module types for libraries, binaries, and tests that inherit from each other. Bug: 259718110 Test: Verified ninja files are unchanged (they only change in the list of soong sources because I added/deleted files) Change-Id: I1e836e2cc4782c7818f91db7df7895de3b8db7ca
2022-10-21Remove flags that enable the new python path behavior Cole Faust
The new behavior has been enabled by default, and these flags aren't necessary anymore. Fixes: 245583294 Test: m py_dont_import_folder_of_entrypoint_test && /ssd/aosp-master/out/host/linux-x86/testcases/py_dont_import_folder_of_entrypoint_test/x86_64/py_dont_import_folder_of_entrypoint_test Change-Id: I5b6f98da51791bc5d28662ef799a10c1bb6a35a0
2022-10-11Revert^2 "Enable new python path behavior by default" Cole Faust
This reverts commit 4d5cc709fa16f5dc78f690369e946fcf66156312. Reason for revert: Acloud issue was fixed in aosp/2243150, make sure the host unit tests are run on this cl via go/abtd before submitting Change-Id: Ie148430d331a01a9877adaf3cb88f1c57af136a0 Bug: 245583294 Bug: 247578564 Bug: 251688241
2022-10-07Revert "Enable new python path behavior by default" Lais Andrade
This reverts commit 5fe655d1c71ee079dd46e93c96b54aae39a10d16. Reason for revert: DroidMonitor: Potential culprit for Bug b/251688241 - verifying through ABTD before revert submission. This is part of the standard investigation process, and does not mean your CL will be reverted. Change-Id: I1ea4df27557fc891d8f0156ec5681f9b0c601c98
2022-10-05Enable new python path behavior by default Cole Faust
Now that usages have been updated, enable the new behavior by default. If everything goes well with this, we can then remove the flags. Bug: 245583294 Bug: 247578564 Test: Presubmits Change-Id: Idac8d2dc0f96ae750dba4a99b32525676d0d6f25
2022-09-23Add flag to not add top-level modules to PYTHONPATH Cole Faust
stub_template_host.txt added all the top-level modules to the PYTHONPATH, which isn't correct, and caused absl.logging to override the built-in logging module. Removing this also makes it more consistent with python binaries built with embedded_launcher: true. embedded_launcher: true binaries don't add the top-level modules. Fixes: 245583294 Test: m py_dont_add_top_level_dirs_test && out/host/linux-x86/testcases/py_dont_add_top_level_dirs_test/x86_64/py_dont_add_top_level_dirs_test Change-Id: Id3069565d2b2c4b2bda0ff5301e757a7b4201751
2022-09-20Make protobufs respect pkg_path properly Cole Faust
Currently, python protobuf sources are generated as if pkg_path didn't exist, but then are moved into the pkg_path directory after being generated. This means they're generated with import statements in them that don't include the pkg_path. These import statements won't work at all when pkg_path is at least 2 levels deep, but currently erroneously work with a 1 level deep pkg_path because we mistakenly add the top-level modules in a soong-built python zip to the PYTHONPATH. We want to remove those modules from the PYTHONPATH, so the generated protobuf source files have to use the correct imports. Since there are existing cases of code that needs to be updated, guard this new behavior behind a flag, protos_respect_pkg_path. We will set this to true on modules individually as we update them, and then eventually change the default to true and remove this flag. Bug: 247578564 Test: m py_proto_pkg_path_test && out/host/linux-x86/nativetest64/py_proto_pkg_path_test/py_proto_pkg_path_test Change-Id: I3695cf5521837da087592f2ad5350201035b7b0e
2022-08-17Reformat build/soong for go 1.19 Colin Cross
Test: none Change-Id: I132368f0fcbdb5ea088b5b84dbe4ccfdd9e94cad
2022-06-09Allowlist apexer for bp2build Cole Faust
This also introduces a workaround for the fact that apexer depends on aapt2, but aapt2 doesn't build with bp2build yet. Aapt2 is removed from apexer's requirements during bp2build. Bug: 204244290 Test: ./build/bazel/ci/bp2build.sh Change-Id: I837597ce035c7d5c06e1a3957166583a7a94b5c7
2022-05-26Add bp2build converter for python protobuf files Cole Faust
Bug: 196084681 Test: b run //build/bazel/examples/python/protobuf:build_bazel_examples_python_protobuf_main --config=linux_x86_64 Change-Id: I4d806902d262351231f64686a5d24513a25d9749
2022-03-08Support building python pars against static or shared musl libc Colin Cross
Support building python pars against musl libc by either adding libc_musl as an install dependency or using the static version of the python launchers. Bug: 179809553 Test: m USE_HOST_MUSL=true apexer deapexer Test: m USE_HOST_MUSL=true BUILD_HOST_static=true apexer deapexer Change-Id: I4a56efb227bea746836eb785d929eb4b5d8e15f1
2021-12-14Use one mutator for all bp2build conversion. Liz Kammer
Each conversion required defining a separate mutator, which will each operate on _all_ modules and requires each to repeat checks whether the mutator should operator. Instead, we introduce a single mutator and modules can define a ConvertWithBp2build to implement bp2build conversion for that module. Test: bp2build.sh Bug: 183079158 Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
2021-10-01Have python_*{,_host} handle arch-variants Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
Bug: 196081778 Test: TestPython*{,Host}ArchVariance Test: go test Test: mixed_{libc,droid}.sh Change-Id: I89304e58f5bacd61534732bade4ad6bb5f2671c0
2021-09-20Remove arch-variant from some python properties Liz Kammer
Version and package should not vary based on architecture. Test: m nothing Change-Id: Id9cb655d9c25caee3b6068a261b3db36e1654467
2021-09-14Merge "Add module based host-tools snapshot" Rob Seymour
2021-09-13Add module based host-tools snapshot Rob Seymour
Add new module: host_snapshot { name: "host-snapshot" deps: [ (list of host tools) ], ... } Package host tools using android.PackagingBase to capture host tools and transitive packaging data. Add JSON meta data to snapshot that allows snapshot to be installed via development/vendor_snapshot/update.py Add support to generate a fake host snapshot of all host modules that is used to detect required modules via development/vendor_snapshot/update.py. Bug: 192556798 Bug: 194799048 Bug: 192896149 Test: m HOST_FAKE_SNAPSHOT_ENABLE=true host-fake-snapshot dist -- check snapshot exists in dist Change-Id: I849c4db801cd858408f6fe6a3ce69262a23a5be9
2021-08-20Fix inverted error message Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
The code checks to see if the `child` is not a Python library, but then when reporting the error it uses the module name as the dependency name and the child (dependency) as the module name Test: Existing tests pass Change-Id: Ied606342291312d8485a8fd2ddcc580ad24ae82f
2021-05-03Support blueprint_go_binary in PathForModuleSrc Colin Cross
Allow blueprint_go_binary to be used as test data by supporting it in PathForModuleSrc. Also make python use GetDirectDepsWithTag instead of VisitDirectDeps so it doesn't fail on the dependency to a Blueprint (non-Android) module. Fixes: 186528269 Test: manual Change-Id: Ibc1d7caf7bf1fa5db805c3b885a95df4baee0ecf
2021-03-17Add preparers for python Paul Duffin
Bug: 181070625 Test: m nothing Change-Id: I590ad62db6fd62719c83286780563cf6d55e3e17
2021-03-09bp2build: add python_binary_host converter. Jingwen Chen
This CL adds a simple python_binary_host to native py_binary converter for standalone modules that don't depend on libs. It also adds support for the conditional py2/py3 build target based on the version prop. Test: milestone-2/demo.sh full Test: bazel query 'kind("py_binary rule", //bionic/...)' Test: bazel run //bionic/libc/tools:genfunctosyscallnrs Test: bazel run //bionic/libc/tools:genseccomp Test: go tests Fixes: 182236395 Change-Id: Ibe5ec6cd0dc12a61b3a449a8c723d80b891fae42
2020-12-01Add comments/refactor for python.go Liz Kammer
Test: all soong tests Test: m nothing Change-Id: Ib3b95d7c2831b97026e76a39af515fd51c6cb2c7
2020-11-26Merge "Refactor python to use AndroidMkEntries." Liz Kammer
2020-11-24Refactor python to use AndroidMkEntries. Liz Kammer
This is to enable consolidating test compatibility suite handling in Soong's AndroidMk files. Test: go test soong tests Test: m nothing and compare soong/Android mk files -- only diffs is that lines matching "^LOCAL_SHARED_LIBRARIES := $" are absent Change-Id: I4922a1c8b0231cd98ebd2bc7257956eb9c0d240c
2020-11-24Merge "Annotate dependency tags for dependencies of installed files" Colin Cross
2020-11-23Annotate dependency tags for dependencies of installed files Colin Cross
Relands Ic22603a5c0718b5a21686672a7471f952b4d1017 with a minor change to track libc++ dependencies for python hosts and after a fix to an internal genrule that depended on transitively installed java libraries (ag/13068670). Soong currently assumes that installed files should depend on installed files of all transitive dependencies, which results in extra installed file dependencies through genrules, static libs, etc. Annotate dependency tags for dependencies for which the installed files are necessary such as shared libraries and JNI libraries. This avoids extra installed files, and is also a first step towards genrules using their own copy of tools instead of the installed copy. Bug: 124313442 Test: m checkbuild Test: java.TestBinary Test: cc.TestInstallSharedLibs Test: deptag_test.go Change-Id: I725871249d561428e6f67bba6a7c65b580012b72
2020-11-23Add java_data parameter to python modules Colin Cross
csuite has a python module that wants to embed the outputs of java modules. This has caused issues with mismatched variants bewteen the arch-specific python module and the common java modules. Add a java_data property that is similar to the data property but uses the common arch variant. Bug: 173977903 Test: m checkbuild Change-Id: I0f2f0e4159650cd5d42b510d5177678e7ee91b4d
2020-11-23Use local variations for python version splits Colin Cross
Use a local variation for python version splits. This causes dependencies from python modules not to look for variations that match the version split, which will simplify a python module depending on a java module (for example for test data) because Soong won't look for a python version variant of the java module. Bug: 173977903 Test: go test ./python Change-Id: Ib034140c478ffbc7467ab830d3cfa5683c31d05c
2020-11-16Fix bug in python module hasExtention. Liz Kammer
Test: m nothing Test: soong go tests Change-Id: I1944a3bbb871bc1021c841d944e765bb0cadba86
2020-11-02Consolidate python version properties in mutator Liz Kammer
Test: m Bug: 165114590 Change-Id: Iaf874f7098317a52827b113167e3dc1ed044e907
2020-07-27Add `data_native_bins` property to java_test_host Liz Kammer
When multiple os/arch variants are supported, java_test_host could not find a matching arch due to java having arch:common, whereas native binaries support a specific architecture. This change adds the property `data_native_bins` in order to support binaries with the appropriate os/arch variants. Test: m FirmwareDtboVerification with data_native_bins Test: forrest Bug: 153848038 Change-Id: I45adebff0fde2811d5ef5620c697b97b768c951f
2020-06-22Implement OutputFiles for python modules. Liz Kammer
This enables python binaries to be included in data properties with the format ":module". Test: m FirmwareDtboVerification with local changes Bug: 153848038 Change-Id: I2a02dc382fcbb142d8e4fbdbb6ca461986627d25
2020-01-20Python3 on device links against liblog Dan Willemsen
Fixes: 148037229 Test: cherry-pick https://r.android.com/1212194; m try_python_binary_test Change-Id: I6e26ba232d7698a4889a6e0be1aeb09d54cc4dc5
2019-11-04Add python3 embedded launcher support Dan Willemsen
Test: m par_test{,3}; build/soong/python/tests/runtest.sh Change-Id: I9c0fac9e2947616fdeedbfc55026dc3065966e71
2019-10-16Add method to determine variations from a Target Colin Cross
The arch variants are hardcoded in every module type. Refactor them out into a Target.Variations() method in preparation for splitting the arch mutator into two, which will require using different variations. Test: m checkbuild Change-Id: I28ef7cd5168095ac888fe77f04e27f9ad81978c0
generated by cgit v1.2.3-59-g8ed1b (git 2.39.0) at 2025-10-20 17:08:24 +0000