summaryrefslogtreecommitdiff
path: root/python/python.go
AgeCommit message (Collapse)Author
2025-03-14Apply gofmt to build/soong/python Spandan Das
Test: go build ./python Change-Id: I1a25ff919071797eedb95b4abcca4d7c09faab97
2025-03-12Add Shared libraries support for Android python libraries Brett Brotherton
- New shared_libs property to Python code for specify shared libs that should be packed into the library for modules with compiled code. - Expand shared_lib dependencies when building the python library. - Change python main entrypoint to extract the zip if there are any shared libraries in the executable to workaround linker limitations Bug: 395678202 Change-Id: Id94caebf36f7eb1cfa492b2fa78f8c56623e9a43 Test: m py-grpcio
2025-02-24Revert "Temporarily disable python precompilation for py3.13" Dan Willemsen
This reverts commit 7ca4d103891b6a508266d8b37def22de535a1811. prebuilts/build-tools has been updated to 3.13. Bug: 388344853 Test: treehugger Change-Id: Ifb3bc461aa77f693a57782284da1784ad651a6ad
2025-02-10Merge "Temporarily disable python precompilation for py3.13" into main Dan Willemsen
2025-02-03Temporarily disable python precompilation for py3.13 Dan Willemsen
The precompilation logic assumes that the source tree matches the python prebuilts, which is not necessarily true. This very much breaks during major version upgrades, so just disable precompilation temporarily while we generate new prebuilts. Bug: 388344853 Test: treehugger Change-Id: I8d412c5cc112566d1bdad1f03ade606331dacb02
2025-01-31Remove python2 support from soong rules Cole Faust
Python2 has been removed from the android build. Removing the python version transition mutator should lead to some slight analysis time performance improvements. I also made embedded_launcher in base_properties no longer blueprint:"mutated", so that we can migrate usages to the main property struct and remove the `version` property struct. Many python tests were not being run due to a `if d.desc != "module with duplicate runfile path" { continue }` line, I removed that and fixed the tests. I also removed the test for an error when there are no source files, because that check has been relaxed since, and I think it's legitimate to have a python library that only has data files, not source files. And there are some of those libraries in the source tree. Bug: 203436762 Test: m nothing --no-skip-soong-tests Change-Id: Idf18e39233418237ecc83f70f65d83d4ea496633
2025-01-21Convert python modules to use ModuleProxy. Yu Liu
Bug: 377723687 Test: Unit tests and compare the ninja and mk files generated. Change-Id: Ide5bdf2bc1bf41efaf694a66769df907888e51cb
2025-01-07Add device_first_data to python modules Inseob Kim
Bug: 354060773 Test: atest avf_early_vm_test Change-Id: I3c6065598bf4c3e8670e6fa78dfdfeea75e5fd38
2024-12-18Remove source_file_provider.go after it's last usage was removed. Yu Liu
Bug: 377723687 Test: Unit tests and compare the ninja and mk files generated. Change-Id: Iff10448dc4dda49e1f02258a4c113db75778a95d
2024-10-21Add new properties to aid in removing the 1-variant fallback Cole Faust
These new properties are essentially methods to specify "outgoing transitions" in blueprint files. There are lots of host tests that want to include apps built for device in their data, so they need a property that adds dependencies based on the device variants instead of copying the same host variants. After this cl is submitted, I'll do an LSC to update all the usages that are relying on the 1-variant fallback to use these properties instead. Bug: 372091092 Test: m nothing --no-skip-soong-tests Change-Id: I45b8fb024da120ad61606e3a21de86e4392be2a4
2024-09-09Remove BUILD_BROKEN_USES_SOONG_PYTHON2_MODULES Cole Faust
It's been available for over a year, lets progress the python3 migration by removing it. Bug: 203436762 Test: Presubmits Change-Id: I46a6477ec65e98b7e6df5cc1359f676dcbd0f08e
2024-07-17Convert python_version mutator to TransitionMutator Colin Cross
Replace python.versionMutator with a TransitionMutator. Bug: 319288033 Flag: EXEMPT refactor Test: all soong tests pass Change-Id: I12eafbe0563b202db8cadf69cbacdf39ce6f11b5
2024-05-08relax the restriction on .proto files in data Raphael Blistein
Context: yaqs/7447701439926763520#a1 Bug: b/339454115 Test: go test Change-Id: If3ed81334a192993d32404301b55ca7b4f5780ae
2024-04-22Merge "Revert^2 "Make embedded_launcher true by default"" into main Cole Faust
2024-04-18Revert^2 "Make embedded_launcher true by default" Cole Faust
This reverts commit 578066be90d3e1960c02b5c80c29e92bca70a70b. Reason for revert: Relanding Change-Id: I63f01362cc11c02c4800ae1c21652c7cf4b8a799
2024-04-16Implement test-only for python_ rules Ronald Braunstein
Similar to: aosp/3040036 and aosp/3035329, implements a test-only property for python_library and sets it for python_host. This combined with aosp/3045317 produces: gqui from "flatten(~/aosp-main-with-phones/out/soong/ownership/all_teams.pb, teams)" proto ~/aosp-main-with-phones/build/soong/android/team_proto/team.proto:AllTeams 'select teams.kind, count(*) total, sum(teams.test_only) as test_only, sum(teams.top_level_target) as top_level, where teams.kind like "%py%" group by teams.kind' +-----------------------------+-------+-----------+-----------+ | teams.kind | total | test_only | top_level | +-----------------------------+-------+-----------+-----------+ | cpython3_cc_defaults | 1 | 0 | 0 | | cpython3_cc_prebuilt_binary | 4 | 0 | 0 | | python_binary_host | 222 | 0 | 0 | | python_defaults | 27 | 0 | 0 | | python_library | 38 | 0 | 0 | | python_library_host | 131 | 0 | 0 | | python_test | 5 | 5 | 5 | | python_test_host | 172 | 172 | 172 | +-----------------------------+-------+-----------+-----------+ Change-Id: Iea14fcdbfd782d37ac0a3c780820deeed5f1923d Test: m blueprint_tests Test: go test ./python
2024-04-11Merge "Revert "Make embedded_launcher true by default"" into main Charlie Boutier
2024-04-11Revert "Make embedded_launcher true by default" Charlie Boutier
This reverts commit 8a038ed871230ba6280642b5c0e83a37d076aa79. Reason for revert: investigate breaking test suits Avatar and BumbleBluetoothTests Change-Id: I8e4498fb5854bf62df6d161ac0c6cce49d471afe
2024-04-10Merge "Make embedded_launcher true by default" into main Cole Faust
2024-03-28Remove the mkdir of the parent Directories for protobufs. Haamed Gheibi
Bug: 330794730 Change-Id: I7b221ffc9dde67c27186848727c4a698644507aa
2024-03-26Make embedded_launcher true by default Cole Faust
embedded_launcher causes the python interpreter to be bundled in the built zip file. This is beneficial because: - You can export the built executable and run it on systems without python or an old version of python. The CI servers have python 3.7, so it's useful to update that. - Because the python version is known, the python sources will be precompiled. - It will start running the code directly from the zip file, without extracting the binary to a temporary directory like non-embedded launcher binaries. This is particuarly useful because some developers use the extracted files directly instead of using `importlib.resources`, so having the non-extracting implementation gives them a nudge in the right direction. Make embedded_launcher the default so that more developers will do the right thing when developing python programs. Bug: 331488610 Test: Presubmits Change-Id: Ideb113a1d4d3b29ac04d57a48d111a99f77b2dfc
2023-12-14Convert SetProvider to generic providers API Colin Cross
Convert all of the callers of SetProvider to use the type-safe android.SetProvider API. Bug: 316410648 Test: builds Change-Id: If58f4b5355264ddab2045bc3591a4eac19cd58fc
2023-12-08Remove ConvertWithBp2build implementations Colin Cross
Remove the ConvertWithBp2build implementations from all the module types, along with the related code. Bug: 315353489 Test: m blueprint_tests Change-Id: I212672286686a318893bc7348ddd5a5ec51e77a7
2023-12-05Add source file provider for genrule/srcs, python libraries and rust libraries. Aditya Choudhary
Change-Id: I2d7d4684a10c15aeecc27b8db800ab27a807d2e2
2023-08-31Merge "Revert "Temporarily disable python precompilation for py3.11"" into main Dan Willemsen
2023-08-28Merge changes from topic "java-proto-include-dir" into main Treehugger Robot
* changes: Handle proto.include_dirs for java Translate python_libray.pkg_path to proto.import_prefix
2023-08-24Revert "Temporarily disable python precompilation for py3.11" Dan Willemsen
This reverts commit b1094d602bbd1fc6d4d60ace63ff82a92b479b23. Reason for revert: prebuilts updated in aosp/2726579 Bug: 278602456 Change-Id: I750b0f7e4dd94329a4455499f869a1823e660d77
2023-08-24Translate python_libray.pkg_path to proto.import_prefix Spandan Das
If a python_library uses a pkg_path foo/bar, then the proto srcs in that libray need to import the dep .proto as foo/bar/proto.proto. This behavior is restricted to python modules. To implement this is in bp2build, this CL creates a new interface with a single method `PkgPath`. Only python module structs implement this interface, and this method is only available during bp2build Test: Added a bp2build unit test Test: TH Change-Id: If8d207c0b321f75337a053795826b283a5eaaf46
2023-08-22Temporarily disable python precompilation for py3.11 Dan Willemsen
The precompilation logic assumes that the source tree matches the python prebuilts, which is not necessarily true. This very much breaks during major version upgrades, so just disable precompilation temporarily while we generate new prebuilts. Bug: 278602456 Test: treehugger Change-Id: Idb206bb1d971e8c0fa556c75419623c3bc457898
2023-08-16Prepare soong for python 3.11 Dan Willemsen
Due to upstream changes in python path calculations, I'm simplifying the python launcher to work more like a standard python distribution. This is effectively changing the stdlib path from `internal/stdlib` to `internal/python3.10` (or `3.11`, etc). This allows us to specify the zip file as PYTHONHOME, set PYTHONPLATLIBDIR to `internal` and use the default detection after that. That does mean during upgrades that the stdlib pkg path will change, so move the source vs prebuilt calculation from the Android.bp into Soong to choose which stdlib module to pick up (with the corresponding `pkg_path`) Bug: 278602456 Test: treehugger with python3.10 Test: a python3.11 source + 3.10 prebuilt build Test: a python3.11 source+prebuilt build Change-Id: I8b02e7b22a1f1d1e02819ae1a31a99cdc985542c
2023-05-04Delete Python 2 embedded launcher test. Krzysztof Kosiński
There are no remaining Python 2 binaries using an embedded launcher in Android. Bug: 245854393 Test: m par_test py2-cmd py3-cmd && build/soong/python/tests/runtest.sh Change-Id: I241bbaa417060b51b4d2883011ccb43e22ace4c8
2023-04-13bp2build support for python_test(_host) Cole Faust
There was a request for using b with python tests. bp2build python tests exactly the same way as python binaries so that they can be used with `b`. Bug: None Test: go test Change-Id: Id68a6a73572745a4885b3e5bb1b8452e36baa982
2023-03-09Make using python 2 an error Cole Faust
Except if BUILD_BROKEN_USES_SOONG_PYTHON2_MODULES is set, and except for some core py2 modules that can't be removed until python2 is fully gone. Bug: 203436762 Test: m nothing Change-Id: I62ccb6f5687eab1e79c372ffc234a90ca5b566ac
2023-02-28Replace SortedStringKeys with SortedKeys Cole Faust
Now that we have generics. Bug: 193460475 Test: presubmits Change-Id: I1594fd8feb505175d5c09c03ef397e5ffd5b09cb
2023-02-13Revert "Dynamically assemble the list of python launcher shared libs for ↵ Cole Faust
precompiling" This reverts commit aa5b377ee956f2e7ff2e0fb6630f636ecfe15b84. Reason for revert: this broke musl builds by removing the install dependency from python binaries to the launcher shared libraries: https://android-build.googleplex.com/builds/submitted/9600042/unit_tests_x86_64_musl/latest/view/logs/build_error.log Change-Id: I4e13f333f287831ba13724f13ab2be789a92b476
2023-02-02Dynamically assemble the list of python launcher shared libs for precompiling Cole Faust
Still fall back to the hardcoded list for what to put in LOCAL_SHARED_LIBRARIES, because I couldn't figure out a good way to get those modules at the moment. Bug: 259718110 Test: Presubmits Change-Id: Ia992ffea8819d8c67280b08bc45505f77af09de0
2023-01-27Precompile python sources Cole Faust
This signifigantly improves the startup time of soong-built python binaries. For example, running `m apexer && time out/host/linux-x86/bin/apexer` gives 0.734s before this cl, and 0.094s after. Fixes: 259718110 Test: Presubmits Change-Id: Ib19e83e2c60c39a849525be117279c318de3afa7
2023-01-25Refactor python rules Cole Faust
The goal of this cl is to simplify the python rules, mostly by removing the "decorator" pattern that they currently use, and instead making separate module types for libraries, binaries, and tests that inherit from each other. Bug: 259718110 Test: Verified ninja files are unchanged (they only change in the list of soong sources because I added/deleted files) Change-Id: I1e836e2cc4782c7818f91db7df7895de3b8db7ca
2022-10-21Remove flags that enable the new python path behavior Cole Faust
The new behavior has been enabled by default, and these flags aren't necessary anymore. Fixes: 245583294 Test: m py_dont_import_folder_of_entrypoint_test && /ssd/aosp-master/out/host/linux-x86/testcases/py_dont_import_folder_of_entrypoint_test/x86_64/py_dont_import_folder_of_entrypoint_test Change-Id: I5b6f98da51791bc5d28662ef799a10c1bb6a35a0
2022-10-11Revert^2 "Enable new python path behavior by default" Cole Faust
This reverts commit 4d5cc709fa16f5dc78f690369e946fcf66156312. Reason for revert: Acloud issue was fixed in aosp/2243150, make sure the host unit tests are run on this cl via go/abtd before submitting Change-Id: Ie148430d331a01a9877adaf3cb88f1c57af136a0 Bug: 245583294 Bug: 247578564 Bug: 251688241
2022-10-07Revert "Enable new python path behavior by default" Lais Andrade
This reverts commit 5fe655d1c71ee079dd46e93c96b54aae39a10d16. Reason for revert: DroidMonitor: Potential culprit for Bug b/251688241 - verifying through ABTD before revert submission. This is part of the standard investigation process, and does not mean your CL will be reverted. Change-Id: I1ea4df27557fc891d8f0156ec5681f9b0c601c98
2022-10-05Enable new python path behavior by default Cole Faust
Now that usages have been updated, enable the new behavior by default. If everything goes well with this, we can then remove the flags. Bug: 245583294 Bug: 247578564 Test: Presubmits Change-Id: Idac8d2dc0f96ae750dba4a99b32525676d0d6f25
2022-09-23Add flag to not add top-level modules to PYTHONPATH Cole Faust
stub_template_host.txt added all the top-level modules to the PYTHONPATH, which isn't correct, and caused absl.logging to override the built-in logging module. Removing this also makes it more consistent with python binaries built with embedded_launcher: true. embedded_launcher: true binaries don't add the top-level modules. Fixes: 245583294 Test: m py_dont_add_top_level_dirs_test && out/host/linux-x86/testcases/py_dont_add_top_level_dirs_test/x86_64/py_dont_add_top_level_dirs_test Change-Id: Id3069565d2b2c4b2bda0ff5301e757a7b4201751
2022-09-20Make protobufs respect pkg_path properly Cole Faust
Currently, python protobuf sources are generated as if pkg_path didn't exist, but then are moved into the pkg_path directory after being generated. This means they're generated with import statements in them that don't include the pkg_path. These import statements won't work at all when pkg_path is at least 2 levels deep, but currently erroneously work with a 1 level deep pkg_path because we mistakenly add the top-level modules in a soong-built python zip to the PYTHONPATH. We want to remove those modules from the PYTHONPATH, so the generated protobuf source files have to use the correct imports. Since there are existing cases of code that needs to be updated, guard this new behavior behind a flag, protos_respect_pkg_path. We will set this to true on modules individually as we update them, and then eventually change the default to true and remove this flag. Bug: 247578564 Test: m py_proto_pkg_path_test && out/host/linux-x86/nativetest64/py_proto_pkg_path_test/py_proto_pkg_path_test Change-Id: I3695cf5521837da087592f2ad5350201035b7b0e
2022-08-17Reformat build/soong for go 1.19 Colin Cross
Test: none Change-Id: I132368f0fcbdb5ea088b5b84dbe4ccfdd9e94cad
2022-06-09Allowlist apexer for bp2build Cole Faust
This also introduces a workaround for the fact that apexer depends on aapt2, but aapt2 doesn't build with bp2build yet. Aapt2 is removed from apexer's requirements during bp2build. Bug: 204244290 Test: ./build/bazel/ci/bp2build.sh Change-Id: I837597ce035c7d5c06e1a3957166583a7a94b5c7
2022-05-26Add bp2build converter for python protobuf files Cole Faust
Bug: 196084681 Test: b run //build/bazel/examples/python/protobuf:build_bazel_examples_python_protobuf_main --config=linux_x86_64 Change-Id: I4d806902d262351231f64686a5d24513a25d9749
2022-03-08Support building python pars against static or shared musl libc Colin Cross
Support building python pars against musl libc by either adding libc_musl as an install dependency or using the static version of the python launchers. Bug: 179809553 Test: m USE_HOST_MUSL=true apexer deapexer Test: m USE_HOST_MUSL=true BUILD_HOST_static=true apexer deapexer Change-Id: I4a56efb227bea746836eb785d929eb4b5d8e15f1
2021-12-14Use one mutator for all bp2build conversion. Liz Kammer
Each conversion required defining a separate mutator, which will each operate on _all_ modules and requires each to repeat checks whether the mutator should operator. Instead, we introduce a single mutator and modules can define a ConvertWithBp2build to implement bp2build conversion for that module. Test: bp2build.sh Bug: 183079158 Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
2021-10-01Have python_*{,_host} handle arch-variants Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
Bug: 196081778 Test: TestPython*{,Host}ArchVariance Test: go test Test: mixed_{libc,droid}.sh Change-Id: I89304e58f5bacd61534732bade4ad6bb5f2671c0
generated by cgit v1.2.3-59-g8ed1b (git 2.39.0) at 2025-10-23 10:40:54 +0000