summaryrefslogtreecommitdiff
path: root/python/python.go
AgeCommit message (Collapse)Author
2022-10-21Remove flags that enable the new python path behavior Cole Faust
The new behavior has been enabled by default, and these flags aren't necessary anymore. Fixes: 245583294 Test: m py_dont_import_folder_of_entrypoint_test && /ssd/aosp-master/out/host/linux-x86/testcases/py_dont_import_folder_of_entrypoint_test/x86_64/py_dont_import_folder_of_entrypoint_test Change-Id: I5b6f98da51791bc5d28662ef799a10c1bb6a35a0
2022-10-11Revert^2 "Enable new python path behavior by default" Cole Faust
This reverts commit 4d5cc709fa16f5dc78f690369e946fcf66156312. Reason for revert: Acloud issue was fixed in aosp/2243150, make sure the host unit tests are run on this cl via go/abtd before submitting Change-Id: Ie148430d331a01a9877adaf3cb88f1c57af136a0 Bug: 245583294 Bug: 247578564 Bug: 251688241
2022-10-07Revert "Enable new python path behavior by default" Lais Andrade
This reverts commit 5fe655d1c71ee079dd46e93c96b54aae39a10d16. Reason for revert: DroidMonitor: Potential culprit for Bug b/251688241 - verifying through ABTD before revert submission. This is part of the standard investigation process, and does not mean your CL will be reverted. Change-Id: I1ea4df27557fc891d8f0156ec5681f9b0c601c98
2022-10-05Enable new python path behavior by default Cole Faust
Now that usages have been updated, enable the new behavior by default. If everything goes well with this, we can then remove the flags. Bug: 245583294 Bug: 247578564 Test: Presubmits Change-Id: Idac8d2dc0f96ae750dba4a99b32525676d0d6f25
2022-09-23Add flag to not add top-level modules to PYTHONPATH Cole Faust
stub_template_host.txt added all the top-level modules to the PYTHONPATH, which isn't correct, and caused absl.logging to override the built-in logging module. Removing this also makes it more consistent with python binaries built with embedded_launcher: true. embedded_launcher: true binaries don't add the top-level modules. Fixes: 245583294 Test: m py_dont_add_top_level_dirs_test && out/host/linux-x86/testcases/py_dont_add_top_level_dirs_test/x86_64/py_dont_add_top_level_dirs_test Change-Id: Id3069565d2b2c4b2bda0ff5301e757a7b4201751
2022-09-20Make protobufs respect pkg_path properly Cole Faust
Currently, python protobuf sources are generated as if pkg_path didn't exist, but then are moved into the pkg_path directory after being generated. This means they're generated with import statements in them that don't include the pkg_path. These import statements won't work at all when pkg_path is at least 2 levels deep, but currently erroneously work with a 1 level deep pkg_path because we mistakenly add the top-level modules in a soong-built python zip to the PYTHONPATH. We want to remove those modules from the PYTHONPATH, so the generated protobuf source files have to use the correct imports. Since there are existing cases of code that needs to be updated, guard this new behavior behind a flag, protos_respect_pkg_path. We will set this to true on modules individually as we update them, and then eventually change the default to true and remove this flag. Bug: 247578564 Test: m py_proto_pkg_path_test && out/host/linux-x86/nativetest64/py_proto_pkg_path_test/py_proto_pkg_path_test Change-Id: I3695cf5521837da087592f2ad5350201035b7b0e
2022-08-17Reformat build/soong for go 1.19 Colin Cross
Test: none Change-Id: I132368f0fcbdb5ea088b5b84dbe4ccfdd9e94cad
2022-06-09Allowlist apexer for bp2build Cole Faust
This also introduces a workaround for the fact that apexer depends on aapt2, but aapt2 doesn't build with bp2build yet. Aapt2 is removed from apexer's requirements during bp2build. Bug: 204244290 Test: ./build/bazel/ci/bp2build.sh Change-Id: I837597ce035c7d5c06e1a3957166583a7a94b5c7
2022-05-26Add bp2build converter for python protobuf files Cole Faust
Bug: 196084681 Test: b run //build/bazel/examples/python/protobuf:build_bazel_examples_python_protobuf_main --config=linux_x86_64 Change-Id: I4d806902d262351231f64686a5d24513a25d9749
2022-03-08Support building python pars against static or shared musl libc Colin Cross
Support building python pars against musl libc by either adding libc_musl as an install dependency or using the static version of the python launchers. Bug: 179809553 Test: m USE_HOST_MUSL=true apexer deapexer Test: m USE_HOST_MUSL=true BUILD_HOST_static=true apexer deapexer Change-Id: I4a56efb227bea746836eb785d929eb4b5d8e15f1
2021-12-14Use one mutator for all bp2build conversion. Liz Kammer
Each conversion required defining a separate mutator, which will each operate on _all_ modules and requires each to repeat checks whether the mutator should operator. Instead, we introduce a single mutator and modules can define a ConvertWithBp2build to implement bp2build conversion for that module. Test: bp2build.sh Bug: 183079158 Change-Id: I99d4b51f441c2903879092c5b56313d606d4338d
2021-10-01Have python_*{,_host} handle arch-variants Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
Bug: 196081778 Test: TestPython*{,Host}ArchVariance Test: go test Test: mixed_{libc,droid}.sh Change-Id: I89304e58f5bacd61534732bade4ad6bb5f2671c0
2021-09-20Remove arch-variant from some python properties Liz Kammer
Version and package should not vary based on architecture. Test: m nothing Change-Id: Id9cb655d9c25caee3b6068a261b3db36e1654467
2021-09-14Merge "Add module based host-tools snapshot" Rob Seymour
2021-09-13Add module based host-tools snapshot Rob Seymour
Add new module: host_snapshot { name: "host-snapshot" deps: [ (list of host tools) ], ... } Package host tools using android.PackagingBase to capture host tools and transitive packaging data. Add JSON meta data to snapshot that allows snapshot to be installed via development/vendor_snapshot/update.py Add support to generate a fake host snapshot of all host modules that is used to detect required modules via development/vendor_snapshot/update.py. Bug: 192556798 Bug: 194799048 Bug: 192896149 Test: m HOST_FAKE_SNAPSHOT_ENABLE=true host-fake-snapshot dist -- check snapshot exists in dist Change-Id: I849c4db801cd858408f6fe6a3ce69262a23a5be9
2021-08-20Fix inverted error message Alex Márquez Pérez Muñíz Díaz Púras Thaureaux
The code checks to see if the `child` is not a Python library, but then when reporting the error it uses the module name as the dependency name and the child (dependency) as the module name Test: Existing tests pass Change-Id: Ied606342291312d8485a8fd2ddcc580ad24ae82f
2021-05-03Support blueprint_go_binary in PathForModuleSrc Colin Cross
Allow blueprint_go_binary to be used as test data by supporting it in PathForModuleSrc. Also make python use GetDirectDepsWithTag instead of VisitDirectDeps so it doesn't fail on the dependency to a Blueprint (non-Android) module. Fixes: 186528269 Test: manual Change-Id: Ibc1d7caf7bf1fa5db805c3b885a95df4baee0ecf
2021-03-17Add preparers for python Paul Duffin
Bug: 181070625 Test: m nothing Change-Id: I590ad62db6fd62719c83286780563cf6d55e3e17
2021-03-09bp2build: add python_binary_host converter. Jingwen Chen
This CL adds a simple python_binary_host to native py_binary converter for standalone modules that don't depend on libs. It also adds support for the conditional py2/py3 build target based on the version prop. Test: milestone-2/demo.sh full Test: bazel query 'kind("py_binary rule", //bionic/...)' Test: bazel run //bionic/libc/tools:genfunctosyscallnrs Test: bazel run //bionic/libc/tools:genseccomp Test: go tests Fixes: 182236395 Change-Id: Ibe5ec6cd0dc12a61b3a449a8c723d80b891fae42
2020-12-01Add comments/refactor for python.go Liz Kammer
Test: all soong tests Test: m nothing Change-Id: Ib3b95d7c2831b97026e76a39af515fd51c6cb2c7
2020-11-26Merge "Refactor python to use AndroidMkEntries." Liz Kammer
2020-11-24Refactor python to use AndroidMkEntries. Liz Kammer
This is to enable consolidating test compatibility suite handling in Soong's AndroidMk files. Test: go test soong tests Test: m nothing and compare soong/Android mk files -- only diffs is that lines matching "^LOCAL_SHARED_LIBRARIES := $" are absent Change-Id: I4922a1c8b0231cd98ebd2bc7257956eb9c0d240c
2020-11-24Merge "Annotate dependency tags for dependencies of installed files" Colin Cross
2020-11-23Annotate dependency tags for dependencies of installed files Colin Cross
Relands Ic22603a5c0718b5a21686672a7471f952b4d1017 with a minor change to track libc++ dependencies for python hosts and after a fix to an internal genrule that depended on transitively installed java libraries (ag/13068670). Soong currently assumes that installed files should depend on installed files of all transitive dependencies, which results in extra installed file dependencies through genrules, static libs, etc. Annotate dependency tags for dependencies for which the installed files are necessary such as shared libraries and JNI libraries. This avoids extra installed files, and is also a first step towards genrules using their own copy of tools instead of the installed copy. Bug: 124313442 Test: m checkbuild Test: java.TestBinary Test: cc.TestInstallSharedLibs Test: deptag_test.go Change-Id: I725871249d561428e6f67bba6a7c65b580012b72
2020-11-23Add java_data parameter to python modules Colin Cross
csuite has a python module that wants to embed the outputs of java modules. This has caused issues with mismatched variants bewteen the arch-specific python module and the common java modules. Add a java_data property that is similar to the data property but uses the common arch variant. Bug: 173977903 Test: m checkbuild Change-Id: I0f2f0e4159650cd5d42b510d5177678e7ee91b4d
2020-11-23Use local variations for python version splits Colin Cross
Use a local variation for python version splits. This causes dependencies from python modules not to look for variations that match the version split, which will simplify a python module depending on a java module (for example for test data) because Soong won't look for a python version variant of the java module. Bug: 173977903 Test: go test ./python Change-Id: Ib034140c478ffbc7467ab830d3cfa5683c31d05c
2020-11-16Fix bug in python module hasExtention. Liz Kammer
Test: m nothing Test: soong go tests Change-Id: I1944a3bbb871bc1021c841d944e765bb0cadba86
2020-11-02Consolidate python version properties in mutator Liz Kammer
Test: m Bug: 165114590 Change-Id: Iaf874f7098317a52827b113167e3dc1ed044e907
2020-07-27Add `data_native_bins` property to java_test_host Liz Kammer
When multiple os/arch variants are supported, java_test_host could not find a matching arch due to java having arch:common, whereas native binaries support a specific architecture. This change adds the property `data_native_bins` in order to support binaries with the appropriate os/arch variants. Test: m FirmwareDtboVerification with data_native_bins Test: forrest Bug: 153848038 Change-Id: I45adebff0fde2811d5ef5620c697b97b768c951f
2020-06-22Implement OutputFiles for python modules. Liz Kammer
This enables python binaries to be included in data properties with the format ":module". Test: m FirmwareDtboVerification with local changes Bug: 153848038 Change-Id: I2a02dc382fcbb142d8e4fbdbb6ca461986627d25
2020-01-20Python3 on device links against liblog Dan Willemsen
Fixes: 148037229 Test: cherry-pick https://r.android.com/1212194; m try_python_binary_test Change-Id: I6e26ba232d7698a4889a6e0be1aeb09d54cc4dc5
2019-11-04Add python3 embedded launcher support Dan Willemsen
Test: m par_test{,3}; build/soong/python/tests/runtest.sh Change-Id: I9c0fac9e2947616fdeedbfc55026dc3065966e71
2019-10-16Add method to determine variations from a Target Colin Cross
The arch variants are hardcoded in every module type. Refactor them out into a Target.Variations() method in preparation for splitting the arch mutator into two, which will require using different variations. Test: m checkbuild Change-Id: I28ef7cd5168095ac888fe77f04e27f9ad81978c0
2019-04-02Add support for protoc plugins Colin Cross
Add a proto.plugin property to allow specifying a custom protoc plugin to generate the code. Fixes: 70706119 Test: m am StreamingProtoTest Change-Id: I1ecdd346284b42bbcc8297019d98d2cd564eb94c
2019-04-02Move proto compilation to RuleBuilder Colin Cross
Using blueprint.Rule for protoc commands was causing code duplication because there was no good way to run the same protoc for cc, java and python but then run custom source packaging steps for java and python. Move most of the code into a common function that returns a RuleBuilder, and then let java and python add their own commands at the end of the rule. Bug: 70706119 Test: All Soong tests Test: m checkbuild Change-Id: Ic692136775d273bcc4f4de99620ab4878667c83a
2019-03-20Replace ctx.ExpandSources with android.PathsForModuleSrc Colin Cross
Move the logic from ctx.ExpandSources into android.PathsForModuleSrc and ctx.ExpandSource into android.PathForModuleSrc, and deprecate them. When combined with the pathDepsMutator this will let all properties that take source paths also take filegroups or genrule outputs, as long as they are tagged with `android:"path"`. Test: All soong tests Change-Id: I01625e76b5da19240e9649bf26a014eeeafcab8f
2019-03-07Annotate paths and deprecate ExtractSource(s)Deps Colin Cross
Add `android:"path"` to all properties that take paths to source files, and remove the calls to ExtractSource(s)Deps, the pathsDepsMutator will add the necessary SourceDepTag dependency. Test: All soong tests Change-Id: I488ba1a5d680aaa50b04fc38acf693e23c6d4d6d
2019-02-17Support building a par file that does not automatically run Dan Willemsen
Mainly so that we can build a `python` prebuilt that acts like the normal python install, though you could also build different ones with more packages pre-installed. Bug: 117811537 Test: move built py2-cmd into prebuilts/build-tools/path/linux-x86/python and build Change-Id: I21215f6fd3754d89f8c65e1dfeb3f2deea23239f
2019-01-29Add missing dependencies for python_test Logan Chien
This commit adds missing shared lib dependencies for `python_test` modules with embedded launcher. Bug: 119086738 Test: CHECK_ELF_FIELS=true make check-elf-files Change-Id: I26f8e1eb9086930093f60c7daa54469850fab32d
2018-07-22Fix `go vet` issues Dan Willemsen
Test: go vet ./... Change-Id: Ifb936ccc5e2b5a2c3fcbbbcb54f680e2973ea1b3
2018-06-22Use WalkDeps instead of VisitDepsDepthFirst Colin Cross
VisitDepsDepthFirst is almost never correct, as it can't query dependency tags of multiple dependencies between the same two modules. Replace VisitDepsDepthFirst with WalkDeps in sanitize.go and python.go. Also verify the dependency tag before continuing to recurse to ensure that they don't recurse through genrules and into unrelated modules. Test: m checkbuild Change-Id: I2f7560126f56b51a40ec39dfbdcc18b5891489f7
2018-06-04Fix embedded_launcher can't find files Nan Zhang
The problem came from Python libraries doesn't know the information that we enabled embedded_launcher (only Python binary knows about that). And we can't simply remove runfiles dir for Python libraries since host Python mode need this. Bug: b/80441699 Test: m perf_profo_flames Change-Id: I73ffc4d7504f95a708ae7cca47bc6c15a673aa31
2018-05-23Allow "-" to be part of Python file names Nan Zhang
Test: m -j py-setuptools Bug: b/79751992 Change-Id: I4cd0cf8671a9bb912a8ba819265274ab0c9e0074
2018-05-23Fix Protobuf importing issue for embedded Python Nan Zhang
As I mentioned in b/79751992 #4, when launcher loads __main__ as entry point, the module dict got set using absolute pkg path which was runfiles.google.protobuf which caused the type checking failed (expected: google.protobuf for type()). After then, starting with non entrypoint python files, the importing module dict would be set with from...import... So remove dir:runfiles/ from zip section so that during entrypoint Python file importing the proto files using from...import..., the type checking can be passed. Bug: b/79751992 Test: m clean && m out/host/linux-x86/bin/perf_proto_stack; And execute it: usage: perf_proto_stack [-h] [--syms SYMS] [--json-out JSON_OUT] [--print-samples] [--skip-kernel-syms] [--print-pid-histogram] [--print-sym-histogram] [--print-dso-histogram] [--parallel PARALLEL] file [file ...] perf_proto_stack: error: too few arguments Change-Id: I95a4a9ee95ce7f90d893ea0529c384d7b944abe4
2018-04-10Consistently use Bool instead of proptools.Bool Colin Cross
Use Bool instead of proptools.Bool and String instead of proptools.String. Test: m checkbuild Change-Id: I32d84add9f27128c7a65413e9612fd920613584f
2018-03-22Add Python protobuf support. Nan Zhang
Python protobuf std libs will be wrapped in final binary/test par file. Bug: b/70568913 Test: manually create real examples. Change-Id: I7376ec9175f3e03d1adbd20858a7f74e826387ad
2018-03-09Remove timestamp based filelist file for tracking Python dependencies Nan Zhang
Each Python module will generate a zip file containing source & data files. The Python binary will collect all its dependencies and use merge_zips to merge each zip file to create a final .par file. Test: m -j checkbuild && real examples: Bug: b/70568913 Change-Id: I9ff232d461d33e1c06026e7dcb5b124bf02c3ce5
2018-02-15Revert "Revert "Support filegroup in exclude_srcs"" Nan Zhang
This reverts commit 606e9de344fae07473dd79e5ac556886a72035de. Reason for revert: <try to fix the broken build yesterday> Change-Id: I2963b9af63c7c7398159e5e9a1e448266e1c81d5 Test: unittest
2018-02-09Revert "Support filegroup in exclude_srcs" Tobias Thierer
This reverts commit f36a3d9b6da654bf8bd7a49315b1625cf0e774ce. Reason for revert: Broke several builds. I'm acting build cop, reverting. Bug: 70351683 Change-Id: I775ada4e9cb6473519d51420b41b818af163da44
2018-02-07Support filegroup in exclude_srcs Nan Zhang
Test: add unit-test, m -j32 Bug: b/70351683 Change-Id: Iff83c56d45dd668d9df6131c7df2e23e5c73a21b
generated by cgit v1.2.3-59-g8ed1b (git 2.39.0) at 2025-10-30 08:53:44 +0000