| Age | Commit message (Collapse) | Author |
|---|
|
Due to upstream changes in python path calculations, I'm simplifying the
python launcher to work more like a standard python distribution. This
is effectively changing the stdlib path from `internal/stdlib` to
`internal/python3.10` (or `3.11`, etc). This allows us to specify the
zip file as PYTHONHOME, set PYTHONPLATLIBDIR to `internal` and use the
default detection after that.
That does mean during upgrades that the stdlib pkg path will change, so
move the source vs prebuilt calculation from the Android.bp into Soong
to choose which stdlib module to pick up (with the corresponding
`pkg_path`)
Bug: 278602456
Test: treehugger with python3.10
Test: a python3.11 source + 3.10 prebuilt build
Test: a python3.11 source+prebuilt build
Change-Id: I8b02e7b22a1f1d1e02819ae1a31a99cdc985542c
|
|
This signifigantly improves the startup time of soong-built
python binaries. For example, running
`m apexer && time out/host/linux-x86/bin/apexer` gives
0.734s before this cl, and 0.094s after.
Fixes: 259718110
Test: Presubmits
Change-Id: Ib19e83e2c60c39a849525be117279c318de3afa7
|
|
The new behavior has been enabled by default, and these
flags aren't necessary anymore.
Fixes: 245583294
Test: m py_dont_import_folder_of_entrypoint_test && /ssd/aosp-master/out/host/linux-x86/testcases/py_dont_import_folder_of_entrypoint_test/x86_64/py_dont_import_folder_of_entrypoint_test
Change-Id: I5b6f98da51791bc5d28662ef799a10c1bb6a35a0
|
|
The python interpreter will by default add the directory of
the entrypoint script to the beginning of sys.path. This
can be disabled in python 3.11+ (which is not released yet)
using the PYTHON_SAFE_PATH environment variable or the -P flag.
As a workaround to have this behavior in older python versions,
we can make an __soong_entrypoint_redirector__.py file at the
root of the zip file that is the entrypoint, and then that
file will redirect to the real entrypoint.
This brings non-embedded-launcher python modules closer to
the embedded launcher version. The embedded launcher binaries
already act like this because they start at an __main__.py file
at the root of the zip file.
Bug: 245583294
Test: m py_dont_import_folder_of_entrypoint_test && out/host/linux-x86/nativetest64/py_dont_import_folder_of_entrypoint_test/py_dont_import_folder_of_entrypoint_test
Change-Id: I39aaf04fb19c3ba7f5c9d98220872d6d08abf736
|
|
stub_template_host.txt added all the top-level modules to the
PYTHONPATH, which isn't correct, and caused absl.logging to
override the built-in logging module.
Removing this also makes it more consistent with python binaries
built with embedded_launcher: true. embedded_launcher: true
binaries don't add the top-level modules.
Fixes: 245583294
Test: m py_dont_add_top_level_dirs_test && out/host/linux-x86/testcases/py_dont_add_top_level_dirs_test/x86_64/py_dont_add_top_level_dirs_test
Change-Id: Id3069565d2b2c4b2bda0ff5301e757a7b4201751
|
|
Make sure the shebang of the par file is consistent with the main python
script rather than use /usr/bin/python arbitrarily.
Bug: 185399398
Bug: 185094069
Test: m atest ; atest-dev --help
m atest-py2 ; atest-py2-dev --help
Change-Id: Ice4590b3d173e6d11e608c2ca29b297df0dc4089
|
|
When multiple os/arch variants are supported, java_test_host could not
find a matching arch due to java having arch:common, whereas native
binaries support a specific architecture. This change adds the property
`data_native_bins` in order to support binaries with the appropriate
os/arch variants.
Test: m FirmwareDtboVerification with data_native_bins
Test: forrest
Bug: 153848038
Change-Id: I45adebff0fde2811d5ef5620c697b97b768c951f
|
|
android/soong/common was renamed to android/soong/android long
ago, but the pctx package path was still "android/soong/common".
This required all users of rules defined in android/soong/android
to import "android/soong/android" and then
pctx.Import("android/soong/common").
Test: m checkbuild
Change-Id: I20d096522760538f7cfc2bec3d4bfeba99b275d4
|
|
Mainly so that we can build a `python` prebuilt that acts like the
normal python install, though you could also build different ones with
more packages pre-installed.
Bug: 117811537
Test: move built py2-cmd into prebuilts/build-tools/path/linux-x86/python and build
Change-Id: I21215f6fd3754d89f8c65e1dfeb3f2deea23239f
|
|
Uses more python rather than C++, and skips less of Py_Main.
Test: build/soong/python/tests/runtests.sh
Change-Id: I03997d88e2e16047c96bb4e00e530229c42b3325
|
|
Soong doesn't wrap everything in bash, but relies on the ninja default
of /bin/sh. We should probably improve that in the future, but for now,
just fix the build.
On Mac, /bin/sh and /bin/bash are both bash, but /bin/sh implies `set -o
posix`, and ignores the `-n` flag.
Test: treehugger
Test: build on a mac
Change-Id: Icf41b1c5a1ce9eb0a56e39e68c433fc80b53620f
|
|
Instead of doing these string operations in the C++ code at every
startup of an embedded par, trim and convert the entry point file to a
python module path at build time.
Test: m
Change-Id: I04a6459a80c84a704b8a25a144ab1e19622d6cc3
|
|
The zip file format does support scripts/programs prefixed to the
archive, though many of the offsets are supposed to start from the
beginning of the file. Some tools (python and zipinfo) are able to
read zip files with arbitrary prefixes, but others (libziparchive and
zipdetails) fail.
So pass the file to prefix onto the zip file into merge_zips so that we
can set the offsets from the real start of the file.
Test: m sepolicy_tests (runs the embedded python interpreter)
Test: zipinfo out/host/linux-x86/bin/sepolicy_tests
Test: zipdetails out/host/linux-x86/bin/sepolicy_tests
Change-Id: If73d4c2465581f7de5aa47959284ecf2059df091
|
|
The root cause is we didn't check if the optionalpath is valid or not,
the registerbuildation function directly invoke the path var and
cause the invalid memory error. We just return if the launcher doesn't
exist.
The ctx.VisitDirectDepsWithTag() also handles allowmissingdependency so
we are ok if launcher doesn't exist.
Test: N/A
Bug: b/116698229, b/67510844
Change-Id: I40941079a64d7797ab879fc5edaa29e835b493a0
|
|
The problem came from Python libraries doesn't know the information that
we enabled embedded_launcher (only Python binary knows about that). And
we can't simply remove runfiles dir for Python libraries since host
Python mode need this.
Bug: b/80441699
Test: m perf_profo_flames
Change-Id: I73ffc4d7504f95a708ae7cca47bc6c15a673aa31
|
|
Python protobuf std libs will be wrapped in final binary/test par file.
Bug: b/70568913
Test: manually create real examples.
Change-Id: I7376ec9175f3e03d1adbd20858a7f74e826387ad
|
|
Each Python module will generate a zip file containing source & data
files. The Python binary will collect all its dependencies and use
merge_zips to merge each zip file to create a final .par file.
Test: m -j checkbuild && real examples:
Bug: b/70568913
Change-Id: I9ff232d461d33e1c06026e7dcb5b124bf02c3ce5
|
|
Now that android.ModuleContext does not include blueprint.ModuleContext
we can rename android.ModuleContext.ModuleBuild to
android.ModuleContext.Build without colliding with
blueprint.ModuleContext.Build. Leave ModuleBuild as a wrapper around
Build for now to avoid having to update all the users outside
build/soong simultaneously.
Test: m checkbuild
Change-Id: I18eb8cc04faf002049a11d9aac97e9732ff5d638
|
|
launcher.
For Python2, we bundle embedded launcher as bootstrapper within every
.par file. This feature is only enabled for linux_x86_64 for now. We
provide a user flag: hermetic_enabled within bp file. By default, Pyhon2
still use classic bootstrapping way to construct .par file and relys on
host interpreter. Once embedded_launcher is enabled, launcher will be
used to bootstrap .par file and execute user program.
For Python3, the launcher will be ready soon, and for now it still relys
on classic bootstrapping.
Test: Real example is used to test.
Bug: b/63018041
Change-Id: I28deba413d8ad3af407595e46f77d663e79a3705
|
|
Added params to WriteFile rule that allow users to turn on the
specific 'echo' options, such as "enable the interpretation of backslash
escapes".
Bug: b/38261000
Test: tested with real data files.
Change-Id: I5145056b5c2522a3a72b9436055ec95d2fbd94c0
|
|
Descriptions currently look like:
[ 0% 4/29328] cc out-soong/.intermediates/external/clang/lib/Sema/libclangSema/android_arm_armv7-a-neon_denver_static_core/obj/external/clang/lib/Sema/SemaCodeComplete.o
This is not very helpful - most of the characters are used to show the
output path, which contains useful information like target architecture,
but also contains most of the path to the source files twice, and less
useful information like the exact variant name used by soong.
Make the descriptions look like:
[ 0% 3/29329] //external/clang/lib/Sema:libclangSema clang++ SemaTemplateInstantiate.cpp
This is //path/to/module:modulename tool relative/path/to/source/file
Test: builds, looks pretty
Change-Id: I3087aa7d4eb1860ef6239d77407b8b35445616d7
|
|
The base module handles all the common functionalites, such as version
compatibilty check, version variations split, source file format check,
source/data file duplicate check.
The library/binary module focuses on how to generate binary build actions,
such as setting up stub script, zipping, filling in __init__.py in
runfiles dir tree.
Bug: b/31676493
Test: go test under python package
Change-Id: I06608369f350f7195873d459e1c8d1bdb811e77e
|