summaryrefslogtreecommitdiff
path: root/java/builder.go
AgeCommit message (Collapse)Author
2021-02-16Limit the number of the Java source files in a single compilation unit Sasha Smundak
KYTHE_JAVA_SOURCE_BATCH_SIZE environment variable controls this setting. The limit is 1000 if this variable is not set. Fixes: 179932118 Test: run prebuilts/build-tools/build-prebuilts.sh, use it to build kzips, check Change-Id: I9ad57dfd1d2c2dce5cff755b1bd61cf933420bd3
2021-02-08Turn missing jarjar output files into errors Colin Cross
Jarjar doesn't exit with a nonzero return code when there is a syntax error in a rules file and doesn't write the output file. Without a nonzero return code ninja prints a warning but continues, which leads to stale results on incremental builds where the output file already exists, or delayed errors on clean builds whre the output file didn't exist. Delete the output file before running jarjar, and then check if it exists after running jarjar. Fixes: 119516143 Test: m out/target/common/obj/APPS/TeleService_intermediates/classes.jar Change-Id: I7e9f7ff34de565a986ab0dc6e375f606c91c3846
2020-12-09Merge "Soong: remove output file before running SoongZip" Treehugger Robot
2020-11-25Soong: remove output file before running SoongZip Ulf Adams
It looks like there is a bug in rewrapper/reproxy that makes it upload the output file as an input if it still exists in the output tree. This causes unnecessary cache misses on incremental builds, and can also cause the action to fail remotely (depending on the service). By removing the output file at the beginning of the action, we ensure that it can't be uploaded. Test: ran against a remote execution server; actions no longer fail remotely Change-Id: I98fcc0cf105553a9920b0e96965a438e052d7c05
2020-11-14Replace android.WriteFile rule with android.WriteFileRule Colin Cross
The android.WriteFile rule takes careful escaping to produce the right contents. Wrap it in an android.WriteFileRule that handles the escaping. Test: compare all android.WriteFile outputs Change-Id: If71a5843af47a37ca61714e1a1ebb32d08536c31
2020-09-03Check UseRBE is set before replacing any template with the RE version. Ramy Medhat
Test: presubmit Change-Id: I6df58b4e700f0d231367af2710672d731d20a736
2020-07-01Output apkcerts file for android_app_set. Jaewoong Jung
Soong and Make have no ways to figure out what splits will be outputted from a given android_app_set, so it's impossible for them to provide full PACKAGES.$(LOCAL_MODULE).CERTIFICATE entries, which are required to build a final apkcerts.txt. This change makes extract_apks produce apkcerts.txt files for each input modules instead. The Make-side counterpart of this change merges all local apkcerts.txt into a final one. Fixes: 160119159 Test: main_test.go Test: m apkcerts-list Merged-In: I321e80fd636a955213761f56a3ac64bfe7f7f7c0 Change-Id: I321e80fd636a955213761f56a3ac64bfe7f7f7c0
2020-06-03Revert "Always run package check" Colin Cross
This reverts commit 91206d9ed1a1e3cb81d1ace021580c4be8362f1f. Reason for revert: package-check.sh doesn't work on mac Bug: 158081251 Bug: 157649935 Change-Id: I50af71d54d5f1f8b516bfcf1efbcf6217e89c83a
2020-06-02Always run package check Paul Duffin
Previously, the package check was only run when building checkbuild or the phony target created for a specific module. It would not run when building a module that depended upon a library with the permitted_packages property. That was because the package check was only run when the package-check.stamp file was built and that was only added to the phony and checkbuild targets. Instead of touching a separate package-check.stamp file to indicate that the check has been performed this change copies the input jar file to the package check to a separate ../package-check/<jar> file which is then treated as the output of the library and is the input for any dex processing. So, any modules that depend on this library will transitively depend on the output file produced by the package check command and so will ensure that the package check is always run. Test: Removed "android.net" from the permitted_packages for "framework-tethering", build "framework-tethering" which triggered and failed the package check and "com.android.tethering" which did not. Made this change. Built "com.android.tethering" which triggered and then failed the package check. Removed change to "framework-tethering" Bug: 157649935 Change-Id: Ib01aa09e13f80282218049270eb7a58ec5f9f605
2020-05-28Add support for remote-execution / caching of jar/zip actions Kousik Kumar
Test: Ran a build with `RBE_JAR="true" RBE_JAR_EXEC_STRATEGY="remote" RBE_ZIP="true" RBE_ZIP_EXEC_STRATEGY="remote" ... use_rbe m` and that succeeded. Bug: b/156765207 Change-Id: Ia17b532fbb282be414adf879914870082dd0841b
2020-05-20Add support for remote-execution / caching of turbine actions Kousik Kumar
Test: Ran a sample turbine action with: RBE_TURBINE_EXEC_STRATEGY="remote" RBE_TURBINE="true" ... use_rbe m out/soong/.intermediates/external/protobuf/libprotobuf-java-micro/android_common/turbine/libprotobuf-java-micro.jar and checked in reproxy_log.txt that remote-metadata had status SUCCESS. Bug: b/156765475 Change-Id: I85c334e6998c6dd2a68460b3580d50fffd900e6f
2020-05-14Implement android_app_set module Sasha Smundak
Bug: 152319766 Test: manual and builtin Change-Id: Id0877476f9ae23311d92c0b59a9c568140ab4119
2020-04-28Refactor RBE support for Javac/R8/D8 to use the remoteexec package. Ramy Medhat
Test: build with and without RBE_JAVAC/RBE_R8/RBE_D8 set. Change-Id: I1607f8cfb4d2c3cbb3672152bbeb561d9968bc60
2020-04-24Merge "Repeat kapt processor argument for multiple processors" Treehugger Robot
2020-04-22Repeat kapt processor argument for multiple processors Colin Cross
kapt claims to support a comma separated list of annotation processors, but it errors if multiple annotation processors are given. Surrounding the the list with {} does not error, but it also doesn't even warn if the second element in the list is garbage, so it may not be running the second processor. Repeat the processor argument for each annotation processor class instead. Bug: 154736649 Test: TestKapt Test: m checkbuild Change-Id: I4c7c161dbf867d7fba1aaf16fd5e502647e3f682
2020-04-20Remove Android compat annotations on jarjar transform. Artur Satayev
Set a system property to enable Android specific transformer that drops UnsupportedAppUsage annotation for repackaged classes to avoid adding unneeded hidden api symbols on jarjar. Bug: 146418363 Test: m, manual diff hiddenapi-flags.csv Exempt-From-Owner-Approval: patch approved by owner Change-Id: I10b49feb15257301574162b5d389cbc30277f672
2020-03-05Disable goma for javac actions in Soong Colin Cross
goma can't handle the --system argument that Android must pass to javac. Bug: b/143658984 Test: treehugger Change-Id: I207668d8baa4ab28160938732a38f696aeac3f44
2020-01-28Add support for experimentally enabling RBE support on specific rules. Ramy Medhat
This CL adds RBE support to javac, r8, and d8 rules which is only enabled if respective environment variables are set. Test: an aosp_crosshatch build with and without the new variables. Change-Id: Ic82f3627944f6a5ee7b9f3228170c2709b1bfcb8
2020-01-15Save compilation units in protobuf format. Sasha Smundak
Bug: 146224091 Test: manual Change-Id: I7a856bab13f54c78efa0061421c3fcb9341bc6e7
2019-11-07Fix turbine classpath arguments Colin Cross
We were passing "--classpath foo.jar --classpath bar.jar" to turbine, it now wants (and may always have expected?) "--classpath foo.jar bar.jar". Test: m checkbuild Change-Id: Ib7e0ca64a34e236110f7b785d6a0fb51ed75567e
2019-11-06Merge "Run non-RBE supported actions in the local pool when USE_RBE is set." Ramy Medhat
2019-11-05Run non-RBE supported actions in the local pool when USE_RBE is set. Ramy Medhat
Bug: 143938974 Test: ran CTS build at -j500 successfully. Change-Id: I55074bd67308cd716972e24fb56a20bc393d5d9d
2019-11-04Make the changes to emitXrefRule matching those in aosp/1145829 Sasha Smundak
Test: build kzip Bug: 121267023 Change-Id: Id3a70d113dd7c24071cbb7cbfbf138527991cb77
2019-10-29Use java language 1.9 for sdk_version: "current" Colin Cross
Java language was set for 1.8 for anything building against the current SDK because the stubs were not built in Soong, so the system modules could not be built. The stubs have been built in Soong since Iabd32b30954b3f4a6d9a779fde52a032b684807e, but I5e870c34dd0ebc8ae3f888ec627da590c846a76f missed updating this TODO. Use 1.9 when building against the stubs, but continue using 1.8 for unbundled builds until we have prebuilt system modules. Always use the core-current-stubs-system-modules to avoid splitting android.* packages between the system modules and the classpath, which would cause new classes in android.* packages in classpath jars to be ignored. Add a new java9Classpath field to sdkDep that will contain the stubs jar when targeting Java language level 1.9, and plumb it through to javac and turbine. Rename the modules field to bootclasspath. Bug: 142896162 Test: m checkbuild Change-Id: Icfd32d0a863b2303a997c7cf03cb3708aade4724
2019-10-29Use system modules for turbine Colin Cross
turbine supports taking system modules on the command line, now that we target Java language level 1.9 and use system modules by default switch turbine to match javac. Test: m javac-check Change-Id: Ieee07502151da0d5693bb8929213d495c039106b
2019-10-28Make javaVersion an enum Colin Cross
Remove the hardcoded checks against "1.9" by making javaVersion an enum and implementing javaVersion.usesJavaModules(). Test: TestClasspath Change-Id: I559eeb1f45880bb8177269c6d977ee4dfbadce57
2019-09-27Pass filename mappings to C++ and Java extractors. Sasha Smundak
Android builds by default put artifacts into out/ subdirectory of the source tree, causing the extractor to record their names as relative. The indexer considers such files as sources, which is wrong. Fortunately, the extractor can be fed a set of filename rewriting rules (see build/tools/vnames.json). Also, undo previous unsuccessful attempt use to absolute path for the output directory to distinguish between source code and artifacts. Bug: 141385476 Test: run the build, inspect compilation units of the kzip file Change-Id: I89ec3aed8fd14f43ea6e0b226d54f643346f6125
2019-09-05Shard kythe invocations when javac is sharded Colin Cross
The kythe pipeline has trouble injesting all 6000 files of framework.jar at once. Shared invocations of the kythe extractor when javac is sharded to produce multiple smaller kzip files. Bug: 140426870 Test: no change to build.ninja Test: m out/soong/.intermediates/frameworks/base/framework/android_common/framework0.kzip Test: TestSharding in java_test.go Change-Id: I867db4ef5cb1e7f3ce8359a46aac2c00ed8a8912
2019-07-26Support source code cross-referencing for C++ and Java Sasha Smundak
Use Kythe (https://kythe.io) to build cross reference for the Android source code. ~generate the input for it during the build. This is done on demand: if XREF_CORPUS environment variable is set, build emits a Ninja rule to generate Kythe input for each compilation rule. It also emits two consolidation rules (`xref_cxx` and `xref_java`), that depend on all Kythe input generation rules for C++ and Java. The value of the XREF_CORPUS environment variable is recorded in the generated files and thus passed to Kythe. For the AOSP master branch it is `android.googlesource.com/platform/superproject`, so the command to build all input for Kythe on that branch is: ``` XREF_CORPUS=android.googlesource.com/platform/superproject m xref_cxx xref_java ``` Each Kythe input generation rule generates a single file with .kzip extension. Individual .kzip files have a lot of common information, so there will be a post-build consolidation step run to combine them. The consolidated .kzip file is then passed to Kythe backend. The tools to generate .kzip files are provided by Kythe (it calls them 'extractors'). We are going to build them in toolbuilding branches (clang-tools and build-tools) and check them in as binaries into master and other PDK branches: For C++, `prebuilts/clang-tools/linux-x86/bin/cxx_extractor` for Java, `prebuilts/build-tools/common/framework/javac_extractor.jar` Bug: 121267023 Test: 1) When XREF_CORPUS is set, build generates Ninja rules to create .kzip files; 2) When XREF_CORPUS is set, building `xref_cxx`/`xref_java` creates .kzip files; 3) Unless XREF_CORPUS is set, build generates the same Ninja rules as before Change-Id: If957b35d7abc82dbfbb3665980e7c34afe7c789e
2019-07-18Add a systemModules utility type Colin Cross
Storing system modules in a classpath is clumsy, as there should only ever be one system modules, and it needs to store both a directory to pass as the argument and a set of generated files to use as dependencies. Store them in a separate systemModules type instead. Test: m checkbuild Change-Id: I020556c736bd5091865bcca51dc0fb9e4db6b45b
2019-07-18Convert droidstubs to android.RuleBuilder Colin Cross
The droiddoc rules are ripe for converting to android.RuleBuilder, they conditionally use many input files which is hard to track with the standard blueprint.Rule style, as the argument and the dependency have to be handled independently. Start converting to android.RuleBuilder by converting the droidstubs module. Test: m docs Test: m checkapi Test: m updateapi Change-Id: I08713e91149471e88a40115f69824cf5eaf88fb6
2019-06-18Depend on all the files from system modules Dan Willemsen
Instead of just one of the files that we pass into javac. Test: treehugger Change-Id: I8478e88656487c9f667893d7c17839f0ea63c78f
2019-06-11Tune JVM, print full diagnostics on crash. Sasha Smundak
JVM occasionally (0.5% time) crashes on entering a method with SIGSEGV. It might be due to excessive load on the host machine. Reduce the number of the compiler threads to 6, and do not start all the GC threads immediately. If JVM crashes, copy full diagnostics to stderr. Bug: 132766811 Test: treehugger Change-Id: Ib7bfd568f6d32456bf623cbb615ffc990c326a34
2019-06-05Revert "Revert "Revert "Print full diagnostics file when Java process ↵ Jimmy Chen
crashes.""" This reverts commit b4be09285522c0b55d9d7a2a64984d31cde57720. Reason for revert: break build Change-Id: I11fc5eb5f635a5d99eba962f5817851629e50ab2
2019-06-03Revert "Revert "Print full diagnostics file when Java process crashes."" Alexander Smundak
This reverts commit d71dc6b5e0b01e4835a033bb51c278a10b0c1481. Reason for revert: ag/974065 fixes the problem that caused the revert Change-Id: I3ba9eecdab34602aaf19fa518520ac932c9b97af
2019-05-31Revert "Print full diagnostics file when Java process crashes." Alexander Smundak
This reverts commit 9d6df2246c076142428b10c08f614b6ea615b095. Reason for revert: This change set the value of JAVA make variable to a string containing "foo bar". When soong_ui is asked to evaluate such string, it drops the quotes, causing bash to treat 'foo' and 'bar' as separate command line arguments. Change-Id: If4e3c4c950c305bb67a572de3d86119c2da81dc1
2019-05-29Print full diagnostics file when Java process crashes. Sasha Smundak
Bug: 132766811 Test: treehugger Change-Id: I8cca2486f2194a8643007a53e4a695d2a18abbf8
2019-04-18Build framework.aidl in Soong Colin Cross
Move the rules to build framework.aidl into Soong, and use it when compiling aidl files with sdk_version: "current". Also fixes incorrectly using the aidl includes exported by the "framework" module when the proguardRaiseDep dependency was added. Bug: 130798034 Test: sdk_test.go Change-Id: I126adf1d9e7b6acb528875ff62b974ba7ad9a337
2019-04-08Check package restrictions for Java libs. Vladimir Marko
Test: m checkbuild; inspect verbose log. Test: Manual - compile with unmet restrictions. Bug: 122937705 Change-Id: I9360ae8b6d9ce016b7827be5e8ffc6eb521809b7
2019-04-04Fix package path of android/soong/android pctx Colin Cross
android/soong/common was renamed to android/soong/android long ago, but the pctx package path was still "android/soong/common". This required all users of rules defined in android/soong/android to import "android/soong/android" and then pctx.Import("android/soong/common"). Test: m checkbuild Change-Id: I20d096522760538f7cfc2bec3d4bfeba99b275d4
2019-04-02Move proto compilation to RuleBuilder Colin Cross
Using blueprint.Rule for protoc commands was causing code duplication because there was no good way to run the same protoc for cc, java and python but then run custom source packaging steps for java and python. Move most of the code into a common function that returns a RuleBuilder, and then let java and python add their own commands at the end of the rule. Bug: 70706119 Test: All Soong tests Test: m checkbuild Change-Id: Ic692136775d273bcc4f4de99620ab4878667c83a
2019-03-04Replace *Escape with *EscapeList Colin Cross
Follow the change to blueprint to make *Escape take and return a string and add *EscapeList that take and return slices of strings. Fix up a few places that were unnecessarily converting a string to a slice and back to a string. Test: m nothing Change-Id: I3fa87de175522205f36544ef76aa2f04aef1b936
2019-02-08Make manifest and APK agree on uncompressed native libs Colin Cross
Only put uncompressed native libs in an APK if the min_sdk_version supports it (>= 23, Marshmallow), and set android:extractNativeLibs="false" in the AndroidManifest.xml so that the platform won't extract them anyways. Bug: 117618214 Test: m checkbuild Change-Id: I760017e48bf3c6b618aabde0982df45995765d48
2019-01-25Clean up srcjar directories after each rule Colin Cross
The srcjar directories just confuse people looking for generated files, clean them up after each rule. Test: m checkbuild Change-Id: If712ce7d4922619fd8420ae8f89830fe7824114d
2019-01-24Remove annotation_processors property Colin Cross
Now that there are no uses left of annotation_processors remove the property. If there are no annotation processor classes for javac for a module, due to an empty "plugins" property or using kapt for annotation processors, pass -proc:none to javac to ensure it does not try to run any annotation processors found in the classpath. Bug: 77284273 Test: plugin_test.go, kotlin_test.go Change-Id: I3823d9fec8d3d07d2e49b1d97839f7fcbdd35647
2019-01-23Replace annotation_processors with plugins Colin Cross
Follow bazel's modules for annotation processors by introducing a java_plugin module type that can contain extra metadata about the annotation processor, the processor class and a flag to specify if the annotation processor is compatible with the turbine optimization. Deprecate the annotation_processors property, which took a list of java_library_host modules, in favor of the plugins property, which takes a list of java_plugin modules. The annotation_processors property will be removed once all uses have been replaced with plugins. Bug: 77284273 Test: plugin_test.go Test: m caliper Change-Id: I37c1e80eba71ae2d6a06199fb102194a51994989
2019-01-22Pass annotation processors to kotlinc Colin Cross
Enable the kotlin-annotation-processing plugin and pass annotation processors to it. Bug: 122251693 Test: m checkbuild Test: TestKapt in kotlin_test.go Change-Id: I841df454beaaa7edd263eea714ca0d958a03c9de
2019-01-22Move kotlin to it's own file Colin Cross
kapt is going to make kotlin a little more complicated, move the rules and tests to their own files. Bug: 122251693 Test: m checkbuild Change-Id: Ieed78b97995ced210b710bd50c357514cc8e3bc6
2018-12-15Dexpreopt soong modules inside soong Colin Cross
Port the dexpreopt logic from Make to the dexpreopt package in Soong, and use it to dexpreopt Soong modules. The same package is also compiled into the dexpreopt_gen binary to generate dexpreopt scripts for Make modules. This relands Ib67e2febf9ed921f06e8a86b9ec945c80dff35eb and I462182638bd57b1367b5bfb0718e975c11ae66f7, along with multiple fixes to depsfile generation in dexpreopt_gen that caused .odex files for modules in defined make to be missing dependencies on boot.art, and a fix to not dexpreopt and strip tests. Bug: 119412419 Bug: 120273280 Test: no differences to dexpreopt outputs on aosp_sailfish system/, only expected changes to dexpreopt outputs on system_other (.vdex files for privileged Soong modules no longer incorrectly contain .dex contents). Test: OUT_DIR=$PWD/out m Test: NINJA_ARGS="-t deps out/target/product/sailfish/obj/APPS/Contacts_intermediates/dexpreopt.zip" m Change-Id: I6bb2c971cee65d2338839753aa0d84939f335b1b
2018-12-14Revert "Dexpreopt soong modules inside soong" Colin Cross
This reverts commit 29ff88741e710b05743dcf347484c31311e81cda. Test: none Bug: 119412419