summaryrefslogtreecommitdiff
path: root/sdk/sdk.go
AgeCommit message (Collapse)Author
2020-10-02Add prebuilt_visibility property Paul Duffin
Some mainline modules provide an `sdk` and one or more set of module_exports. When the snapshots are unpacked into separate prebuilts directories (one per snapshot) then any dependencies from a member of one snapshot to a member of another may require the latter to have additional visibility rules. Previously, these rules had to be added to each source module. This change allows additional visibility rules to be specified on the sdk/module_exports that are added to all the prebuilts in its snapshot. Bug: 155921753 Bug: 168301990 Test: m nothing Change-Id: Ia3eacb5749981f04770ae9872a8013e43c5c6ef1
2020-07-24Avoid creating APEX variant for sdk member Paul Duffin
Previously, an APEX variant was created for a module that was a member of an SDK just in case it had to be replaced with an APEX requested snapshotted version of that member. That was necessary because that was the only way to have APEX specific replacements. Since then a new method called ReplaceDependenciesIf() has been added which provides fine grained control over which dependencies are replaced. This change uses that new method to handle the replacements which allows the APEX variants to be removed. Bug: 161928524 Test: m nothing Change-Id: If3869dd6753dc182b099af566b20fbc9c9c6eff7
2020-07-07Prepend default to allow overriding in the bp file. Martin Stjernholm
Bug: 143948100 Bug: 151303681 Test: m nothing Change-Id: Iefcf4fbe9d2971ce267242185165f0c38f68db30
2020-07-07Do not follow SDK member dependencies for APEX payloads. Martin Stjernholm
Test: m nothing Test: `m` with prebuilts/runtime in the manifest (along with other fixes) Bug: 151303681 Change-Id: I450d476975c7ab4434228b8c4baf3af192142211
2020-06-25Support multiple dists per Android.bp module, and dist output selection. Jingwen Chen
This CL adds "dists" to the base property struct to support multiple dist file configurations, and generic tag support to dist tagged outputs of modules. Fixes: b/152834186 Test: soong tests and `m sdk dist` Change-Id: I80c86bc9b7b09e671f640a4480c45d438bdd9a2a Signed-off-by: Jingwen Chen <jingwen@google.com>
2020-04-08Stop requiring apex_available on java_library members of sdks Paul Duffin
Previously, adding java_library to an sdk required that the names of any APEXes that transitively compiled against it were added to its apex_available property. This change removes that requirement. Also corrects the dependency path in the TestApexAvailable_IndirectDep error which previously passed through "shared from static" static dependency tags even though those are explicitly NOT followed when checking apex_available settings. Bug: 152878661 Test: m droid Change-Id: I995ed38956c1bc210b09494812de012fed9f9232
2020-04-07Extract DepIsInSameApex and RequiredSdks interfaces Paul Duffin
The DepIsInSameApex() and RequiredSdks() methods were defined in a few places to avoid having to depend on the whole ApexModule/SdkAware interfaces directly. However, that has a couple of issues: 1) It duplicates functionality making it difficult to change, changes to the definitions outside the main interfaces do not cause compile time failures, instead they result in a runtime change in behavior which can be difficult to debug. 2) IDE navigation (specifically in Intellij) does not detect that the duplicate definitions can resolve to the definitions in the main interface. This change extracts the methods into their own interfaces and reuses those interfaces instead of duplicating the methods to fix both of these issues. Bug: 152878661 Test: m nothing Change-Id: I0cfdf342a14eb0bfb82b1bd17e0633d81c7facfb
2020-03-23Sdk snapshot set compile_multilib per OsType Paul Duffin
Previously, when an sdk snapshot only supported a single os type the compile_multilib was set based on the multilib usages by the members of that variant. After the change to support multiple os types per snapshot the multilib setting was based on the multilib usages across all the members of all sdk variants. That meant that if one os type used only "64" and the other used "both" then they would both be treated as "both" leading to missing variants when the snapshot was unpacked. This change tracks the multilib usages per os type and adds a property for each one. It intentionally changes a couple of tests: 1) Either by adding compile_multilib that is missing. 2) By targeting it at a specific os type instead of host. The latter change is important to prevent the snapshot from being used on a host platform (which will match the host target section) but which is a different os type to the ones supported by the snapshot. Bug: 142935992 Test: m nothing Change-Id: I883919b644292c3d019db223bb4fd5c11b39591f
2020-03-13Copy shared_libs and system_shared_libs to module snapshot Paul Duffin
This change ensures that the runtime dependencies between a binary/shared library are correctly specified in the snapshot so that the build can ensure that shared libraries are built before the targets that use them. It adds support for differentiating between references that are required to refer to another sdk member (required) and those that may refer to either an sdk member or a non-sdk member (optional). The latter is used for shared library references as the libraries used by an sdk member may be provided from outside the sdk. e.g. liblog is not part of the ART module but is used by some members of the ART sdk. Bug: 142935992 Test: m nothing Change-Id: Ia8509ffe79b208c23beba1880fe9c8a92b732685
2020-03-10Allow sdk members to vary by os type Paul Duffin
Adds support for specifying separate members to an sdk/module_exports for different os types, e.g. separate ones for android and host. Adds 'android:"arch_variant"' tag to the dynamically generated fields for the sdk member types. Merges the exported members from all variants together. Determines the device/host_supported flags of the member snapshots by whether the OsClasses used by their variants rather than the sdk's host/device supported properties as they may be different. Bug: 150451422 Test: m nothing Change-Id: I41fbbcd8723aafd54826aad9b78eced9f66ef51c
2020-03-04Add a nice install paths for module SDKs and exports. Nicolas Geoffray
Following how NDK also creates its own install path. Bug: 142935992 Test: cc_sdk_test.go, java_sdk_test.go Change-Id: I98a3656903f37f6d7c90e6cf609431b2461a6161
2020-03-02Add CommonOS variant for sdk Paul Duffin
Adds a CommonOS variant for sdk that depends on the os specific variants and is used to generate a single sdk for multiple OsTypes, e.g. host linux and android. At the minute the member types only support a single OsType but the basic mechanism for managing the CommonOS variant and collating the variants across all of them is there. The only visible effect of this change is that the location of the generated snapshot is changed, it is no longer os specific and instead is in the same location irrespective of which os it is built for. A lot of tests needed to be changed to specify "common_os" as the variant type instead of the specific os type. As that is the same across all tests it is hard coded in CheckSnapshot method. Test: m nothing Bug: 150451422 Change-Id: If36be39b06d6910453649f7c288c2d34f688b2f4
2020-02-28Avoid invoking sdk member to add empty list of dependencies Paul Duffin
Simplifies debugging sdk membership code when it is only called if an appropriate member is present. Test: m nothing Change-Id: I32bc93e0484c829bdbd6b050822249feb6404f89
2020-02-20Do not add dependencies from disabled sdk variants to its members Paul Duffin
If an sdk variant is disabled (e.g. say windows) then it should not add dependencies on corresponding sdk member variants as if those variants have not been created then the build breaks unnecessarily. This is needed to make progress on unbundling art which provides at least one host tool (dex2oat) that is not supported on all platforms and some (hiddenapi) that only produce 64 bit versions. Initially, the art prebuilts will only support building 64 bit versions of linux host tools. Bug: 142935992 Test: m art-module-sdk art-module-host-exports art-module-test-exports Change-Id: Idfbb40bb3cabc6eb731a2b38f045ed14f0a713cc
2020-02-07Differentiate between exported and internal sdk members Paul Duffin
Internal sdk members are used by an sdk member but not exported by the sdk. The exported sdk members are those listed explicitly in one of the sdk member list properties, e.g. java_header_libs. The prebuilts of an internal sdk member use a unique name so that they do not clash with the source module. The use of the module internally is an implementation detail that must not have any effect outside the snapshot. Having the same name as the source module could cause it to override the source module, hence why it needs a unique name. Similarly, they are marked as private so as to prevent their accidental use from outside the snapshot. Bug: 142940300 Test: m nothing Change-Id: Id5364b410be0592f65666afb3e40e9d3f020251c
2020-01-30Make sdkMemberDependencyTag usable outside sdk package Paul Duffin
Moves the struct to android/sdk.go and abstracts it behind a factory method and interface. That allows it to be used outside the sdk package. This change is in preparation for adding support for module types that have transitive sdk members. Bug: 142940300 Test: m nothing Change-Id: I71e5e0adf839b28a3a0952f82637637887f02688
2020-01-15Exclude sdkMemberVersionedDepTag from visibility enforcement Paul Duffin
The implicit dependency from an unversioned sdk member to the snapshot versions should be excluded from visibility enforcement as it provides no benefit and would require snapshot members to be visible to their original source modules which would complicate visibility handling. Also, corrects a spelling mistake in sdkMemberVersionedDepTag. Bug: 142940300 Test: m nothing Change-Id: Ib7ea6a3d3904d042dca0ea12f6b9196d40de970b
2019-12-31Restrict SdkMemberTypes that can be used with sdk/sdk_snapshot Paul Duffin
By default SdkMemberTypes are only supported on module_exports module type. Support for sdk module type has to be explicitly specified. The java_header_libs, native_shared_libs and stubs_sources are supported on sdk. The latter is required to provide the stubs source for an API specified in java_header_libs as they should be kept in sync. Bug: 146341462 Test: m nothing Change-Id: I19b9e60792780a797458d4a9e489506602b13144
2019-12-30Added module_exports/_snapshot as alias for sdk/_snapshot Paul Duffin
Bug: 146341462 Test: m nothing Change-Id: I27e1ef494a2b0874074aa43614612189b17e7860
2019-12-13Decouple addition of new sdk member types from sdk code Paul Duffin
Previously, adding a new SdkMemberType would require adding a new sdkMemberListProperty instance to the sdkMemberListProperties as well as adding a new property into the sdkProperties struct. They are potential sources of conflict and couple the sdk code with all the packages that add members to it. This change switched to a registration model that allows each package to register its sdk member types decoupling them from the sdk code. Adds an SdkPropertyName() method to SdkMemberType that specifies the name of the property to use in the sdk/sdk_snapshot. Also provides an SdkMemberTypeBase struct to be used by providers of SdkMemberType implementations. SdkMemberType instances are registered using the RegisterSdkMemberType() func which sorts the registered instances by their SdkPropertyName() to ensure the behavior is consistent and not affected by order of registration. When creating a new sdk module a dynamicSdkMemberTypes instance is created that contains the following: * A properties struct is created dynamically that contains a field for each registered SdkMemberType, corresponding to that type's SdkPropertyName(). * A list of sdkMemberListProperty instances is also created, one for each registered SdkMemberType. The dynamicSdkMemberTypes instance is cached using a key that uniquely identifies the set of registered types just in case new types are registered after one has been created, e.g. by tests. Bug: 142918168 Test: m checkbuild Change-Id: I4bf2bf56a2a49025aa41454048bc1e8ccc6baca2
2019-12-12Added support for using static libraries in sdk snapshot Paul Duffin
Parameterized the cc.librarySdkMemberType to allow it to support both static and shared libraries. Created two instances, one for shared and one for static libraries. A follow up change will add support for libraries that can be both. Added *librarySdkMemberType to nativeMemberInfo as information from there is needed when generating the snapshot. Made organizeVariants() func a method of *librarySdkMemberType so that it can initialize the new field. Moved it to be with all the other methods of that type. Added host and device tests for the new module type. Bug: 142918168 Test: m nothing Change-Id: I00b1e8424b9d81f7d15edc4883971d10668ec2cc
2019-12-12Sort sdkProperties by package and then name Paul Duffin
Bug: 143678475 Test: m nothing Change-Id: Ib610b799e7d86b413f4321239535a1caf4775ed9
2019-12-11AndroidMkEntries() returns multiple AndroidMkEntries structs Jiyong Park
AndroidMkEntries now returns multiple AndroidMkEntires so that a module can emit multiple Make modules if needed. Bug: 128708192 Test: m Change-Id: I56b6f76d22943b80329951c5acb80a1b932441ad
2019-12-06Support header and implementation jars in sdk Paul Duffin
Add a new java_header_libs property that exports header jars (as java_libs currently does) and switch java_libs to export implementation jars instead. Refactors implementation of the existing library sdk member type so both properties can be supported from common code as they differ only on the jar being exported. Bug: 143678475 Test: m nothing Change-Id: I04642122f72d083bbdfd3290624f957b71ee8875
2019-12-06Organize sdk member properties Paul Duffin
Grouping in alphabetical order by package and then by name within the package should minimize conflicts when making changes. Bug: 143678475 Test: m conscrypt-module-sdk Change-Id: Ia7dbcd41ce8b8dd8675a90b1b6868fcaeaf72ee4
2019-12-06Parameterize the sdk member processing Paul Duffin
Extracts the type specific functionality into the SdkMemberType interface which has to be implemented by each module type that can be added as a member of the sdk. It provides functionality to add the required dependencies for the module type, check to see if a resolved module is the correct instance and build the snapshot. The latter was previously part of SdkAware but was moved because it has to be able to process multiple SdkAware variants so delegating it to a single instance did not make sense. The custom code for handling each member type specific property, e.g. java_libs, has been replaced with common code that processes a list of sdkMemberListProperty struct which associates the property (name and getter) with the SdkMemberType and a special DependencyTag which is passed to the SdkMemberType when it has to add dependencies. The DependencyTag contains a reference to the appropriate sdkMemberListProperty which allows the resolved dependencies to be grouped by type. Previously, the dependency collection methods would ignore a module if it was an unsupported type because they did not have a way of determining which property it was initially listed in. That meant it was possible to add say a droidstubs module to the java_libs property (and because they had the same variants) it would work as if it was added to the stubs_sources property. Or alternatively, a module of an unsupported type could be added to any property and it would just be ignored. However, the DependencyTag provides information about which property a resolved module was referenced in and so it can detect when the resolved module is of the wrong type and report an error. That check identified a bug in one of the tests where the sdk referenced a java_import module (which is not allowed in an sdk) instead of a java_library module (which is allowed). That test was fixed as part of this. A list of sdkMemberListProperty structs defines the member properties supported by the sdk and are processed in order to ensure consistent behaviour. The resolved dependencies are grouped by type and each group is then processed in defined order. Within each type dependencies are grouped by name and encapsulated behind an SdkMember interface which includes the name and the list of variants. The Droidstubs and java.Library types can only support one variant and will fail if given more. The processing for the native_shared_libs property has been moved into the cc/library.go file so the sdk package code should now have no type specific information in it apart from what is if the list of sdkMemberListProperty structs. Bug: 143678475 Test: m conscrypt-module-sdk Change-Id: I10203594d33dbf53441f655aff124f9ab3538d87
2019-12-06Use static build rules in snapshot generation Paul Duffin
It is easier to extract information out of static build rules than it is out of custom build rules built using the builder as the former provides access to the in/out and args separate from the rule whereas the latter only provides access to in/out. Also, cleans up some warnings that appear in Intellij. There is a lot of duplication in the testing code. That will be resolved in a follow up change. Bug: 143678475 Test: m conscrypt-module-sdk Change-Id: I973bc0c90b0affd84487f1b222dd3e6c22c07ec0
2019-11-28Remove unused frozenVersions() method Paul Duffin
Bug: 143678475 Test: m nothing Change-Id: Ia5d417cfccfcd90b02a95b5b806b211b97e340db
2019-11-26Check the contents of an SDK snapshot's generated Android.bp Paul Duffin
Test: m nothing Bug: 143678475 Change-Id: I407d83c79d6b1ad8082e560726f0bfa7cacab3f0
2019-11-25Merge changes I0dcc9c7b,I9bc40642 Colin Cross
* changes: Move cc.imageMutator into the android package Make CreateVariations return []android.Module
2019-11-22Adds droidstubs support to sdk module Paul Duffin
Adds stubs_sources property to sdk and unzips the droidstubs srcjar into the snapshot directory. Adds an UnzipToSnapshot method to the SnapshotBuilder which creates a rule that uses zip2zip to repackage the supplied zip content into a temporary zip file that matches what the required snapshot structure. e.g. if the supplied zip contains foo/Foo.java and that needs to be in the snapshot directory java/foo/stubs then it will create a zip that contains java/foo/stubs/foo/Foo.java. The temporary zip that is the output of that rule is added to the zipsToMerge field for merging later. If the zipsToMerge is empty then the snapshot zip is created as before. Otherwise, a temporary zip file is created. That is then merged with the other zip files in zipsToMerge to create the final snapshot zip. Adds prebuilt_stubs_sources for use by the generated .bp module. Bug: 143678475 Test: added conscrypt sdk module and attempted to build it Change-Id: Ie274263af3a08e36a73c61c0dbf0c341fd6967e2
2019-11-22Simplify building an SDK snapshot from the command line Paul Duffin
Adds a phony target for each sdk module that builds the snapshot zip. Test: built an sdk module from command line and checked the zip was created. Bug: 143678475 Change-Id: I4599332443b8da9adea0a16f00f569ffbd421602
2019-11-20Move cc.imageMutator into the android package Colin Cross
Prepare for making the image mutator available to all modules and moving it between the os and arch mutators by moving it into the android package and using an interface implemented by the module types to control it. Bug: 142286466 Test: No unexpected changes to out/soong/build.ninja Change-Id: I0dcc9c7b5ec80edffade340c367f6ae4da34151b
2019-11-07SDK snapshot is dist'ed Jiyong Park
`m module_sdk dist` produces snapshots of all SDKs in the source tree. A snapshot is a zip file consists of Android.bp, exported headers, exported AIDL files, stubs for native libs and jars. The zip file is expected to be downloaded from the build server and extracted to a directory (which probably will be /prebuilts/module_sdks/<module_name>/current). Bug: 138182343 Test: m (sdk_test.go updated) Change-Id: Idbe4bc24795fe08f26fc1cf7497028f9d162053a
2019-11-06sdk modules are by default compile_multilib: "both" Jiyong Park
Bug: 143948100 Test: m (sdk_test.go amended) Change-Id: I7df1b96af49a6569012e44eeb3c0722fac63fa51
2019-10-29native shared libs in an SDK can be snapshotted Jiyong Park
The snapshot script can now handle native shared libs in an SDK. Bug: 138182343 Test: create following sdk module: sdk { name: "mysdk", native_shared_libs: ["libc", "libdl"], } , then execute `m mysdk` and execute the update_prebuilt-1.sh as prompted. Following directories are generated under the directory where mysdk is defined at: 1 ├── aidl ├── Android.bp ├── arm64 │   ├── include │   ├── include_gen │   └── lib │   ├── libc.so │   └── libdl.so ├── include │   └── bionic │   └── libc │   └── include │   ├── alloca.h │   ├── android │   │   ├── api-level.h <omitted> Change-Id: Ia1dcc5564c1cd17c6ccf441d06d5995af55db9ee
2019-10-18Merge "Prohibit dependencies outside of uses_sdks" Treehugger Robot
2019-10-17Merge "Add method to determine variations from a Target" Colin Cross
2019-10-17Prohibit dependencies outside of uses_sdks Jiyong Park
When an APEX is built with uses_sdks, any depedndency from the APEX to the outside of the APEX should be from the SDKs that the APEX is built against. Bug: 138182343 Test: m Change-Id: I1c2ffe8d28ccf648d928ea59652c2d0070bf10eb
2019-10-16Add method to determine variations from a Target Colin Cross
The arch variants are hardcoded in every module type. Refactor them out into a Target.Variations() method in preparation for splitting the arch mutator into two, which will require using different variations. Test: m checkbuild Change-Id: I28ef7cd5168095ac888fe77f04e27f9ad81978c0
2019-10-15Create scripts to update and freeze a module SDK Jiyong Park
`m <sdk_name>` generates two scripts each of which is use to update the current snapshot of the SDK and to freeze ToT as a new version, respectively. Executing the scripts will copy necessary files (stub libraries, AIDL files, etc.) along with Android.bp into the ./<apiver> directory under the directory where the sdk is defined. This change also introduces a new module type 'sdk_snapshot' that represents a snapshot of an SDK. It will be auto-generated by the above scripts, so developers are not expected to write this manually. The module type 'sdk' is now used to simply specify the list of modules that an SDK has. Finally, this change changes the version separator from '#' to '@' because '#' confuses Make. Bug: 138182343 Test: m Change-Id: Ifcbc3a39a2f6ad5b4f4b200ba55a1ce3281498cf
2019-09-22Introduce module type 'sdk' Jiyong Park
This change introduces a new module type named 'sdk'. It is a logical group of prebuilt modules that together provide a context (e.g. APIs) in which Mainline modules (such as APEXes) are built. A prebuilt module (e.g. java_import) can join an sdk by adding it to the sdk module as shown below: sdk { name: "mysdk#20", java_libs: ["myjavalib_mysdk_20"], } java_import { name: "myjavalib_mysdk_20", srcs: ["myjavalib-v20.jar"], sdk_member_name: "myjavalib", } sdk { name: "mysdk#21", java_libs: ["myjavalib_mysdk_21"], } java_import { name: "myjavalib_mysdk_21", srcs: ["myjavalib-v21.jar"], sdk_member_name: "myjavalib", } java_library { name: "myjavalib", srcs: ["**/*/*.java"], } An APEX can specify the SDK(s) that it wants to build with via the new 'uses_sdks' property. apex { name: "myapex", java_libs: ["libX", "libY"], uses_sdks: ["mysdk#20"], } With this, libX, libY, and their transitive dependencies are all built with the version 20 of myjavalib (the first java_import module) instead of the other one (which is for version 21) and java_library having the same name (which is for ToT). Bug: 138182343 Test: m (sdk_test.go added) Change-Id: I7e14c524a7d6a0d9f575fb20822080f39818c01e