summaryrefslogtreecommitdiff
path: root/filesystem/filesystem.go
AgeCommit message (Collapse)Author
2025-03-24Merge "Build and dist updatepackage" into main Treehugger Robot
2025-03-24Build and dist updatepackage Spandan Das
Equivalent make code: https://cs.android.com/android/_/android/platform/build/+/577341036beabe1cf4dcc479b254b878b8963b8d:core/Makefile;l=7593-7620;drc=577341036beabe1cf4dcc479b254b878b8963b8d;bpv=1;bpt=0 Some custom partitions are not included in the updatepackage. Make determines by looking at a board config variable. To implement this exclusion in Soong, a new `No_flashall` property has been added to `android_filesystem`. Bug: 383902856 Test: Built img.zip files for both make and soong Test: verified that they contain the same no. of files. Change-Id: If4df40a7ceb2ef68de27fb44f9e8db4503695e4e
2025-03-21Merge changes I32e163b9,Ie2d4ad83 into main Treehugger Robot
* changes: Dist Soong built misc_info.txt in Soong only builds Do not write avb_*_key_path of android_filesystem to misc_info.txt
2025-03-21Merge "Create avb_recovery_add_hash_footer_args entry in misc_info.txt" into ↵ Treehugger Robot
main
2025-03-21Do not write avb_*_key_path of android_filesystem to misc_info.txt Spandan Das
This matches the make packaging system implementaton. avb_*_key_path of bootimg, vbmeta and system_other will continue to be written to Soong built misc_info.txt Bug: 398036609 Test: Built and diff'd locally Change-Id: Ie2d4ad8362ce7f44d473ef185562c60718e173b3
2025-03-21Create avb_recovery_add_hash_footer_args entry in misc_info.txt Spandan Das
This CL creates an additional avb_recovery_add_hash_footer_args entry for bootimages with a ramdisk that contains a recovery partition. strings.ReplaceAll will be used to convert ``` --prop com.android.build.vendor_boot.fingerprint:generic/aosp_cf_x86_64_phone/vsoc_x86_64:Baklava/MAIN/eng.spanda:userdebug/test-keys ``` to ``` --prop com.android.build.recovery.fingerprint:generic/aosp_cf_x86_64_phone/vsoc_x86_64:Baklava/MAIN/eng.spanda:userdebug/test-keys ``` Test: Built and diff'd locally Bug: 398036609 Change-Id: I0caa8aa9e2e86b9c78352d3038951e00f689dc1a
2025-03-19Merge "Add system_other partition props to Soong's misc_info.txt" into main Treehugger Robot
2025-03-18Add system_other partition props to Soong's misc_info.txt Spandan Das
Implementation Details - Expand FilesystemInfo with some Avb* props. `system_other` will use this info exported by `system` to create system_other entries for misc_info.txt. This is necessary since system_other does not contain sufficient information to create a standalone buildPropFile, and reuses the buildPropFile of system in `build_image` invocation - Harcode rollback_index to the timestamp parsed from `PlatformSecurityPatch`. Most products (including cuttlefish) do not use a product config variable to override the rollback_index. Eventually, we should dynamically get this value from one of the vbmeta partitions. Bug: 398036609 Test: Built Make and Soong misc_info.txt Test: No diff in system_other entries Change-Id: Iadb0ee34d948cb15cd5a59b0dff7bf7fc64024ed
2025-03-18Merge "Add vintf info and userdata size to Soong's misc_info.txt" into main Treehugger Robot
2025-03-18Add vintf info and userdata size to Soong's misc_info.txt Spandan Das
- vintf info will be read from the global productVariables struct. This will likely need to be revisited when we have multiple android_device's in the tree - userdata has been special-cased to create a `userdata_size` entry in addition to $(filesystem)_partition_size entry. This special case matches the make misc_info.txt implementation. Bug: 398036609 Test: Built Soong's misc_info.txt locally Change-Id: I9f312c8022c5faf84442cb51f1e13043768aed53
2025-03-17Merge "Add android_filesystem props to misc_info.txt" into main Treehugger Robot
2025-03-17Add android_filesystem props to misc_info.txt Spandan Das
Similar to https://r.android.com/3543364, this CL introduces a separate "partition qualified" prop file to `android_filesystem`. The contents of the different filesystem's will be coalesced by the top-level android_device's misc_info.txt This implementation introduces duplicates since some properties like `erofs_default_compressor` will be written multiple times to misc_info.txt. To prevent these duplicates, and also to help comparison with Make built misc_info.txt, `sort -u` has been added to misc_info.txt (There are still a lot of diffs between Make and Soong misc_info.txt) Test: Built Soong's misc_info.txt locally Bug: 398036609 Change-Id: If5ffee116d008dda5528bff0354719cec871750a
2025-03-14Merge "Change CommonModuleInfoProvider to a pointer." into main Yu Liu
2025-03-14Refactor fsTypeStr to a String function on fsType Spandan Das
Test: go build ./filesystem Change-Id: Ic389428e9ba7b60a96d0861bf46277bbe214a4ff
2025-03-14Refactor filesystem's avb_add_hashtree_footer_args into a function Spandan Das
This function will be reused to build misc_info.txt. This CL should be a noop. Test: go build ./android Bug: 398036609 Change-Id: I69d44d8aa05e68db4a77c9c1c3d9ab590cd6550e
2025-03-13Change CommonModuleInfoProvider to a pointer. Yu Liu
Bug: 358427516 Test: Manually verified genereated ninja and mk files, unit tests. Change-Id: I53a6dd718232735decbeb93febfd269dd9449e86
2025-03-07Rename CommonModuleInfoKey to be consistent with others. Yu Liu
Bug: 377723687 Test: Unit tests and compare the ninja and mk files generated. Change-Id: I1ce8bc782666586633ded9cfaf1b9590e3c0efde
2025-03-06Merge "Propagate installed files of Include_files_of dependencies" into main Treehugger Robot
2025-03-06Merge "Modify namings of disted installed files" into main Treehugger Robot
2025-03-06Merge "Add support for SBOM generation in soong-only builds" into main Treehugger Robot
2025-03-07Propagate installed files of Include_files_of dependencies Jihoon Kang
Some partitions (e.g. recovery) are transitive dependencies of android_device module, and are not directly listed as partitions. This change allows the installed files of `Include_files_of` partitions to be propagated to the rdep filesystem module, so that it can reach the android_device module and be disted. Test: m nothing dist --soong-only && aninja -t query _dist_droidcore-unbundled Bug: 395162005 Change-Id: Icc0e902d5d42b925d04470b60b5e62daac4c4a17
2025-03-07Modify namings of disted installed files Jihoon Kang
To match the names to those in soong+make build. Test: m droid dist --soong-only && ls -l out/dist Bug: 395162005 Change-Id: I196fa46de628a3dded9ba9f804337c19ddea119b
2025-03-06Add support for SBOM generation in soong-only builds Wei Li
Add an alternative way of collecting metadata in soong-only build, which is initially done in make. Bug: 398039178 Test: presubmits Test: lunch aosp_cf_x86_64_phone-trunk_staging-eng && m && m sbom Change-Id: I94476db21cf9eac8be7693043f2cd7a2b1bcd8a6
2025-03-06Create apkcerts.txt using Soong Spandan Das
This CL creates an apkcerts.txt of the apps installed on the device, and any apk-in-apex of installed apexes. Note that this behavior is unlike make. There is some ongoing discussion on whether the make behavior can be updated to only list the installed apps as well. To implement this, androidDevice will consult the following providers from the list of installed modules - AppInfoProvider (android_app, android_app_import, ...) - AppInfosProvider (for apk-in-apex) - RuntimeResourceOverlayInfoProvider (for checked-in and autogen rros) Test: With https://r.android.com/3529333, apkcerts.txt is identical for aosp_cf_x86_64_phone between make and soong Bug: 399788149 Change-Id: I10009ea8761a197dd2301acf9615079bd28af3f5
2025-03-05Create a partial implementation of $PRODUCT_OUT/misc_info.txt Spandan Das
This file is packaged into META/target_files.zip. It is also an implicit input for `add_img_to_target_files` invocation, which creates additional metadata files for target_files.zip like apex_info.pb, care_map.pb and vbmeta_digest.txt This CL implements the minimum fields necessary to run `add_img_to_target_files`. Bug: 399788119 Test: Built target_files.zip locally. Test: Verified META/misc_info.txt exists, but is not identical to make Change-Id: I52b9c9ff7b33e74d79b9321aa94312397a830693
2025-03-04Merge changes from topic "revive_system_ext_aconfig_container" into main Treehugger Robot
* changes: Soong filesystem generates aconfig files for subpartitions Revert "Treat system_ext as system container"
2025-02-27Soong filesystem generates aconfig files for subpartitions Justin Yun
When system had system_ext or other subpartitions, aconfig files for the subpartitions were missing. Generate those aconfig files for the subpartitions. With this fix, we don't need to import aconfig files from system_ext to generate system aconfig files. Bug: 385213580 Test: lunch gsi_x86_64-trunk_staging-userdebug && m Change-Id: Ie3db5c73c50af97f3ad0bea5e0c84273c9fce886
2025-02-26Convert tidyPhonySingleton, jdepsGeneratorSingleton and Yu Liu
genNoticeBuildRules to use ModuleProxy. Bug: 377723687 Test: Unit tests and compare the ninja and mk files generated. Change-Id: I7507a580e3533b01f552778a7815bcc43d301e23
2025-02-25Convert the following singletons to use ModuleProxy: Yu Liu
buildTargetSingleton exportedJavaDeclarationsLibrarySingleton javaFuzzPackager kytheExtractAllSingleton kytheExtractJavaSingleton kytheExtractRustSingleton lintSingleton logtagsSingleton makeVarsSingleton ndkAbiDiffSingleton ndkAbiDumpSingleton phonySingleton platformCompatConfigSingleton sdkSingleton Bug: 377723687 Test: Unit tests and compare the ninja and mk files generated. Change-Id: I625ac8a209ca93755b2ba232202cfb44ecefde0a
2025-02-20Build proguard zips in soong-only builds Cole Faust
This builds the zips but doesn't dist them, I'll do that in a followup. Bug: 395160816 Test: Diffed proguard-dict.zip proguard-usage.zip and proguard-dict-mapping.textproto between soong and make Change-Id: Ic112427e4a2fdaf92a1d0a96d0f1c100b57be37a
2025-02-19Merge "Fix nondeterministic FullInstallPathInfo list" into main Treehugger Robot
2025-02-19Merge changes from topic "dependency_sub_dir" into main Treehugger Robot
* changes: Propagate owners info in filesystem provider Introduce Soong API ctx.OtherModuleSubDir(m Module)
2025-02-19Fix nondeterministic FullInstallPathInfo list Cole Faust
This list later shows up in the ninja file in soong-only builds, make sure it's deterministic. Test: Presubmits Change-Id: Id121b8b95a543cd019f7c51f4121aa2c761f8a9f
2025-02-19Propagate owners info in filesystem provider Jihoon Kang
Owners contain information about the name and the variants of the modules that are installed in the partition. This information can be used to determine whether the dependency module is installed or not. Utilization of this information will be done in a follow up change. Test: m nothing Bug: 395989947 Change-Id: I8c63ed5765a3f582ff0d2ce98f63e6e0fc6edad8
2025-02-19Merge "Change depVisitor to use providers instead of type-asserting to ↵ Yu Liu
interfaces directly, the next step is to change it to use ModuleProxy once IsDepInSameApex is ready." into main
2025-02-19Change depVisitor to use providers instead of type-asserting to Yu Liu
interfaces directly, the next step is to change it to use ModuleProxy once IsDepInSameApex is ready. Bug: 377723687 Test: Unit tests and compare the ninja and mk files generated. Change-Id: I13a4e256a26dbf7f9b3b746d628ac8f68b4e598e
2025-02-18Handle the ramdisk implicitly in target-file.zip Cole Faust
We should not be checking ctx.DeviceConfig().BoardMoveRecoveryResourcesToVendorBoot() in android_device, because there may be multiple android_devices in the tree with different partition layouts. Instead, get the recovery image files transitively from the vendor_boot image. Bug: 376539388 Test: Built target_files.zip, checked all the files are still there. Change-Id: I1837aedacf71063879b26b51b8fe28b684b96d78
2025-02-13Merge changes Ie647c190,I3a3997c0 into main Treehugger Robot
* changes: Add Ramdisk_node_list to target_files.zip Generate $partition_filesystem_config.txt for target_files.zip
2025-02-13Generate $partition_filesystem_config.txt for target_files.zip Spandan Das
These files will be packaged into META subdir of target_files.zip Some details - This CL adds the filesystem_config.txt files for system, vendor, ... Some other partitions like system_other are currently missing (different module type) - There is a subtle difference between the make and soong build rule. In the make rule, it uses $TARGET_OUT (all partitions) as the -D in `fs_config`. In the soong rule, this implementation passes the staging directory of the filesystem. This difference does not seem to have a difference on the final result. - Some filesystem_config are currently missing/not bit-identical. These include boot, init_boot. For these, we actually want to use the configs from their ramdisk filesystems. I will do this in a followup CL Test: diff'd target_files.zip The following files are identical except NOTICE.xml.gz entry - odm_dlkm, odm, product, system_dlkm, system_ext, vendor_dlkm, vendor The following files are _not_ identical in names and/or contents - boot_filesystem_config.txt - root_filesystem_config.txt - system_other_filesystem_config.txt - vendor_boot_filesystem_config.txt Bug: 388633394 Change-Id: I3a3997c0ffd8100c44b4a50a63bf0709a61d4472
2025-02-13Make soong-only builds adb-sync compatible Cole Faust
Historically, the build has put the staging directories for partitions in out/target/product. But in soong-only builds, the main staging directory is in the out/soong/.intermediates path for the filesystem modules, and then later also replicated in out/target/product for backwards compatibility with tools like adb sync. However, adb sync determines which files to push by if they have a different timestamp than on device. Since the device images were built using a different staging directory than the one adb sync is looking at, the have different timestamps, and adb sync syncs all files on the device the first time it's run. To work around this, ensure the files in both staging directories have the same timestamp by preserving the timestamp from their source file with the `cp -p` flag. This cl only does that change for real files right now, not symlinks. We could also do it for symlinks in the future, but they're more complicated and I didn't want to deal with that right now. This means the first adb sync will resync all symlinks on the device, but there are relatively few of them and they're small so it's not that big of an issue. Long term, we should rewrite adb sync to not depend on staging directories. If it looks at the actual module intermediates files, it will see a more consistent timestamp. Bug: 396466053 Test: Manually ran m sync workflows. Also testing togging soong-only on and off and rebuilding Change-Id: Ia0076a2f8e58404c5c3a0bf0d33b09b6846e95bd
2025-02-12Copy selinux_fc file to target_files.zip Spandan Das
This CL introduces a new field in FilesystemInfo with the selinux_fc path of the filesystem(s). android_device will copy this file to the META subdirectory of target_files.zip Test: Built soong target_files.zip Bug: 388633394 Change-Id: Ia81bd2c2d90d6beb116d57adf53deb5736579c3a
2025-02-12Package erofs compress hints txt in target_files.zip Spandan Das
Test: Built the soong target_files.zip locally Bug: 388633394 Change-Id: I2a87d24dc675dfb13eb02941956ec5a8593e5bb5
2025-02-12Merge "Dist installed-files.* in soong only build" into main Treehugger Robot
2025-02-12Dist installed-files.* in soong only build Jihoon Kang
Implementation details: - Define static rules for building installed-files.txt and installed-files.json - Avoided adding dependency on the root dir and added dependency on the output image file instead - Propagate the generated installed-files.* via filesystem info provider and generate the dist rule in the main android device Test: m droid dist --soong-only && ls -l out/dist Bug: 395162005 Bug: 394365683 Change-Id: I615b0374c557fd11c19fcd190232cab411e2d299
2025-02-12Include hermetic .img files in target_files.zip Spandan Das
https://r.android.com/3470135 disabled target_files.zip generation since the previous implementation built two copies of .img files when building droid. This increased full build times slightly. This CL adds the build rule for creating the hermetic version of .img files, but keeps it outside the transitive closure of droid. Implementation details - Split the monolithic cmd into three - cmd1 assembles the staging dir and creates a timestamp file - cmd2 depends on timestamp file and creates .img file using build_image - cmd3 depends on timestamp file and creates a hermetic .img file using build_image. It does so by creating an intermediate propfile with `use_fixed_timestamp=true` Bug: 393203512 Test: Built the soong target_files.zip locally (There are some preexisting diffs with the make target_files.zip, which I will resolve in ongoing work) Change-Id: Ibd14da93a6534d5544deffa2c44bac3fb97052ad
2025-02-11Export SOONG_DEFINED_SYSTEM_IMAGE_PATH to make Spandan Das
This makes the make packaging system resilient to filename changes. Test: lunch aosp_cf_x86_64_phone-userdebug && m systemimage Change-Id: I166d78cd49f72a875b9648ca9285f7ebc2307a76
2025-02-11Ensure AVB rollback_index_location >= 1 Luca Stefani
Index 0 is reserved for the vbmeta partition. This works even for chained vbmeta partitions as the "father" index location is used instead of the one bundled within the image description. Test: aosp_arm64; m --soong-only Change-Id: Ibb033878e4e56e77b1621a32ba1f2ba7d8ceeedd
2025-02-04Merge "Default avb_hash_algorithm to sha256 for android_filesystem modules" ↵ Nikita Ioffe
into main
2025-02-04Default avb_hash_algorithm to sha256 for android_filesystem modules Nikita Ioffe
This change only impacts android_filesytem and android_system_image soong modules which are currently only used to build microdroid images. Also add a neverallow rule to discourage usage of sha1. Bug: 341123987 Test: m Test: manually set avb_hash_algorithm: "sha1" check build fails Change-Id: I5449f711c751752db42dcca785ff1a2de054fb2d
2025-01-31Install apex symbol file Cole Faust
This is a partial revert of aosp/3308088. We need the symbol file to still be installed, because a later ctx.InstallSymlink file adds a dependency on the it. We removed that install initially so that it didn't get installed in filesystems, to replace that, just exclude files installed to the "apex" partition from filesystems. Bug: 393617060 Test: cd bionic; mm Change-Id: Id4abe966c630d1e356575386af233eab92849934