Enable building art and tests with linux_bionic
A linux_bionic built runtime can be created by modifying the
$OUT_DIR/soong/soong.config json file to add a '"Host_bionic": true'
entry and then running the following command:
% m --skip-make $OUT_DIR/soong/host/linux_bionic-x86/<target-file>
A script 'tools/build_linux_bionic_tests.sh' was also added that will
build linux-bionic versions of host-test related files.
Also modify tests so they can be run. Currently all run-tests pass
when run on a linux_bionic host build. All gtests pass except for the
following 9 tests:
ArmVIXLAssemblerTest.VixlJniHelpers
ArmVIXLAssemblerTest.VixlLoadFromOffset
ArmVIXLAssemblerTest.VixlStoreToOffset
DwarfTest.DebugFrame
DwarfTest.DebugFrame64
DwarfTest.x86_64_RegisterMapping
DwarfTest.DebugLine
DwarfTest.DebugLineSpecialOpcodes
DwarfTest.DebugInfo
Since these tests must be run manually without the standard
(makefile-based) testrunner it's not clear yet whether these failures
are important or what must be done to fix them. Several tests had
dependencies on the actual location of the libart library that needed
to be fixed, these tests might as well.
Test: ./tools/build_linux_bionic_tests.sh
Test: ./test/testrunner/testrunner.py --no-build --host --64 --run-test-option=--bionic
Test: find $OUT_DIR/soong/host/linux_bionic-x86/nativetest64 -mindepth 2 -type f | while read abc; do $abc; done | tee all-tests.txt
Test: tail -n +0 -f all-tests.txt | grep FAILED
Bug: 31559095
Bug: 118778408
Change-Id: I5fae4e54371e7410678619f18fcb54a3bd34e9b0
diff --git a/compiler/utils/assembler_thumb_test.cc b/compiler/utils/assembler_thumb_test.cc
index 053e202..3d26296 100644
--- a/compiler/utils/assembler_thumb_test.cc
+++ b/compiler/utils/assembler_thumb_test.cc
@@ -125,7 +125,7 @@
// Assemble the .S
snprintf(cmd, sizeof(cmd), "%sas %s -o %s.o", toolsdir.c_str(), filename, filename);
int cmd_result = system(cmd);
- ASSERT_EQ(cmd_result, 0) << strerror(errno);
+ ASSERT_EQ(cmd_result, 0) << cmd << strerror(errno);
// Disassemble.
snprintf(cmd, sizeof(cmd), "%sobjdump -D -M force-thumb --section=.text %s.o | grep '^ *[0-9a-f][0-9a-f]*:'",
diff --git a/libartbase/base/file_utils_test.cc b/libartbase/base/file_utils_test.cc
index f7c9c5e..c917307 100644
--- a/libartbase/base/file_utils_test.cc
+++ b/libartbase/base/file_utils_test.cc
@@ -83,11 +83,15 @@
ASSERT_EQ(0, unsetenv("ANDROID_ROOT"));
std::string android_root3 = GetAndroidRootSafe(&error_msg);
// This should be the same as the other root (modulo realpath), otherwise the test setup is
- // broken.
- UniqueCPtr<char> real_root(realpath(android_root.c_str(), nullptr));
+ // broken. On non-bionic. On bionic we can be running with a different libart that lives outside
+ // of ANDROID_ROOT
UniqueCPtr<char> real_root3(realpath(android_root3.c_str(), nullptr));
+#if !defined(__BIONIC__ ) || defined(__ANDROID__)
+ UniqueCPtr<char> real_root(realpath(android_root.c_str(), nullptr));
EXPECT_STREQ(real_root.get(), real_root3.get());
-
+#else
+ EXPECT_STRNE(real_root3.get(), "");
+#endif
// Reset ANDROID_ROOT, as other things may depend on it.
ASSERT_EQ(0, setenv("ANDROID_ROOT", android_root_env.c_str(), /* overwrite */ 1));
diff --git a/test/099-vmdebug/check b/test/099-vmdebug/check
index d124ce8..6a3fed5 100755
--- a/test/099-vmdebug/check
+++ b/test/099-vmdebug/check
@@ -15,6 +15,6 @@
# limitations under the License.
# Strip the process pids and line numbers from exact error messages.
-sed -e '/^dalvikvm\(\|32\|64\) E.*\] /d' "$2" > "$2.tmp"
+sed -e '/^.*dalvikvm\(\|32\|64\) E.*\] /d' "$2" > "$2.tmp"
diff --strip-trailing-cr -q "$1" "$2.tmp" >/dev/null
diff --git a/test/143-string-value/check b/test/143-string-value/check
index 2a3476c..b5e51ce 100755
--- a/test/143-string-value/check
+++ b/test/143-string-value/check
@@ -15,6 +15,6 @@
# limitations under the License.
# Strip error log messages.
-sed -e '/^dalvikvm\(\|32\|64\) E.*\] /d' "$2" > "$2.tmp"
+sed -e '/^.*dalvikvm\(\|32\|64\) E.*\] /d' "$2" > "$2.tmp"
diff --strip-trailing-cr -q "$1" "$2.tmp" >/dev/null
diff --git a/test/etc/run-test-jar b/test/etc/run-test-jar
index 900b1d7..1233aca 100755
--- a/test/etc/run-test-jar
+++ b/test/etc/run-test-jar
@@ -31,6 +31,8 @@
GDB_SERVER="gdbserver"
HAVE_IMAGE="y"
HOST="n"
+BIONIC="n"
+CREATE_ANDROID_ROOT="n"
INTERPRETER="n"
JIT="n"
INVOKE_WITH=""
@@ -209,6 +211,13 @@
HOST="y"
ANDROID_ROOT="$ANDROID_HOST_OUT"
shift
+ elif [ "x$1" = "x--bionic" ]; then
+ BIONIC="y"
+ # We need to create an ANDROID_ROOT because currently we cannot create
+ # the frameworks/libcore with linux_bionic so we need to use the normal
+ # host ones which are in a different location.
+ CREATE_ANDROID_ROOT="y"
+ shift
elif [ "x$1" = "x--no-prebuild" ]; then
PREBUILD="n"
shift
@@ -374,6 +383,10 @@
done
fi
+if [ "$CREATE_ANDROID_ROOT" = "y" ]; then
+ ANDROID_ROOT=$DEX_LOCATION/android-root
+fi
+
if [ "x$1" = "x" ] ; then
MAIN="Main"
else
@@ -644,6 +657,15 @@
fi
fi
+if [ "$BIONIC" = "y" ]; then
+ # This is the location that soong drops linux_bionic builds. Despite being
+ # called linux_bionic-x86 the build is actually amd64 (x86_64) only.
+ if [ ! -e "$OUT_DIR/soong/host/linux_bionic-x86" ]; then
+ echo "linux_bionic-x86 target doesn't seem to have been built!" >&2
+ exit 1
+ fi
+fi
+
# Prevent test from silently falling back to interpreter in no-prebuild mode. This happens
# when DEX_LOCATION path is too long, because vdex/odex filename is constructed by taking
# full path to dex, stripping leading '/', appending '@classes.vdex' and changing every
@@ -671,6 +693,21 @@
mkdir_locations="${DEX_LOCATION}/dalvik-cache/$ISA"
strip_cmdline="true"
sync_cmdline="true"
+linkroot_cmdline="true"
+linkroot_overlay_cmdline="true"
+
+linkdirs() {
+ find "$1" -maxdepth 1 -mindepth 1 -type d | xargs -i ln -sf '{}' "$2"
+}
+
+if [ "$CREATE_ANDROID_ROOT" = "y" ]; then
+ mkdir_locations="${mkdir_locations} ${ANDROID_ROOT}"
+ linkroot_cmdline="linkdirs ${ANDROID_HOST_OUT} ${ANDROID_ROOT}"
+ if [ "${BIONIC}" = "y" ]; then
+ # TODO Make this overlay more generic.
+ linkroot_overlay_cmdline="linkdirs $OUT_DIR/soong/host/linux_bionic-x86 ${ANDROID_ROOT}"
+ fi
+fi
# PROFILE takes precedence over RANDOM_PROFILE, since PROFILE tests require a
# specific profile to run properly.
@@ -949,7 +986,8 @@
for var in ANDROID_PRINTF_LOG ANDROID_DATA ANDROID_ROOT LD_LIBRARY_PATH DYLD_LIBRARY_PATH PATH LD_USE_LOAD_BIAS; do
echo EXPORT $var=${!var}
done
- echo "mkdir -p ${mkdir_locations} && $profman_cmdline && $dex2oat_cmdline && $dm_cmdline && $vdex_cmdline && $strip_cmdline && $sync_cmdline && $cmdline"
+ echo "$(declare -f linkdirs)"
+ echo "mkdir -p ${mkdir_locations} && $linkroot_cmdline && $linkroot_overlay_cmdline && $profman_cmdline && $dex2oat_cmdline && $dm_cmdline && $vdex_cmdline && $strip_cmdline && $sync_cmdline && $cmdline"
fi
cd $ANDROID_BUILD_TOP
@@ -963,6 +1001,8 @@
export ASAN_OPTIONS=$RUN_TEST_ASAN_OPTIONS
mkdir -p ${mkdir_locations} || exit 1
+ $linkroot_cmdline || { echo "create symlink android-root failed." >&2 ; exit 2; }
+ $linkroot_overlay_cmdline || { echo "overlay android-root failed." >&2 ; exit 2; }
$profman_cmdline || { echo "Profman failed." >&2 ; exit 2; }
$dex2oat_cmdline || { echo "Dex2oat failed." >&2 ; exit 2; }
$dm_cmdline || { echo "Dex2oat failed." >&2 ; exit 2; }
diff --git a/test/run-test b/test/run-test
index 229e201..2363152 100755
--- a/test/run-test
+++ b/test/run-test
@@ -88,6 +88,8 @@
export ANDROID_HOST_OUT=${OUT_DIR}/host/linux-x86
fi
+host_lib_root=${ANDROID_HOST_OUT}
+
# Allow changing DESUGAR script to something else, or to disable it with DESUGAR=false.
if [ -z "$DESUGAR" ]; then
export DESUGAR="$ANDROID_BUILD_TOP/art/tools/desugar.sh"
@@ -385,6 +387,14 @@
run_args="${run_args} --64"
suffix64="64"
shift
+ elif [ "x$1" = "x--bionic" ]; then
+ # soong linux_bionic builds are 64bit only.
+ run_args="${run_args} --bionic --host --64"
+ suffix64="64"
+ target_mode="no"
+ DEX_LOCATION=$tmp_dir
+ host_lib_root=$OUT_DIR/soong/host/linux_bionic-x86
+ shift
elif [ "x$1" = "x--trace" ]; then
trace="true"
shift
@@ -584,7 +594,7 @@
if [ "$target_mode" = "no" ]; then
guess_host_arch_name
run_args="${run_args} --boot ${ANDROID_HOST_OUT}/framework/core${image_suffix}.art"
- run_args="${run_args} --runtime-option -Djava.library.path=${ANDROID_HOST_OUT}/lib${suffix64}:${ANDROID_HOST_OUT}/nativetest${suffix64}"
+ run_args="${run_args} --runtime-option -Djava.library.path=${host_lib_root}/lib${suffix64}:${host_lib_root}/nativetest${suffix64}"
else
guess_target_arch_name
run_args="${run_args} --runtime-option -Djava.library.path=/data/nativetest${suffix64}/art/${target_arch_name}"
@@ -711,6 +721,7 @@
echo " --output-path [path] Location where to store the build" \
"files."
echo " --64 Run the test in 64-bit mode"
+ echo " --bionic Use the (host, 64-bit only) linux_bionic libc runtime"
echo " --trace Run with method tracing"
echo " --strace Run with syscall tracing from strace."
echo " --stream Run method tracing in streaming mode (requires --trace)"
diff --git a/tools/build_linux_bionic_tests.sh b/tools/build_linux_bionic_tests.sh
new file mode 100755
index 0000000..dba2d50
--- /dev/null
+++ b/tools/build_linux_bionic_tests.sh
@@ -0,0 +1,101 @@
+#!/bin/bash
+#
+# Copyright (C) 2018 The Android Open Source Project
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+
+if [[ -z $ANDROID_BUILD_TOP ]]; then
+ pushd .
+else
+ pushd $ANDROID_BUILD_TOP
+fi
+
+if [ ! -d art ]; then
+ echo "Script needs to be run at the root of the android tree"
+ exit 1
+fi
+
+source build/envsetup.sh >&/dev/null # for get_build_var
+
+out_dir=$(get_build_var OUT_DIR)
+
+# TODO(b/31559095) Figure out a better way to do this.
+#
+# There is no good way to force soong to generate host-bionic builds currently
+# so this is a hacky workaround.
+
+# First build all the targets still in .mk files (also build normal glibc host
+# targets so we know what's needed to run the tests).
+build/soong/soong_ui.bash --make-mode "$@" test-art-host-run-test-dependencies build-art-host-tests
+if [ $? != 0 ]; then
+ exit 1
+fi
+
+tmp_soong_var=$(mktemp --tmpdir soong.variables.bak.XXXXXX)
+
+echo "Saving soong.variables to " $tmp_soong_var
+cat $out_dir/soong/soong.variables > ${tmp_soong_var}
+python3 <<END - ${tmp_soong_var} ${out_dir}/soong/soong.variables
+import json
+import sys
+x = json.load(open(sys.argv[1]))
+x['Allow_missing_dependencies'] = True
+x['HostArch'] = 'x86_64'
+x['CrossHost'] = 'linux_bionic'
+x['CrossHostArch'] = 'x86_64'
+if 'CrossHostSecondaryArch' in x:
+ del x['CrossHostSecondaryArch']
+json.dump(x, open(sys.argv[2], mode='w'))
+END
+if [ $? != 0 ]; then
+ mv $tmp_soong_var $out_dir/soong/soong.variables
+ exit 2
+fi
+
+soong_out=$out_dir/soong/host/linux_bionic-x86
+declare -a bionic_targets
+# These are the binaries actually used in tests. Since some of the files are
+# java targets or 32 bit we cannot just do the same find for the bin files.
+#
+# We look at what the earlier build generated to figure out what to ask soong to
+# build since we cannot use the .mk defined phony targets.
+bionic_targets=(
+ $soong_out/bin/dalvikvm
+ $soong_out/bin/dalvikvm64
+ $soong_out/bin/dex2oat
+ $soong_out/bin/dex2oatd
+ $soong_out/bin/profman
+ $soong_out/bin/profmand
+ $soong_out/bin/hiddenapi
+ $soong_out/bin/hprof-conv
+ $(find $ANDROID_HOST_OUT/lib64 -type f | sed "s:$ANDROID_HOST_OUT:$soong_out:g")
+ $(find $ANDROID_HOST_OUT/nativetest64 -type f | sed "s:$ANDROID_HOST_OUT:$soong_out:g"))
+
+echo building ${bionic_targets[*]}
+
+build/soong/soong_ui.bash --make-mode --skip-make "$@" ${bionic_targets[*]}
+ret=$?
+
+mv $tmp_soong_var $out_dir/soong/soong.variables
+
+# Having built with host-bionic confuses soong somewhat by making it think the
+# linux_bionic targets are needed for art phony targets like
+# test-art-host-run-test-dependencies. To work around this blow away all
+# ninja files in OUT_DIR. The build system is smart enough to not need to
+# rebuild stuff so this should be fine.
+rm -f $OUT_DIR/*.ninja $OUT_DIR/soong/*.ninja
+
+popd
+
+exit $ret