summaryrefslogtreecommitdiff
path: root/compiler/optimizing/intrinsics_x86_64.cc
AgeCommit message (Collapse)Author
2025-03-20Revert^2 "Call target method in accessor MHs when it is set." Almaz Mingaleev
Previous LUCI crashes were due to b/404465902, which was fixed in aosp/3551420. Bug: 297147201 Test: ART_HEAP_POISONING=true art/test/testrunner/testrunner.py -b \ --host --baseline --debug --gcstress --64 Test: art/test/testrunner/testrunner.py -b \ --host --baseline --debug --gcstress --64 Change-Id: If93b9c6559ac8ddbc3211df1cbc4dec192cb054a
2025-03-19Do not override register storing MethodType in invokeExact. Almaz Mingaleev
As it might be in a callee-saved register and also known by GC to have a reference. arm64 and riscv are fine as they load MethodHandle's field into a temp register and unpoison that register and not call site one. Bug: 404465902 Test: ART_HEAP_POISONING=true art/test/testrunner/testrunner.py -b --host --optimizing --64 Test: ART_HEAP_POISONING=true art/test/testrunner/testrunner.py -b --host --optimizing --gcstress --64 Test: art/test/testrunner/testrunner.py -b --host --optimizing --64 Test: art/test/testrunner/testrunner.py -b --host --optimizing --gcstress --64 Change-Id: Ib1e80ee106a30b282f6bf2725c460f9c24d20c0e
2025-03-10Revert "Call target method in accessor MHs when it is set." Almaz Mingaleev
Revert submission 3382609-target-accessors Reason for revert: x86.poison.64 failures on LUCI Reverted changes: /q/submissionid:3382609-target-accessors Change-Id: I384945809bc18ac4dfc937af088c7dbe30ac5c5c
2025-03-06Call target method in accessor MHs when it is set. Almaz Mingaleev
Bug: 297147201 Test: ./art/tools/run-libcore-tests.sh --mode host --variant X64 Test: ./art/test/testrunner/testrunner.py -b --host --64 Test: ./art/test/testrunner/testrunner.py -b --target --64 Change-Id: I5f8dc0761abbc1f5bb4f5727ae1e6966a4ebaf8a
2025-01-28Remove unused dex_pc from InvokeRuntime Santiago Aboy Solanes
Bug: 392802982 Test: art/test/testrunner/testrunner.py --host --64 -b --optimizing Test: m test-art-host-gtest Change-Id: I6e40215a5b1b18223c5f17e9e0ac70e05515fa94
2025-01-28Remove explicit dex_pc from RecordPcInfo Santiago Aboy Solanes
Special cases considered: * Frame entry (hardcoded to be 0) or block entry. * Native debuggable + slow paths, which is the only case where we use the instruction's dex_pc. Test: m test-art-host-gtest Test: art/test/testrunner/testrunner.py --host --64 -b --optimizing Change-Id: Ic5e0a6b5106395b891a9a45ea48da39dfb44a0a5
2025-01-02Rename `GetBootImageVarHandleField()`, drop "Boot". Vladimir Marko
This is a minor cleanup after https://android-review.googlesource.com/3048514 . Test: m test-art-host-gtest Test: testrunner.py --host --optimizing --speed-profile Bug: 38313278 Change-Id: Ic47063231fd48656b612ede3ec100ceb8a379050
2025-01-02Rename Unsafe.putOrdered* intrinsic identifiers. Vladimir Marko
Test: m test-art-host-gtest Test: testrunner.py --host --optimizing Change-Id: I1658a61953aaf109d68a2df7a534c3972679c291
2024-12-30x86_64: Clean up `movd()`, `movq()`. Vladimir Marko
Rename the 64-bit `movd()` assembler functions to `movq()` and remove the `is64bit` argument from the other `movd()` functions, making them always emit 32-bit MOVD. Change `CodeGeneratorX86_64::Exchange32(XmmRegister, int)` to use the 32-bit `MOVD` by not updating the old `movd()` call, letting it resolve to the 32-bit version now. Test: m test-art-host-gtest Test: testrunner.py --host --optimizing Change-Id: I3d8cdaf2c097ebc6044b09d20139aeb20ab00e11
2024-11-28Add poisoning/unpoisoning to invokeExact intrinsic. Almaz Mingaleev
Bug: 297147201 Test: ./art/test/testrunner/testrunner.py -b --host --64 -t 2277 Test: ART_HEAP_POISONING=1 ./art/test/testrunner/testrunner.py -b --host --64 -t 2277 Change-Id: Iad4a86faf84c834a44a2b622fc4eaab7752c2cba
2024-11-27x86-64: handle invoke-interface in invokeExact intrinsic. Almaz Mingaleev
Bug: 297147201 Test: ./art/test/testrunner/testrunner.py -b --host --64 Test: ./art/test/testrunner/testrunner.py -b --jvm Change-Id: I2c07ae919921363e8e419ec7296cd24696e7f3b5
2024-11-12Remove now unnecessary checks from invokeExact intrinsic. Almaz Mingaleev
Also rewrote tests to make sure that MethodHandle-s created via const-method-handle and Java API behave identically. They can be slightly different even if lookup is done in the same class for the same method as const-method-handle can target copied methods. Bug: 297147201 Test: ./art/test/testrunner/testrunner.py -b --host --64 Test: ./art/test/testrunner/testrunner.py -b --jvm Change-Id: I69cdf2c44f2838ee0e57eaa2e93a6ed5276c39fb
2024-11-12Add missing Location::kNoOutputOverlap Santiago Aboy Solanes
This can save some ParallelMove instructions. For x86(_64) all FPToFP intrinsics can add it. For RISC-V, MathSqrt can add it but the ones that call GenDoubleRound can't. Also, we can add it for MathMultiplyHigh. Test: art/test/testrunner/testrunner.py --host --64 -b --optimizing Test: LUCI run https://ci.chromium.org/b/8731845964396026257 Change-Id: I28e13caf84cd850566538efbd285c0264ce80a1a
2024-11-06Don't virtual dispatch non-copied method defined in an interface. Almaz Mingaleev
Following what Class::FindVirtualMethodForVirtualOrInterface does. Currently findVirtual sets MH's kind to INVOKE_INTERFACE only if class on which lookup is done is interface. Bug: 297147201 Test: ./art/test/testrunner/testrunner.py -b --host --64 Change-Id: Ib2576f317f7932111a05e464334de69a00e24ca1
2024-10-23x86-64: handle invoke-direct in invokeExact intrinsic. Almaz Mingaleev
Also fix NPE not being thrown in the interpreter when receiver is null in invoke-direct MethodHandle. Bug: 297147201 Test: ./art/test/testrunner/testrunner.py --host --64 --optimizing -b Change-Id: I30d369316b56713dc7428d264fcff98954c63d37
2024-10-17Revert^2 "Add intrinsics for the absolute forms of unsafe.{get,put}Int" Shai Barack
This reverts commit c1bd4376b18ff46e5156cc597bd3311dd86fabd9. Reason for revert: fixed original root cause for breakage Change-Id: Ic9c214ebd691abf7f163b6399e8a94fc6f277ea0
2024-10-15Remove unused variable. Christopher Ferris
Test: Builds with -Wunused-variable Change-Id: I4ecca28528cd146b5b60fccc8f378bbbfe185a8f
2024-10-11Revert "Add intrinsics for the absolute forms of unsafe.{get,put}Int" Santiago Aboy Solanes
This reverts commit bcb5c19e5e200607fe76294aeb5273ddac5f04ae. Bug: 370098695 Reason for revert: Breaks LUCI builds for arm https://ci.chromium.org/ui/p/art/builders/ci/angler-armv7-non-gen-cc/4506/overview Change-Id: Ia72938c65c45db0ccd14f8901e715a9ec2930087
2024-10-11Add intrinsics for the absolute forms of unsafe.{get,put}Int Shai Barack
Praise the Absolute! 🦑 Bug: 370098695 Change-Id: Icd0435c33a7a3705674eb2e3b027056a6c1298ba Tested: UnsafeTest Tested: manually inspected output disassembly
2024-10-07Address comments from aosp/3282234 Santiago Aboy Solanes
Follow up to aosp/3282234 regarding comments after submission. Bug: 362916226 Test: art/test/testrunner/testrunner.py --host --64 --optimizing -b Change-Id: Id572dd42e5aa877f4ae4a20cd43ad7a778e92815
2024-09-25Revert^2 "Add VarHandle implementations for void getAndUpdate methods" Santiago Aboy Solanes
This reverts commit 8567fcdf972ade85785b689a15e3593a7d3bff9d. Reason for revert: RISC-V fix Test: LUCI run for RISC-V Change-Id: I6392bfa045e7737b5c45d78da4f7ecb8fbd3b89a
2024-09-25Revert "Add VarHandle implementations for void getAndUpdate methods" Ulya Trofimovich
This reverts commit 9ff2b617341bfe574e2d8706553f0cd65a1a2fc8. Reason for revert: failed ART test 712-varhandle-invocations on riscv64 `F dex2oat64: intrinsics_riscv64.cc:4821 Check failed: locations->GetTempCount() == 2u (locations->GetTempCount()=3, 2u=2)` Change-Id: I68a3830ccd730afa66e9cff277fe2a06829ddabe
2024-09-24x86_64: remove subtype check from invoke-virtual fast path. Almaz Mingaleev
After aosp/3248173 and aosp/3250172 a MH object with invoke-virtual kind can target only virtual methods. As there is callsite check against MH type no receiver checks are needed. That's in line with JVMS 4.4.8. Bug: 297147201 Test: ./art/test/testrunner/testrunner.py --host --64 --optimizing -b Test: ./art/test.py --host -g Test: CtsLibcoreTestCases Test: CtsLibcoreOjTestCases Change-Id: I2c12760a75b83ddf7cc21d34bcc8bb58621e6cca
2024-09-24Add VarHandle implementations for void getAndUpdate methods Santiago Aboy Solanes
We were missing to intrinsify getAndUpdate methods whose value was unused. By adding this support, they get a considerable runtime improvement. Bug: 362916226 Test: art/test/testrunner/testrunner.py --host --64 --optimizing -b Test: art/test/testrunner/testrunner.py --host --32 --optimizing -b Test: art/test/testrunner/testrunner.py --target --64 --optimizing Test: art/test/testrunner/testrunner.py --target --32 --optimizing Change-Id: I63d5a897ac9ea3c133167ac186afd8595f75bb13
2024-09-24x86_64: Handle invoke-static in invokeExact fast path. Almaz Mingaleev
Bug: 297147201 Test: ./art/test/testrunner/testrunner.py --host --64 --optimizing -b Test: ./art/test.py --host -g Change-Id: Ia49285c116e4abfc3da2d78495f85d9131f111b5
2024-09-20Check receiver for null in invokeExact intrinsic. Almaz Mingaleev
Bug: 297147201 Test: ./art/test/testrunner/testrunner.py --host --64 --optimizing --jvm -b Change-Id: I6633bdab20b13335afccfe0506f6b91391c19fe3
2024-09-02Address follow-up comments from aosp/2721077. Almaz Mingaleev
Bug: 297147201 Test: ./art/test/testrunner/testrunner.py --host --64 -b --optimizing Change-Id: Ie89d310633339b785de5d9f3daf653abfbff875c
2024-08-28cleanup: Use AddRegisterTemps where appropriate Santiago Aboy Solanes
Test: art/test/testrunner/testrunner.py --host --64 --optimizing -b Change-Id: I2fc3cd539ee1f64d1757c3b2a0a532ff6e7366f7
2024-08-28x86_64: Add instrinsic for MethodHandle::invokeExact... Almaz Mingaleev
... which targets invoke-virtual methods. New entrypoint changes deliverException's offset, hence arm test change. Bug: 297147201 Test: ./art/test/testrunner/testrunner.py --host --64 -b --optimizing Test: ./art/test.py --host -g Change-Id: I636fc60c088bfdf9b695c92de47f1c539e3956f1
2024-07-08Fix code generation of Unsafe.putByte in x86 and x64. Nicolas Geoffray
We were not using the right instruction. Test: serial benchmark Bug: 339571886 Change-Id: Ice38a8c34acdb65908e3c41cebb3aad26a774beb
2024-05-15Fix bugprone-macro-parentheses clang-tidy issues Stefano Cianciulli
Bug: 264654008 Test: m tidy-art Change-Id: I6a96bee1510449760634dbef5c1fac21a7e1c80e
2024-04-22Optimizing: Treat app image objects as non-movable. Vladimir Marko
Treat app image objects similar to boot image objects and avoid unnecessary read barriers for app image `HLoadClass` and `HInstanceOf` checks with app image `HLoadClass` input. Extend other optimizations to treat app image classes the same way as boot image classes even though this remains mostly dormant because we currently do not initialize app image classes with class initializers; the experimental flag `--initialize-app-image-classes` is false by default. Test: m test-art-host-gtest Test: testrunner.py --host --optimizing --speed-profile Bug: 38313278 Change-Id: I359dd8897f6d128213602f5731d40edace298ab8
2024-04-17Fix typo in x86_64 VisitJdkUnsafePutByte Santiago Aboy Solanes
The type should have been Int instead of Uint like others in r.android.com/2751787 Bug: 313284159 Test: art/test/testrunner/testrunner.py --target --64 --optimizing Change-Id: Id43a744a33b40318b86046df2b2e1932aff6b9e8
2024-03-14Make sure there are no lone UNREACHABLEs Santiago Aboy Solanes
Either remove them, or add a LOG(FATAL) before them. Bug: 328756212 Test: art/test/testrunner/testrunner.py --host --64 --optimizing -b Test: m test-art-host-gtest Change-Id: Ibf2bddb0a4add5a844a515a040b3751acc7faf84
2024-01-29Revert^7 "Disable write-barrier elimination pass" Santiago Aboy Solanes
This reverts commit 1ba3516e8c3e2b86c73084893dd297f468469181. Reason for revert: PS1 is reland as-is PS2 has two related fixes (for RISC-V and arm64) taking into account that when we store zero, we use a special register. Bug: 301833859 Bug: 310755375 Bug: 260843353 Test: lunch cf_riscv64_wear-trunk_staging-userdebug && m Test: lunch starnix_wear_yukawa-trunk_staging-userdebug && m Change-Id: I5e69890fd56404ddde56ebc457179241363d9243
2024-01-24Revert^6 "Disable write-barrier elimination pass" Matthew Reynolds
This reverts commit 1be176f5a78750e2f0e32470f8c83e3d1643954d. Reason for revert: Potential cause of build breakage for cf_riscv64_wear-trunk_staging-userdebug build 11353124 Change-Id: I5db1c9fba1edd4ab1eef30e2b547bb9649af5c10
2024-01-24Revert^5 "Disable write-barrier elimination pass" Santiago Aboy Solanes
This reverts commit 31b949bc4a76e5c6d00a8e18c346f123b5321a1c. Reason for revert: PS1 is reland as-is PS2 has two fixes: * Missed poisoning heap references in a code path * Removed incorrect DCHECK Change-Id: I81b317ddc704dbd8a173f5d5c624dbc69e2d9e60 Test: art/test/testrunner/testrunner.py --host --64 --optimizing -b Test: art/test/testrunner/testrunner.py --target --64 --optimizing -b Both commnands with `export ART_HEAP_POISONING=true` Bug: 301833859 Bug: 310755375 Bug: 260843353
2024-01-22Revert^4 "Disable write-barrier elimination pass" Santiago Aboy Solanes
This reverts commit b5b98b9bb31acb2deffb692c50d0fbc71476663b. Reason for revert: Breaks tests in arm64 + heap poison configurations Bug: 310755375 Bug: 260843353 Change-Id: I682c74987a365497e0dbe47eba26a9ccf0513561
2024-01-22Revert^3 "Disable write-barrier elimination pass" Santiago Aboy Solanes
This reverts commit 9f8df195b7ff2ce47eec4e9b193ff3214ebed19c. Reason for revert: Fix for x86_64 with heap poison enabled This case uses a temp with index `1` in the regular FieldSet case. This is done like this due to GenerateVarHandleSet also calling HandleFieldSet. The bug was that we were allocating only one temp in the regular FieldSet case and therefore not having the temp with index `1` available. PS1 is the revert as-is. PS2 contains the fix. Test: art/test/testrunner/testrunner.py --host --64 --optimizing -b Test: Same command with heap poison enabled too Bug: 301833859 Bug: 310755375 Bug: 260843353 Change-Id: Ie2740b4c443158c4e72810ce1d8268353c5f0055
2024-01-19Revert^2 "Disable write-barrier elimination pass" Santiago Aboy Solanes
This reverts commit 7c1dd6e2d1893f288214413c4b97273980f3aa4a. Reason for revert: build breakages, using a different number of temps vs the expected (crashing in https://cs.android.com/android/platform/superproject/main/+/main:art/compiler/optimizing/code_generator_x86_64.cc;l=5488;drc=7c1dd6e2d1893f288214413c4b97273980f3aa4a) Change-Id: I843c039394dd666776ea5bcb5b10b1f47df12d53
2024-01-19Revert "Disable write-barrier elimination pass" Santiago Aboy Solanes
This reverts commit 5a3271d7caafefd10a20f5a5db09d2c178838b76. Reason for revert: This CL has two fixes (codegen not doing a null check if a write barrier is being relied on, and codegen not recomputing skipping write barriers), regression tests, a new runtime check which runs in debug mode for the CC GC to ensure that the card table is set correctly for skipped write barriers, and new compile time (graph checker) tests to ensure graph consistency. This patchset updates the WriteBarrierKind to be {emit being relied on, emit not being relied on, dont emit}, which leaves the null check skip implementation to codegen. Test 2247- is removed from knownfailures.json but still skipped in MTS due to SLO requirements. Test: art/test/testrunner/testrunner.py --host --64 --optimizing -b Bug: 301833859 Bug: 310755375 Bug: 260843353 Change-Id: I025597e284b2765986e2091538680ee629fb5ae7
2024-01-15riscv64: Skip WB for zero-length SystemArrayCopy. Vladimir Marko
Skip the unnecessary write barrier for zero-length arraycopy as we do on x86/x86-64 since https://android-review.git.corp.google.com/2893223 . Test: testrunner.py --target --64 --ndebug --optimizing Bug: 283082089 Change-Id: Ie30dc9e6ee66bee884c6f61141b14b7e02b404e3
2024-01-09x86, x86-64: Simplify SystemArrayCopy intrinsics. Vladimir Marko
This applies improvements from https://android-review.googlesource.com/2869995 to the x86 and x86-64 architectures and also changes the fast-path to use REP MOVSL. Test: m test-art-host-gtest Test: testrunner.py --host --optimizing Change-Id: Iba8e44f6c7002a94cfe0216518cc319fbe843b5b
2023-12-07Fix "the the" typos in the compiler. Vladimir Marko
Test: Rely on TreeHugger. Change-Id: I2ca2ef0d0fd710931a621cac2d03226889658af2
2023-12-04x86-64: Implement Unsafe.getAndAdd/-Set intrinsics. Vladimir Marko
Test: m test-art-host-gtest Test: testrunner.py --host --optimizing Bug: 26264765 Bug: 202868177 Change-Id: I1c7127cf970055dcc176f274d9c2f32c4ed77ac7
2023-11-28Implement getByte/putByte intrinsics Santiago Aboy Solanes
Note that in Android we don't use the volatile, release, or acquire versions for get/putByte. Bug: 313284159 Test: art/test/testrunner/testrunner.py --target --64 --optimizing Change-Id: Ic0953db52e2721f90e81bb600eaa95395941f578
2023-11-28Clean up duplicated intrinsics code. Vladimir Marko
Test: testrunner.py --host --optimizing Change-Id: Ie985bb26e9e834f017a1209763e9b592056e7864
2023-11-23Update `JdkUnsafe` intrinsics. Vladimir Marko
The `jdk.internal.misc.Unsafe.getObject*()` methods have been replaced with `getReference*()` and the old methods just forward to the new ones for backward compatibility. Update ART to pick up the new methods as intrinsics to support code that references them directly. Similarly update intrinics for other deprecated `JdkUnsafe.*Object` methods. Test: m test-art-host-gtest Test: testrunner.py --host --optimizing Bug: 202868177 Change-Id: Ib07e6aa54c67d13079d632efabe4de2185694f4b
2023-11-22Add intrinsic support for CompareAndSetReference Santiago Aboy Solanes
It replaced CompareAndSetObject in aosp/2319000 Bug: 310002432 Fixes: 310002432 Test: art/test/testrunner/testrunner.py --host --64 --optimizing -b Change-Id: I4cbea0c9c4ece8e34b8d31d8f0d8fa76da3e0793
2023-11-22Address review comments. Nicolas Geoffray
For https://android-review.googlesource.com/2835572 Test: test.py Change-Id: Ia1584de58fe945a46b8f49548e062ce212be0a23
generated by cgit v1.2.3-59-g8ed1b (git 2.39.0) at 2025-10-20 09:28:18 +0000