[AMDGPU][GISel] RegBankLegalize rules for the QSAD intrinsics
llvm.amdgcn.{qsad.pk.u16.u8,mqsad.pk.u16.u8,mqsad.u32.u8} all reached
AMDGPURegBankLegalize without a rule and failed with "No AMDGPU
RegBankLegalize rules defined for opcode".
The two pk forms share a shape - an i64 result from an i64, an i32 and an i64,
all in VGPRs - so they get one rule. mqsad_u32_u8 returns <4 x i32> and takes
a <4 x i32> accumulator, which has no slot in the Standard fast rules, so it
uses explicit Any rules on Uni/DivV4S32 instead.
Convert the lit tests to autogenerated checks and run them through both
SelectionDAG and GlobalISel.
Co-Authored-By: Claude Opus 5 (1M context) <noreply at anthropic.com>
[AMDGPU][GISel] RegBankLegalize rules for the SAD intrinsics
llvm.amdgcn.{sad.u8,sad.hi.u8,sad.u16,msad.u8} all reached
AMDGPURegBankLegalize without a rule and failed with "No AMDGPU
RegBankLegalize rules defined for opcode". They share one shape - an i32
result from three i32 sources, with the clamp operand of the underlying VOP3
left alone - so give them a single rule mapping every source to a VGPR.
Convert the lit tests to autogenerated checks and run them through both
SelectionDAG and GlobalISel.
Co-Authored-By: Claude Opus 5 (1M context) <noreply at anthropic.com>
[llvm][ABI][NFC] Name the required-argument boundary in FunctionInfo (#214100)
`FunctionInfo::create` took the declared-parameter count as a bare
`std::optional<unsigned>`, where an absent value meant the signature has
no ellipsis, because `isVariadic()` was `NumRequired.has_value()`. A
caller who reads that parameter name and passes the real count for a
non-variadic signature makes `isVariadic()` true even though there is no
ellipsis. A reviewer read the parameter that way on
[#213315](https://github.com/llvm/llvm-project/pull/213315) and asked
for this move as a follow-up.
`RequiredArgs` moves out of `CallConvLoweringPass` and into the library,
so both producers name the case they mean instead of encoding it. This
is the same shape as clang's `RequiredArgs` in `CGFunctionInfo.h`, minus
the members that would have no caller here. The x86_64 classifier in
`llvm/lib/ABI/Targets/X86.cpp` is untouched, since
`getNumRequiredArgs()` keeps its signature.
Assisted-by: Cursor / claude-opus-5
[AMDGPU][GISel] RegBankLegalize rule for amdgcn_fdot2_f32_bf16
llvm.amdgcn.fdot2.f32.bf16 reached AMDGPURegBankLegalize without a rule and
failed with "No AMDGPU RegBankLegalize rules defined for opcode". Its reg-bank
shape is the same as amdgcn_fdot2 - an f32 result from two packed 16-bit
vector sources and an f32 accumulator, with the clamp ImmArg left alone - so
add it to that rule set. V2S16 already matches <2 x bf16>, as the existing
amdgcn_fdot2_bf16_bf16 rule relies on.
Enable GlobalISel in the lit test and regenerate the checks. Source modifiers
(neg_lo/neg_hi) still fold on the GlobalISel path.
Co-Authored-By: Claude Opus 5 (1M context) <noreply at anthropic.com>
[AMDGPU][GISel] RegBankLegalize rule for amdgcn_cvt_pk_u8_f32
llvm.amdgcn.cvt.pk.u8.f32 reached AMDGPURegBankLegalize without a rule and
failed with "No AMDGPU RegBankLegalize rules defined for opcode". It has the
same reg-bank shape as the neighbouring cvt_pk_*_f32 intrinsics - an i32
result from three VGPR sources - so add it to that rule set.
Convert the lit test to autogenerated checks and run it through both
SelectionDAG and GlobalISel. The byte-index inline constants still fold into
src1, and the gfx803 output is identical between the two paths.
Co-Authored-By: Claude Opus 5 (1M context) <noreply at anthropic.com>
[CIR] Update get_global type for rewritten callees (#214082)
When the pass rewrites a callee's signature, any `cir.get_global`
holding that function's address is left behind at the old type. The
verifier ties a get_global's pointee to the symbol it names, so the
module stops verifying. Returning the address of a function that returns
a 32-byte struct is enough to hit it.
Each address now gets retyped to whatever signature its callee ended up
with, then cast back so the existing uses still see the type they were
built for. That happens in the same iteration that rewrites the callee,
so the module is
only ever inconsistent for one function at a time.
Assisted-by: Cursor / claude-opus-5
[AMDGPU][GISel] RegBankLegalize rule for amdgcn_lerp
llvm.amdgcn.lerp reached AMDGPURegBankLegalize without a rule and failed with
"No AMDGPU RegBankLegalize rules defined for opcode". V_LERP_U8 is a plain
VOP3 producing an i32 from three i32 sources, so map all of them to VGPRs and
read a uniform result back with UniInVgprS32.
Convert the lit test to autogenerated checks, run it through both
SelectionDAG and GlobalISel, and add a non-immediate case so the all-register
operand form is covered too.
Co-Authored-By: Claude Opus 5 (1M context) <noreply at anthropic.com>
[AMDGPU][GISel] RegBankLegalize rule for amdgcn_alignbyte
llvm.amdgcn.alignbyte reached AMDGPURegBankLegalize without a rule and failed
with "No AMDGPU RegBankLegalize rules defined for opcode". Its shape matches
amdgcn_perm exactly - i32 result from three VGPR i32 sources - so add it to
that rule set. All V_ALIGNBYTE_B32 selection patterns, including the true16
one that extracts lo16 from src2, take a VGPR_32 src2, so a single mapping
covers every subtarget.
Enable GlobalISel in the lit test and regenerate the checks.
Co-Authored-By: Claude Opus 5 (1M context) <noreply at anthropic.com>
bhyve: Fix assignment of *niov2 in split_iov()
niov2 returns the number of entries in the iovec starting at offset
"offset". Here we are unconditionally setting it to 1, which of course
isn't right.
Fixes: a28cf86c4171 ("bhyve/virtio: Rework iovec handling functions for efficiency and clarity")
Reported by: Claude and Ada Logics
Reviewed by: Hans Rosenfeld <rosenfeld at grumpf.hope-2000.org>
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D58625
[AMDGPU][GISel] RegBankLegalize rules for s_incperflevel/s_decperflevel
llvm.amdgcn.s.{inc,dec}perflevel reached AMDGPURegBankLegalize without a rule
and failed with "No AMDGPU RegBankLegalize rules defined for opcode". Their
only argument is an ImmArg, so like s_setprio and s_sleep they belong in the
existing no-register-operand intrinsic rule set.
Enable GlobalISel in the corresponding lit tests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply at anthropic.com>
[MLIR][XeGPU] Fix XeGPUToXeVM crash on non-integer memref memory spaces (#211053)
## Summary
`convert-xegpu-to-xevm` asserts/crashes when a memref's memory space
isn't an
`IntegerAttr` — e.g. `memref<1024xf32,
#spirv.storage_class<StorageBuffer>>`.
`LoadStoreToXeVMPattern` and `PrefetchToXeVMPattern` compute the LLVM
pointer
address space via the deprecated `MemRefType::getMemorySpaceAsInt()`,
which
asserts on anything but an integer. Memref memory spaces aren't
integer-only
in general (GPU address spaces and XeVM's own `xevm::AddrSpaceAttr` are
both
legal here) — the file's own `isSharedMemRef` already handles this
correctly
elsewhere, these three call sites just didn't.
[30 lines not shown]
ctfmerge.1: Fix uniqlabel typos
The flag is -D, but it was written as a second -d. Add a period too.
MFC after: 3 days
(cherry picked from commit 4f293e32e4529617dd05bd64fd3c22a57a56a355)
ctfmerge.1: Import ENVIRONMENT from NetBSD
Import the ENVIRONMENT section from NetBSD, minus the variable that our
ctfmerge does not have. Alphabetize them, polish grammar and alignment,
and add the variables to the man database. While here, remove whitespace
from the end of some lines to quiet linter.
MFC after: 3 days
PR: 291186
Co-authored-by: Alexander Ziaee <ziaee at FreeBSD.org>
Obtained from: NetBSD (christos <christos at NetBSD.org>, 8a0c0d8)
Differential Revision: https://reviews.freebsd.org/D54054
(cherry picked from commit 32cf24b725fdf899fb642c47004b69fcfae9b9db)
[CodeGen] Fix -fsanitize=array-bounds for __sized_by / _or_null pointers
`EmitCountedByBoundsChecking()` assumed a CountAttributedType is always
a __counted_by pointer. That isn't true, there are four versions of the
attribute:
* `__counted_by`: Already handled correctly.
* `__counted_by_or_null`: Incorectly handled.
* `__sized_by`: Incorreclty handled.
* `__sized_by_or_null`: Incorreclty handled.
In particular:
* __sized_by / __sized_by_or_null: the loaded bound is a byte count, but the
element index was compared against it directly, so an access was only
flagged once the index exceeded the byte count -- missing out-of-bounds
accesses for a pointee larger than one byte. Scale the index to bytes
('index * sizeof(element)') before comparing. counted_by counts elements
and is unchanged; a void (or otherwise zero-sized) pointee uses the GNU
[15 lines not shown]
