[offload] Add the ability to record and replay IR bitcode modules (#207278)
This PR allows the user to use JIT with kernel record & replay by
expanding the latter to also work with IR bitcode images.
- Add a new field to `DeviceImageTy` for storing the IR image.
- Expand `RecordReplayTy::FileTy` to also include IR bitcode and
associate it with the file extension `.bc`
- Add the `--load-bitcode` command line option to
`llvm-omp-kernel-replay` tool.
---------
Co-authored-by: Giorgi Gvalia <gvalia1 at llnl.gov>
linux: batch DMU reads of non-resident pages in mappedread()
When a range being read has at least one page in the page cache,
zfs_read() routes the whole chunk through mappedread(), which falls
back to a separate dmu_read_uio_dbuf() call for every non-resident
PAGE_SIZE piece. Since cached pages outlive munmap(), a file which
was mapped at some point may sit mostly outside the page cache and
still pay this cost: one DMU call per 4K page instead of one per
chunk, measured in #16031 as a 4-10x sequential read slowdown.
Commit 39be46f43 ("Linux 5.18+ compat: Detect filemap_range_has_page")
fixed the detection side so fully uncached chunks bypass mappedread()
again, but a chunk holding even one resident page still degrades to
page-sized DMU reads for everything else.
Instead of issuing one DMU read per absent page, probe the page cache
with find_get_page() and extend the read over the whole run of
non-resident pages which follows, restoring chunk-sized DMU reads for
the uncached parts of a mapped file.
[30 lines not shown]
[AMDGPU][NFC] Templatise and roundtrip gfx12_asm_vopc.s
This is just an amended version of the corresponding gfx11 test.
Eliminates the current largest contributor into the undesired
delta vs the downstream True16 branch across MC tests.
Reland "[Clang] Enable -Wunused-template under -Wall" (#208001)
Reland of #206123, which was reverted in #207848.
What has changed since:
- flang-rt occurrences are fixed in #207979
- openmp had one more occurrence which is fixed in #207983
- The remaining -Wunused-template hits anywhere in CI logs are the tsan
Go runtime warnings, which are warnings only (buildgo.sh does not use
-Werror on Linux)
Marked as draft until #207979 and #207983 land.
Closes #202945
[flang][cuda] Defer on_device() folding in host copies of OpenACC routines (#208125)
Add a `defer-acc-routines` option to `cuf-function-rewrite`. When set,
`on_device()` is not folded in the host copy of an OpenACC routine (has
`acc.routine_info`, not in a `gpu.module`), because that body is later
cloned into the device routine and would otherwise bake in the host
value (`.false.`). A later run folds each copy in its own context. Calls
already in a `gpu.module` are still folded.
BPF: Remove unnecessary isReg check on phi operand (#208243)
These must be a register. This pass has quite a lot of
defensive code against invalid MIR that should be deleted.
[mlir][linalg] Fix mask rank for masked contiguous `tensor.extract` (#206207)
### Summary
Note: This fix was made mostly by Claude based on a failure case in
IREE. It addresses issue
https://github.com/llvm/llvm-project/issues/206209
When `vectorizeTensorExtract` lowers a `tensor.extract` recognized as a
*contiguous load*, it builds a `vector.transfer_read` whose permutation
map broadcasts the leading iteration dims and only reads the trailing
`min(dstRank, srcRank)` dims of the source. Until now this read was
returned unmasked and masked later by the generic path, which applies a
**full iteration-space identity mask**. When the source rank is smaller
than the loop nest, that mask is over-ranked relative to the
(rank-reduced) read — e.g. a `vector<1x4xi1>` mask on a read whose
inferred mask type is `vector<4xi1>` — and the op fails verification:
```
[65 lines not shown]
[offload][OpenMP] Improve cross-team reduction grid selection
The default cross-team reduction algorithm benefits from larger and
fewer teams. Implement that by using 2 x the default number of threads
and 1/2 x the default number of teams for reduction kernels. This
doesn't change the default total number of threads, it just
redistributes them.
This is a first, rather simple heuristic, derived from (a subset of)
what AOMP does.
The performance benefits I observed for the reduction tests in
https://github.com/ro-i/xteam-test on a gfx942
(c71339705091500f731e2a39f247d2660bacbdce) are up to a few percent, with
no regressions.
Claude assisted with this patch.
[flang] add some missing stackrestore with -fstack-arrays (#208161)
In the StackArray pass that moves array temporaries from the heap to the
stack under -fstack-arrays, when visiting FreeMemOp to insert
stackrestore, the code was not unwrapping converts as done in other
parts of the code leading to the allocation conversion and stacksave
insertion to happen without the emission of the stackrestore.
Reuse the same utility as in the rest of the pass to get consistent
behavior and fix the memory leak.
