pto.tgemv_bias

pto.tgemv_bias is part of the Matrix And Matrix Vector instruction set.

Summary

GEMV with bias add.

Mechanism

Tile-based GEMV with bias add. It operates on tile payloads rather than scalar control state, and its legality is constrained by tile shape, layout, valid-region, and target-profile support.

Let:

  • M = 1
  • K = bMatrix.GetValidRow()
  • N = bMatrix.GetValidCol()

For 0 <= j < N (adds a bias term to the matrix product):

\[ \mathrm{C}_{0,j} = \mathrm{Bias}_{0,j} + \sum_{k=0}^{K-1} \mathrm{A}_{0,k} \cdot \mathrm{B}_{k,j} \]

Accumulator behavior and datatype promotion are concrete per target. On A2/A3: accumulation uses the accumulator tile's native datatype (int32_t or float), with int8 accumulation performed in 32-bit and fp accumulation using standard IEEE round-to-nearest-even; bias is added after the final accumulation round. On A5: accumulation is always in the accumulator tile's native type, fp accumulation follows the accumulator's native rounding mode, and bias is added after accumulation using the accumulator's rounding mode. On CPU simulator: follows A5 semantics.

Syntax

Textual spelling is defined by the PTO ISA syntax-and-operands pages.

Synchronous form:

%acc = tgemv.bias %a, %b, %bias : (!pto.tile<...>, !pto.tile<...>, !pto.tile<...>) -> !pto.tile<...>

AS Level 1 (SSA)

%c = pto.tgemv.bias %a, %b, %bias : (!pto.tile<...>, !pto.tile<...>, !pto.tile<...>) -> !pto.tile<...>

AS Level 2 (DPS)

pto.tgemv.bias ins(%a, %b, %bias : !pto.tile_buf<...>, !pto.tile_buf<...>, !pto.tile_buf<...>) outs(%c : !pto.tile_buf<...>)

C++ Intrinsic

Declared in include/pto/common/pto_instr.hpp:

template <typename TileRes, typename TileLeft, typename TileRight, typename TileBias, typename... WaitEvents>
PTO_INST RecordEvent TGEMV_BIAS(TileRes &cMatrix, TileLeft &aMatrix, TileRight &bMatrix, TileBias &biasData, WaitEvents &... events);

template <AccPhase Phase, typename TileRes, typename TileLeft, typename TileRight, typename TileBias,
          typename... WaitEvents>
PTO_INST RecordEvent TGEMV_BIAS(TileRes &cMatrix, TileLeft &aMatrix, TileRight &bMatrix, TileBias &biasData, WaitEvents &... events);

Inputs

  • a is the left operand tile (must be TileLeft location).
  • b is the right operand tile (must be TileRight location).
  • bias is the bias tile (must be TileType::Bias, single row).
  • dst names the destination accumulator tile. The operation iterates over dst's valid region.

Expected Outputs

dst holds the biased matrix-vector product: dst[0,j] = bias[0,j] + sum over k of a[0,k] * b[k,j].

Side Effects

No architectural side effects beyond producing the destination tile. Does not implicitly fence unrelated traffic.

Constraints

Constraints

Common shape and location constraints

  • Static shape constraints:

    • TileLeft::Rows == TileRes::Rows
    • TileLeft::Cols == TileRight::Rows
    • TileRight::Cols == TileRes::Cols

  • Tile locations:

    • TileLeft::Loc == Left
    • TileRight::Loc == Right
    • TileRes::Loc == Acc

  • Runtime valid-size constraints:

    • m must be 1
    • k and n (taken from bMatrix.GetValidRow() and bMatrix.GetValidCol()) must be in [1, 4095]

Datatype constraints

  • Bias tile datatype must exactly match TileRes::DType.

  • Bias tile must be configured as a single row.

  • Bias tile location must be TileType::Bias.

Exceptions

Exceptions

  • Illegal operand tuples, unsupported types, invalid layout combinations, or unsupported target-profile modes are rejected by the verifier or by the selected backend instruction set.
  • Programs must not rely on behavior outside the documented legal domain of this operation, even if one backend currently accepts it.

Target-Profile Restrictions

Target-Profile Restrictions

  • Implementation checks (A2A3):

    • Supported (CType, AType, BType) triples:
      • (int32_t, int8_t, int8_t)
      • (float, half, half)
      • (float, float, float)
      • (float, bfloat16_t, bfloat16_t)

  • Implementation checks (A5):

    • Accumulator type must be int32_t or float.
    • If int32_t: AType == int8_t and BType == int8_t.
    • If float: supports half, bfloat16_t, float, and selected fp8 pairs (target-defined).
    • Fractal/layout constraints are enforced:
      • Left: Loc == Left, !isRowMajor, SFractal == RowMajor
      • Right: Loc == Right, isRowMajor, SFractal == ColMajor
      • Acc: Loc == Acc, !isRowMajor, SFractal == RowMajor

Bias-specific constraints

  • Additional A5 note:
    • No separate explicit m/k/n runtime assertions are enforced in the underlying A5 matmul implementation beyond the GEMV contract described above.

Examples

Auto

#include <pto/pto-inst.hpp>

using namespace pto;

void example_auto() {
  using A = TileLeft<half, 1, 16>;
  using B = TileRight<half, 16, 16>;
  using Bias = Tile<TileType::Bias, half, 1, 16>;
  using C = TileAcc<float, 1, 16>;
  A a;
  B b;
  Bias bias;
  C c;
  TGEMV_BIAS(c, a, b, bias);
}

Manual

#include <pto/pto-inst.hpp>

using namespace pto;

void example_manual() {
  using A = TileLeft<half, 1, 16>;
  using B = TileRight<half, 16, 16>;
  using Bias = Tile<TileType::Bias, half, 1, 16>;
  using C = TileAcc<float, 1, 16>;
  A a;
  B b;
  Bias bias;
  C c;
  TASSIGN(a, 0x1000);
  TASSIGN(b, 0x2000);
  TASSIGN(bias, 0x3000);
  TASSIGN(c, 0x4000);
  TGEMV_BIAS(c, a, b, bias);
}

Auto Mode

# Auto mode: compiler/runtime-managed placement and scheduling.
%c = pto.tgemv.bias %a, %b, %bias : (!pto.tile<...>, !pto.tile<...>, !pto.tile<...>) -> !pto.tile<...>

Manual Mode

# Manual mode: bind resources explicitly before issuing the instruction.
# Optional for tile operands:
# pto.tassign %arg0, @tile(0x1000)
# pto.tassign %arg1, @tile(0x2000)
%c = pto.tgemv.bias %a, %b, %bias : (!pto.tile<...>, !pto.tile<...>, !pto.tile<...>) -> !pto.tile<...>

PTO Assembly Form

%acc = tgemv.bias %a, %b, %bias : (!pto.tile<...>, !pto.tile<...>, !pto.tile<...>) -> !pto.tile<...>
# AS Level 2 (DPS)
pto.tgemv.bias ins(%a, %b, %bias : !pto.tile_buf<...>, !pto.tile_buf<...>, !pto.tile_buf<...>) outs(%c : !pto.tile_buf<...>)