TGEMV_BIAS¶

Tile Operation Diagram¶

Introduction¶

Tile-based GEMV with bias add.

See also¶

• Base GEMV instruction: docs/isa/TGEMV.md.
• Accumulation variant: docs/isa/TGEMV_ACC.md.

C++ Intrinsic¶

Declared in include/pto/common/pto_instr.hpp:

template <typename TileRes, typename TileLeft, typename TileRight, typename TileBias, typename... WaitEvents>
PTO_INST RecordEvent TGEMV_BIAS(TileRes &cMatrix, TileLeft &aMatrix, TileRight &bMatrix, TileBias &biasData, WaitEvents &... events);

template <AccPhase Phase, typename TileRes, typename TileLeft, typename TileRight, typename TileBias,
          typename... WaitEvents>
PTO_INST RecordEvent TGEMV_BIAS(TileRes &cMatrix, TileLeft &aMatrix, TileRight &bMatrix, TileBias &biasData, WaitEvents &... events);

Math Interpretation¶

Let:

• M = 1
• K = bMatrix.GetValidRow()
• N = bMatrix.GetValidCol()

For 0 <= j < N (adds a bias term to the matrix product):

Note: Exact accumulator behavior and datatype promotion are target/implementation-defined.

Assembly Syntax¶

PTO-AS form: see PTO-AS Specification.

Synchronous form:

%acc = tgemv.bias %a, %b, %bias : (!pto.tile<...>, !pto.tile<...>, !pto.tile<...>) -> !pto.tile<...>

AS Level 1 (SSA)¶

%c = pto.tgemv.bias %a, %b, %bias : (!pto.tile<...>, !pto.tile<...>, !pto.tile<...>) -> !pto.tile<...>

AS Level 2 (DPS)¶

pto.tgemv.bias ins(%a, %b, %bias : !pto.tile_buf<...>, !pto.tile_buf<...>, !pto.tile_buf<...>) outs(%c : !pto.tile_buf<...>)

Constraints¶

Common shape and location constraints¶

• Static shape constraints:
  • TileLeft::Rows == TileRes::Rows
  • TileLeft::Cols == TileRight::Rows
  • TileRight::Cols == TileRes::Cols
• Tile locations:
  • TileLeft::Loc == Left
  • TileRight::Loc == Right
  • TileRes::Loc == Acc
• Runtime valid-size constraints:
  • m must be 1
  • k and n (taken from bMatrix.GetValidRow() and bMatrix.GetValidCol()) must be in [1, 4095]

Datatype constraints¶

• Implementation checks (A2A3):
  • Supported (CType, AType, BType) triples:
    • (int32_t, int8_t, int8_t)
    • (float, half, half)
    • (float, float, float)
    • (float, bfloat16_t, bfloat16_t)
• Implementation checks (A5):
  • Accumulator type must be int32_t or float.
  • If int32_t: AType == int8_t and BType == int8_t.
  • If float: supports half, bfloat16_t, float, and selected fp8 pairs (target-defined).
  • Fractal/layout constraints are enforced:
    • Left: Loc == Left, !isRowMajor, SFractal == RowMajor
    • Right: Loc == Right, isRowMajor, SFractal == ColMajor
    • Acc: Loc == Acc, !isRowMajor, SFractal == RowMajor

Bias-specific constraints¶

• Bias tile datatype must exactly match TileRes::DType.
• Bias tile must be configured as a single row.
• Bias tile location must be TileType::Bias.
• Additional A5 note:
  • No separate explicit m/k/n runtime assertions are enforced in the underlying A5 matmul implementation beyond the GEMV contract described above.

Examples¶

Auto¶

#include <pto/pto-inst.hpp>

using namespace pto;

void example_auto() {
  using A = TileLeft<half, 1, 16>;
  using B = TileRight<half, 16, 16>;
  using Bias = Tile<TileType::Bias, half, 1, 16>;
  using C = TileAcc<float, 1, 16>;
  A a;
  B b;
  Bias bias;
  C c;
  TGEMV_BIAS(c, a, b, bias);
}

Manual¶

#include <pto/pto-inst.hpp>

using namespace pto;

void example_manual() {
  using A = TileLeft<half, 1, 16>;
  using B = TileRight<half, 16, 16>;
  using Bias = Tile<TileType::Bias, half, 1, 16>;
  using C = TileAcc<float, 1, 16>;
  A a;
  B b;
  Bias bias;
  C c;
  TASSIGN(a, 0x1000);
  TASSIGN(b, 0x2000);
  TASSIGN(bias, 0x3000);
  TASSIGN(c, 0x4000);
  TGEMV_BIAS(c, a, b, bias);
}

ASM Form Examples¶

Auto Mode¶

# Auto mode: compiler/runtime-managed placement and scheduling.
%c = pto.tgemv.bias %a, %b, %bias : (!pto.tile<...>, !pto.tile<...>, !pto.tile<...>) -> !pto.tile<...>

Manual Mode¶

# Manual mode: bind resources explicitly before issuing the instruction.
# Optional for tile operands:
# pto.tassign %arg0, @tile(0x1000)
# pto.tassign %arg1, @tile(0x2000)
%c = pto.tgemv.bias %a, %b, %bias : (!pto.tile<...>, !pto.tile<...>, !pto.tile<...>) -> !pto.tile<...>

PTO Assembly Form¶

%acc = tgemv.bias %a, %b, %bias : (!pto.tile<...>, !pto.tile<...>, !pto.tile<...>) -> !pto.tile<...>
# AS Level 2 (DPS)
pto.tgemv.bias ins(%a, %b, %bias : !pto.tile_buf<...>, !pto.tile_buf<...>, !pto.tile_buf<...>) outs(%c : !pto.tile_buf<...>)