TTFT (Time To First Token)

Also: time to first token · first token latency

TTFT is the time from sending a request to receiving the first output token, dominated by prefill and any queueing. It is the latency users feel first, and it is distinct from throughput (tokens/sec), which governs how fast the rest of the response streams once generation starts.

← All terms · Definitions maintained by Mintok. Methodology.