Skip to content

vllm.logprobs

PromptLogprobs module-attribute 

PromptLogprobs = list[dict[int, Logprob] | None]

SampleLogprobs module-attribute 

SampleLogprobs = list[dict[int, Logprob]]

Logprob dataclass

Infos for supporting OpenAI compatible logprobs and token ranks.

Attributes:

Name Type Description
logprob float

The logprob of chosen token
rank int | None

The vocab rank of chosen token (>=1)
decoded_token str | None

The decoded chosen token index
Source code in vllm/logprobs.py 
 9
10
11
12
13
14
15
16
17
18
19
20
21
@dataclass
class Logprob:
    """Infos for supporting OpenAI compatible logprobs and token ranks.

    Attributes:
        logprob: The logprob of chosen token
        rank: The vocab rank of chosen token (>=1)
        decoded_token: The decoded chosen token index
    """

    logprob: float
    rank: int | None = None
    decoded_token: str | None = None

decoded_token class-attribute instance-attribute 

decoded_token: str | None = None

logprob instance-attribute 

logprob: float

rank class-attribute instance-attribute 

rank: int | None = None

__init__ 

__init__(
    logprob: float,
    rank: int | None = None,
    decoded_token: str | None = None,
) -> None