Interface: HasTokenizer<PROMPT>
Type parameters
Name |
---|
PROMPT |
Methods
countPromptTokens
▸ countPromptTokens(prompt
): PromiseLike
<number
>
Parameters
Name | Type |
---|---|
prompt | PROMPT |
Returns
PromiseLike
<number
>
Defined in
packages/modelfusion/src/model-function/generate-text/TextGenerationModel.ts:73
Properties
tokenizer
• tokenizer: BasicTokenizer
| FullTokenizer
Defined in
packages/modelfusion/src/model-function/generate-text/TextGenerationModel.ts:71