Skip to main content

AI_ORT_DEFAULT_BATCH_SIZE

fxcp_core::constants

Constant AI_ORT_DEFAULT_BATCH_SIZE 

Source 
pub const AI_ORT_DEFAULT_BATCH_SIZE: usize = 32;
Expand description

Default batch size for ORT embedding inference.