ONNX Runtime is required for Lantern CLI Embeddings.

​
Installation

1

Get ONNX Runtime library

Get ONNX Runtime library from Github release page

PLATFORM=linux
ARCH=x64
ONNX_VERSION=1.16.1

wget "https://github.com/microsoft/onnxruntime/releases/download/v${ONNX_VERSION}/onnxruntime-${PLATFORM}-${ARCH}-${ONNX_VERSION}.tgz"

If you have GPU, download the GPU version of onnxruntime library.

2

Extract archive

Extract the downloaded archive in /usr/local/lib

  mkdir -p /usr/local/lib
  tar xzf "onnxruntime-${PLATFORM}-${ARCH}-${ONNX_VERSION}.tar" -C /usr/local/lib
3

Setup environment

Export path to onnx library

  export ORT_STRATEGY=system
  export ORT_DYLIB_PATH=/usr/local/lib/onnxruntime-${PLATFORM}-${ARCH}-${ONNX_VERSION}/lib/libonnxruntime.so

On MacOS the library will have .dylib extension instead of .os You can export this variables from your shell profile or add them to ld search path like this:

echo "/usr/local/lib/onnxruntime-${PLATFORM}-${ARCH}-${ONNX_VERSION}/lib/libonnxruntime.so" > /etc/ld.so.conf.d/onnx.conf
ldconfig

Was this page helpful?