Getting off the ground: first pipeline using morpheus


I’m trying to run a simple home-made model through the morpheus inference engine. When I’m trying to deploy my model, no output is generated, and while the CLI doesn’t produce error messages, looking at the log file it looks like there is some issue with the inference engine. Any insight would be most welcome.

Here is a detailed walk-through of what I do:

  1. Followed the installation instruction, up to and including the “RUN NLP SID PIPELINE” here,

everything works as it should at this point. I can see the output file, or read from the output kafka topic and it all looks good.

  1. Deleted the sid deployment
    $ helm delete -n $NAMESPACE my-sid

  2. Created my own toy onnx model, that normalizes a 1 dim input;

#in python
scaler = StandardScaler()
X_norm = scaler.fit_transform(df)
initial_type = [(‘float_input’, FloatTensorType([None, 1]))]
onx = convert_sklearn(scaler, initial_types=initial_type)
with open(“model.onnx”, “wb”) as f:

  1. Setup the model into the morpheus model library:
    copied the model to: \opt\morpheus\common\models\my_model\1\model.onnx

created a config file in

my config file reads:

name: “my_model”
platform: “onnxruntime_onnx”
max_batch_size: 32

default_model_filename: “model.onnx”

input [
name: “float_input”
data_type: TYPE_FP32
dims: [ 1 ]
output [
name: “variable”
data_type: TYPE_FP32
dims: [ 1 ]
4. Published and deployed the model:
$kubectl -n $NAMESPACE exec -it deploy/mlflow – bash
(mlflow)$ python scripts/
–model_name my_model
–model_file /common/models/my_model/1/model.onnx
–model_config /common/models/my_model/config.pbtxt
–flavor onnx

(mlflow)$ mlflow deployments create -t triton
–flavor onnx
–name my_model
-m models:/my_model/1
-C “version=1”

  1. created a pipline using helm:

$helm install --set ngc.apiKey="$API_KEY"
–set sdk.args=“morpheus --log_level=DEBUG run
from-file --filename \
inf-triton --force_convert_inputs=True
–model_name=my_model --server_url=ai-engine:8001
serialize --output_type=‘json’
to-file --filename
/common/data/output/output.jsonlines --overwrite”
–namespace $NAMESPACE

…and it doesn’t work… no errors to CLI, but no ouput file either.

so I look at the relevant log:
$kubectl -n $NAMESPACE logs sdk-cli-my-model-release

and notice something that looks like a some left over labeling file?:

Configuring Pipeline via CLI
Loaded labels file. Current labels: [[‘address’, ‘bank_acct’, ‘credit_card’, ‘email’, ‘govt_id’, ‘name’, ‘password’, ‘phone_num’, ‘secret_keys’, ‘user’]]
Starting pipeline via CLI… Ctrl+C to Quit
“ae”: null,
“class_labels”: [

All this looks like stuff from the now deleted sid model.

Did I miss a step? Any help would be great.


Hi Tzahi,

Can we get some details first please? Specifically, which version of the morpheus container are you using?
What you are seeing is the default labels file implicitly loaded by morpheus, thus it is relevant to the provided SID ONNX model for EA.


Hi Pete,

Thanks for the quick reply,

From my EC2 instance where everything is set up, when I run
$ helm list -n $NAMESPACE

I get the following:

There should be no labels for my model, I think; I didn’t tokenize anything to make it (its just a scikit-learn standardScaler). Does the engine load some defaults and then ignores them?