Morpheus pipeline (miniconda version) test run failed: ImportError: cannot import name 'ControlMessage'

AI & Data Science Cybersecurity
1

(morpheus) [cjliu@sam9 morpheus]$ morpheus --log_level INFO --plugin “inference” --plugin “postprocessing” run --num_threads 1 --use_cpp False --pipeline_batch_size 1024 --model_max_batch_size 32 pipeline-nlp from-file --filename ./models/datasets/validation-data/log-parsing-validation-data-input.csv deserialize preprocess --vocab_hash_file ./morpheus/data/bert-base-cased-hash.txt --stride 64 --column=raw monitor --description “Preprocessing rate” inf-logparsing --model_name log-parsing-onnx --server_url localhost:8001 --force_convert_inputs=True monitor --description “Inference rate” --unit inf log-postprocess --vocab_path ./models/training-tuning-scripts/sid-models/resources/bert-base-cased-vocab.txt --model_config_path=./models/log-parsing-models/log-parsing-config-20220418.json to-file --filename ./log-parsing-output.jsonlines --overwrite monitor --description “Postprocessing rate”

Configuring Pipeline via CLI
Traceback (most recent call last):
File “/hawes/projects/TSAD-ML/miniconda3-py310/envs/morpheus/bin/morpheus”, line 8, in
sys.exit(run_cli())
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/morpheus/cli/run.py”, line 20, in run_cli
cli(obj={}, auto_envvar_prefix=‘MORPHEUS’, show_default=True, prog_name=“morpheus”)
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/click/core.py”, line 1157, in call
return self.main(*args, **kwargs)
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/click/core.py”, line 1078, in main
rv = self.invoke(ctx)
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/click/core.py”, line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/click/core.py”, line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/click/core.py”, line 1697, in invoke
super().invoke(ctx)
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/click/core.py”, line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/click/core.py”, line 783, in invoke
return __callback(*args, **kwargs)
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/morpheus/cli/utils.py”, line 114, in new_func
return f(ctx, *args, **kwargs)
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/morpheus/cli/commands.py”, line 359, in pipeline_nlp
from morpheus.pipeline import LinearPipeline
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/morpheus/pipeline/init.py”, line 28, in
from morpheus.pipeline.multi_message_stage import MultiMessageStage
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/morpheus/pipeline/multi_message_stage.py”, line 22, in
from morpheus.messages import MultiMessage
File “/home/cjliu/miniconda3/envs/morpheus/lib/python3.10/site-packages/morpheus/messages/init.py”, line 21, in
from morpheus._lib.messages import ControlMessage
ImportError: cannot import name ‘ControlMessage’ from ‘morpheus._lib.messages’ (unknown location)
(morpheus) [cjliu@sam9 ~]$

The miniconda Morpheus was installed following the instructions given in the github.com. What did I miss?
I Google searched for that specific Python package, but could not find any relevant info. Any guidance will be helpful.
Does it need Triton server running in the backgroud?
Thanks.
V/R
Dr. CJ Liu

2

If you are building Morpheus using a Conda environment, please make sure you follow all the steps here:

Particularly the ./scripts/compile.sh step.

\Pete

3

Hi, Pete,
Copy that. Will double check.
Must have missed some step(s).
Thanks.
V/R
CJ Liu, D.Sc.

4

And yes, the instructions also explain how to launch a Triton container which the Morpheus pipeline will communicate with.

5

The available instrictions are for Docker, not for Apptainer/Singularity.
Need to translate Docker run parameters to Apptainer corresponding ones.

6

Hi, Pete,
Successfully re-installed and pytest’ed conda morpheus 24.03 cloned from github.com on an Unbuntu 22.04 laptop with CUDA Toolkit 12.2 and driver 535.129.03 following the instructions provided in:
Morpheus/docs/source/developer_guide/contributing.md at branch-23.11 · nv-morpheus/Morpheus · GitHub – Section “Build in a Conda Environment”: steps 1~7.

“pytest --run_slow” run in step 5 still needs to be addressed to properly configure the camouflage server.

Pipeline run in step 7, I reused the log_parsing CLI example in:
Morpheus/examples/log_parsing/README.md at branch-23.11 · nv-morpheus/Morpheus · GitHub working with Triton docker container already running. It looks like a success. Further validation and understanding to be proceeded.
CJ