AsynchronousCondition and async_condition() questions

Hello,

I am exploring AsynchronousCondition and MemoryAvailableCondition for an operator that have written. In this operator the asynchronous event is caused by a thread which is waiting for data. When the data arrives the state of the AsynchronousCondition is set to EVENT_DONE. When the data is processed in compute(), it is emitted to a downstream operator and the AsynchronousCondition is set to EVENT_WAITING. When a downstream operator returns buffers they go back to the upstream operator’s BlockMemoryPool which is associated with the MemoryAvailableCondition.

I would like the upstream operator’s compute() to be called under the situation: AsynchronousCondition OR MemoryAvailableCondition. In this regard a couple questions I have are:

  1. Can an operator have an AsynchronousCondition in addition to the builtin AsynchronousCondition returned by async_condition() ?

  2. Can the underlying Condition of async_condition() be used in an ORCombiner ?

  3. When buffers are returned to a BlockMemoryPool, does it cause the operator’s compute() to be called?

Thanks,

Vangogh

Hi @vangogh,

  1. Can an operator have an AsynchronousCondition in addition to the builtin AsynchronousCondition returned by async_condition()?

Yes, you can create additional AsynchronousCondition objects in the compose() method or in the operator’s initialization method (Operator::initialize()).

Note that the async_condition() method is only available after the operator has been initialized by the executor via run() or run_async(). If it’s called within Application::compose(), it will return nullptr (or None in Python).

  1. Can the underlying Condition of async_condition() be used in an ORCombiner?

Since the underlying Condition of async_condition() is not accessible in the compose() method and becomes available only after the operator’s initialization (e.g., after Operator::initialize() is called), it’s not possible to use the built-in AsynchronousCondition with ORCombiner. However, I believe you can use a separate AsynchronousCondition backed by a BlockMemoryPool with an ORCombiner.

  1. When buffers are returned to a BlockMemoryPool, does it cause the operator’s compute() to be called?

Yes, that’s correct.

Thank you for your response. I was thinking of creating the ORCombiner object in my_operator::initialize(), after it has called Operator::initialize(). Thank you for clarifying this. I will also experiment with a separate AsynchronousCondition object.

Hi @Gigony,

Thank you for the explanation. With the insight you offered I was able to implement an operator. I am able to use the operator under greedy and event based schedulers.

  1. With event based scheduler I am observing the following warning message.

[warning] [event_based_scheduler.cpp:156] Deadlock detected after dispatch due to external event

This is not stopping operation and output of this message is prevented by setting stop_on_deadlock to false. It is not quite clear what to make of this message at this time.

Upstream operator should be scheduled based on ((completion_asynch OR MemoryAvailable) AND asynch_event() AND boolean_condition)

asynch_event() is in its default state READY, while boolean_condition has done an enable_tick()

Thanks,

vangogh