I am exploring AsynchronousCondition and MemoryAvailableCondition for an operator that have written. In this operator the asynchronous event is caused by a thread which is waiting for data. When the data arrives the state of the AsynchronousCondition is set to EVENT_DONE. When the data is processed in compute(), it is emitted to a downstream operator and the AsynchronousCondition is set to EVENT_WAITING. When a downstream operator returns buffers they go back to the upstream operator’s BlockMemoryPool which is associated with the MemoryAvailableCondition.
I would like the upstream operator’s compute() to be called under the situation: AsynchronousCondition OR MemoryAvailableCondition. In this regard a couple questions I have are:
Can an operator have an AsynchronousCondition in addition to the builtin AsynchronousCondition returned by async_condition() ?
Can the underlying Condition of async_condition() be used in an ORCombiner ?
When buffers are returned to a BlockMemoryPool, does it cause the operator’s compute() to be called?
Can an operator have an AsynchronousCondition in addition to the builtin AsynchronousCondition returned by async_condition()?
Yes, you can create additional AsynchronousCondition objects in the compose() method or in the operator’s initialization method (Operator::initialize()).
Note that the async_condition() method is only available after the operator has been initialized by the executor via run() or run_async(). If it’s called within Application::compose(), it will return nullptr (or None in Python).
Can the underlying Condition of async_condition() be used in an ORCombiner?
Since the underlying Condition of async_condition() is not accessible in the compose() method and becomes available only after the operator’s initialization (e.g., after Operator::initialize() is called), it’s not possible to use the built-inAsynchronousCondition with ORCombiner. However, I believe you can use a separate AsynchronousCondition backed by a BlockMemoryPool with an ORCombiner.
When buffers are returned to a BlockMemoryPool, does it cause the operator’s compute() to be called?
Thank you for your response. I was thinking of creating the ORCombiner object in my_operator::initialize(), after it has called Operator::initialize(). Thank you for clarifying this. I will also experiment with a separate AsynchronousCondition object.
Thank you for the explanation. With the insight you offered I was able to implement an operator. I am able to use the operator under greedy and event based schedulers.
With event based scheduler I am observing the following warning message.
[warning] [event_based_scheduler.cpp:156] Deadlock detected after dispatch due to external event
This is not stopping operation and output of this message is prevented by setting stop_on_deadlock to false. It is not quite clear what to make of this message at this time.
Upstream operator should be scheduled based on ((completion_asynch OR MemoryAvailable) AND asynch_event() AND boolean_condition)
asynch_event() is in its default state READY, while boolean_condition has done an enable_tick()