Executor

Receives from Planner new drifts and send to a custom Sink. It can be a Apache Kafka, RabbitMQ, API, and so on…

Retry Config

class driftage.executor.retry_config.RetryConfig(send_timeout: Union[int, float, None] = 1.0, retry_backoff: Union[int, float] = 1.0, max_tries: int = 3, all_tries_timeout: Union[int, float, None] = None, retry_exceptions: Tuple[Exception] = (<class 'Exception'>, ))

Retry configuration for Sink connection. Configuring this retry it will be resilient when sending data to Sink.

Parameters

  • send_timeout (Optional[Union[int, float]], optional) – All timeouts when send in seconds, defaults to 1.0

  • retry_backoff (Union[int, float], optional) – Time to wait to another try in seconds, defaults to 1.0

  • max_tries (int, optional) – Maximum number of tries, defaults to 3

  • all_tries_timeout (Optional[Union[int, float]], optional) – Total timeout from all retries in seconds, defaults to None

  • retry_exceptions (Tuple[Exception], optional) – Exceptions that we should take in account to retry, defaults to (Exception,)

Sink

class driftage.executor.sink.Sink(circuit_breaker: aiobreaker.circuitbreaker.CircuitBreaker = <aiobreaker.circuitbreaker.CircuitBreaker object>, is_available_cache_ttl: Union[int, float] = 1.0, retry_config: driftage.executor.retry_config.RetryConfig = <driftage.executor.retry_config.RetryConfig object>)

Sink base class to implement custom Sinks like Kafka, RabbitMQ MariaDB, or even an API.

Parameters

  • circuit_breaker (CircuitBreaker, optional) – Circuit breaker to protect Sink if it’s down, defaults to CircuitBreaker()

  • is_available_cache_ttl (Union[int, float], optional) – Healthcheck cache means the time to is_available method in how many seconds, defaults to 1.0

  • retry_config (RetryConfig, optional) – Retry configuration to send data to Sink, defaults to RetryConfig()

abstract async drain(data: dict)

Method that sends data to the Sink. This receives predicted data from the Planner and sends it out.

Parameters

data (dict) – Predicted data with timestamp, predicted and identifier

Raises

NotImplementedError – Needs to be implemented when overridden

abstract is_available() → bool

Healthcheck function to know if Sink is available to receive data.

Raises

NotImplementedError – Needs to be implemented when overridden

Returns

True if it is available or False if not

Return type

bool