Knowledge Base

Stores data collected and predicted by the Analyser. Can be queried by Analyser for retraining or Planner for predictions.

Connection

Build on top of SQLAlchemy to interact with Knowledge Base.

class driftage.db.connection.Connection(db_engine: sqlalchemy.engine.base.Engine, bulk_size: int, bulk_time: Union[int, float], circuit_breaker: aiobreaker.circuitbreaker.CircuitBreaker = <aiobreaker.circuitbreaker.CircuitBreaker object>)

Connects with SQLAlchemy Engine to store and query data for concept drift datection.

Parameters

  • db_engine (Engine) – SQLAlchemy Engine to use as backend

  • bulk_size (int) – Quantity of data that connection will wait to make bulk insert

  • bulk_time (Union[int, float]) – Time in seconds between last insert and now. If bulk_size is not reached in bulk_time interval, then an insert was done

  • circuit_breaker (CircuitBreaker, optional) – Circuit Breaker configuration to connect with Database, defaults to CircuitBreaker()

async get_between(column: sqlalchemy.sql.schema.Column, from_datetime: datetime.datetime, to_datetime: datetime.datetime) → pandas.core.frame.DataFrame

Collects data between dates from database.

Parameters

  • column (Column) – Database column from schema

  • from_datetime (datetime) – Start Datetime to search

  • to_datetime (datetime) – End Datetime to search

Returns

Data got from specified date range

Return type

pd.DataFrame

async lazy_insert(df: pandas.core.frame.DataFrame)

Insert in database if bulk size or bulk time reached.

Parameters

df (pd.DataFrame) – Data to be inserted

Schema

Schema from data stored and predicted.

Definitions from table where predicted drifts are stored:

Table:

driftage_kb or defined by DRIFTAGE_TABLENAME enviroment variable

Columns:

  • driftage_jid: ID from the Analyser that predicted and saved data

  • driftage_datetime_monitored: Datetime of collected data

  • driftage_datetime_analysed: Datetime of analysed data

  • driftage_identifier: Identifier from the Monitor that collected data

  • driftage_data: Json type object from data collected

  • driftage_predicted: True or False depending on if data is drift or not