docs: what the pipeline is like now

This commit is contained in:
2025-11-28 14:06:01 +01:00
parent 33c20ec715
commit bdd72b5a85

View File

@@ -0,0 +1,8 @@
# Products
# Agents
# Pipeline
Our pipeline technically should follow principles in a style like this:
- Each step should be defined as an inheriting child of an scikit pipeline step, the granularity of the steps is dictated by the following: a step should be a transformation, augmentation or computation independently, no single stage should run multiple in-itself. This way we can modularize properly all the components and track properly in airflow. A stage can be defined as an sklearn step but then must be transalted to a function that takes the context in our DAG of airflow. All parametrization must be done via contexts.