Airflow addition (#28)

* introducing airflow to run pipeline

* chore: updating dag with upload to registry

* introducing complete provider (non refactored and noisy)

* chore: removing old shit

* generic pricing baselines

* feature: super simple model registry (to be updated maybe third party OS software)

* chore: refactoring the providers docker config and requirements

* chore: refactored and broke down components (braking

* exporting all

* local pipeline excution working

* fix: fixing import structures from nonrelativistic

* chore: enables cross comm pickling with fully e2e pipeline compilation

* docs: what the pipeline is like now

* pipelines local running and pipeline high level definition

* cleaning old pipeline and vectorization

* leaked but fixing, not so important

* test: started with pipeline step testing

* chore: cleaning up provider of prices

* test: extra tests wit hsemantic meaning checks

* migrating pricers

* feature: introducing pricing predictors (pricers)

* chore: e2e is done with new pipeline

* extra session feature extraction

* feature: experiemntal sessin pricer and metrics(vibe)

* chore: redefined and connected pricers (#29)
This commit is contained in:
Daniel Alves Rösel
2025-11-29 17:50:16 +01:00
committed by GitHub
parent 2a0e44ab24
commit ad9423bf59
49 changed files with 3642 additions and 619 deletions

View File

@@ -0,0 +1,48 @@
import numpy as np
import pandas as pd
from procesing.pricers.base import PricingFunction
class StaticPricer(PricingFunction):
"""Static pricing: always return fixed base prices"""
def __init__(self, base_prices: np.ndarray = None):
self.base_prices = base_prices
def fit(self, historical_data: pd.DataFrame):
"""Extract base prices from historical data"""
if 'base_price' in historical_data.columns:
self.base_prices = historical_data['base_price'].values
elif 'price' in historical_data.columns:
self.base_prices = historical_data['price'].values
else:
raise ValueError("historical_data must contain 'base_price' or 'price' column")
return self
def predict(self, state_space) -> np.ndarray:
"""Return static base prices regardless of state"""
if self.base_prices is None:
raise ValueError("Must call fit() or provide base_prices in constructor")
return self.base_prices.copy()
class RandomPricer(PricingFunction):
"""Random pricing within bounds (for baseline comparison)"""
def __init__(self, price_min: float = 50.0, price_max: float = 500.0, seed: int = None):
self.price_min = price_min
self.price_max = price_max
self.seed = seed
self.n_products = None
self.rng = np.random.default_rng(seed)
def fit(self, historical_data: pd.DataFrame):
"""Learn number of products"""
self.n_products = len(historical_data)
return self
def predict(self, state_space) -> np.ndarray:
"""Generate random prices"""
if self.n_products is None:
self.n_products = len(state_space.demand)
return self.rng.uniform(self.price_min, self.price_max, size=self.n_products)