mirror of
https://github.com/velocitatem/PHANTOM.git
synced 2026-05-31 16:43:36 +00:00
ad9423bf59
* introducing airflow to run pipeline * chore: updating dag with upload to registry * introducing complete provider (non refactored and noisy) * chore: removing old shit * generic pricing baselines * feature: super simple model registry (to be updated maybe third party OS software) * chore: refactoring the providers docker config and requirements * chore: refactored and broke down components (braking * exporting all * local pipeline excution working * fix: fixing import structures from nonrelativistic * chore: enables cross comm pickling with fully e2e pipeline compilation * docs: what the pipeline is like now * pipelines local running and pipeline high level definition * cleaning old pipeline and vectorization * leaked but fixing, not so important * test: started with pipeline step testing * chore: cleaning up provider of prices * test: extra tests wit hsemantic meaning checks * migrating pricers * feature: introducing pricing predictors (pricers) * chore: e2e is done with new pipeline * extra session feature extraction * feature: experiemntal sessin pricer and metrics(vibe) * chore: redefined and connected pricers (#29)
24 lines
573 B
Docker
24 lines
573 B
Docker
FROM apache/airflow:2.7.3-python3.11
|
|
|
|
USER root
|
|
|
|
# install system deps if needed
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
build-essential \
|
|
&& apt-get clean \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
USER airflow
|
|
|
|
# copy requirements for pipeline dependencies
|
|
COPY requirements.txt /tmp/requirements.txt
|
|
RUN pip install --no-cache-dir -r /tmp/requirements.txt
|
|
|
|
# install postgres driver and providers
|
|
RUN pip install --no-cache-dir \
|
|
psycopg2-binary \
|
|
apache-airflow-providers-postgres
|
|
|
|
# set airflow home
|
|
ENV AIRFLOW_HOME=/opt/airflow
|