Quant Mania
A production grade, real time data platform for crypto market analysis: streaming ingestion and processing, anomaly and volatility models, orchestration, and a full observability stack.
Outcome
<1s
End to end data latency
In the product
Context
Built for my MSc, Quant Mania pushes past pure ML into data engineering and reliability. The goal was a platform that ingests and processes crypto market data in real time, runs models on it, and is fully observable, the way a real production data system has to be.
Approach
- 01Streamed and processed live trade data in real time with Apache Kafka and Apache Flink.
- 02Ran anomaly detection and volatility prediction models on the stream.
- 03Orchestrated news collection workflows with Apache Airflow, and exposed everything through a FastAPI backend with WebSocket and REST endpoints.
- 04Wrapped the whole platform in a Prometheus, Loki, and Grafana observability stack, which is the real differentiator of the project.
How it works
Ingest
Kafka stream
Real time trade data ingestion
Process
Apache Flink
Stream processing in real time
Model
Anomaly + volatility
Models run directly on the stream
Orchestrate
Airflow
Scheduled news collection workflows
Serve
FastAPI
WebSocket and REST endpoints
Observe
Prometheus + Grafana + Loki
Metrics, logs, and live dashboards
Results
<1s
End to end data latency
+34%
Improvement in model insight accuracy
+55%
Gain in system reliability
Reflection
Quant Mania is the project closest to data engineering and MLOps. Kafka, Flink, Airflow, and a real observability stack is a mature combination, and it is where I learned to think about latency, monitoring, and system reliability rather than just model metrics.