Automating Data Processing and Quality Assurance with Python and Dataiku for +2M Monthly Distribution Items

Posted by TarepSH

I automated the collection, processing, analysis, visualisation and sharing of data from more than 100 warehouses, covering over a million distributed items for more than 3.5 million people in Syria. The pipeline was built with Dataiku and Python.

Some of the code is public: Dataiku auto warehousing and distribution.

SARC Data Warehouse — automating monthly data for 100+ warehouses — first page Tap to open the PDF
SARC Data Warehouse — automating monthly data for 100+ warehouses · 23 pages Open in new tab Download PDF