[Pycon] [new paper] "Francesco Bruni" - Switching from batch ETL to streaming pipelines

[info a pycon.it]

Title: Switching from batch ETL to streaming pipelines
Duration: 45 (includes Q&A)
Q&A Session: 15
Language: en
Type: Talk

Abstract: "ETL is dead , long live streams" is a stunning presentation title for a talk given few years ago at SF. The idea is pretty simple: what if you extract/transform/load some data as soon it has been generated rather than waiting for the producer process complete?

This talk explores the above idea (along with issues and limits) and a simple implementation which involve Python and Apache Kafka as successful duo in two real-time domain-different applications: monitoring a manufacturing production plant and ingesting geo-analytics.

No previous knowledge is required even if knowing Apache Kafka can be helpful.

Tags: [u'streaming', u'kafka', u'Python', u'etl', u'geospatial', u'industry4.0', u'pipeline', u'industry-applications']