Fink is a broker infrastructure enabling a wide range of applications and services to connect to large streams of alerts issued from telescopes all over the world. Fink core is based on the Apache Spark framework, and more specifically it uses the Structured Streaming processing engine. The language chosen for the API is Python, which is widely used in the astronomy community, has a large scientific ecosystem and easily connects with existing tools.

Fink’s goal is twofold: providing a robust infrastructure and state-of-the-art streaming services to LSST scientists, and enabling user-defined science cases in a big data context. Fink decouples resources needed for listening to the stream (online, critical), and resources used for services: scalable, robust, and modular!

We want Fink to be able to filter, aggregate, enrich, consume incoming data streams or otherwise transform into new streams for further consumption or follow-up processing. Following LSST LDM-612, Fink’s ultimate objectives are (no specific order):

Getting started

Learning Fink is easy whether you are a developer or a scientist: