dagster

<div align="center">
  <!-- Note: Do not try adding the dark mode version here with the `picture` element, it will break formatting in PyPI -->
  <div>
    <a target="_blank" href="https://dagster.io" style="background:none">
      <img alt="dagster logo" src="https://raw.githubusercontent.com/dagster-io/dagster/master/.github/dagster-logo-light.svg" height="72">
    </a>
  </div>
  <a target="_blank" href="https://github.com/dagster-io/dagster" style="background:none">
    <img src="https://img.shields.io/github/stars/dagster-io/dagster?labelColor=4F43DD&color=163B36&logo=github">
  </a>
  <a target="_blank" href="https://github.com/dagster-io/dagster/blob/master/LICENSE" style="background:none">
    <img src="https://img.shields.io/badge/License-Apache_2.0-blue.svg?label=license&labelColor=4F43DD&color=163B36">
  </a>
  <a target="_blank" href="https://pypi.org/project/dagster/" style="background:none">
    <img src="https://img.shields.io/pypi/v/dagster?labelColor=4F43DD&color=163B36">
  </a>
  <a target="_blank" href="https://pypi.org/project/dagster/" style="background:none">
    <img src="https://img.shields.io/pypi/pyversions/dagster?labelColor=4F43DD&color=163B36">
  </a>
  <a target="_blank" href="https://twitter.com/dagster" style="background:none">
    <img src="https://img.shields.io/badge/twitter-dagster-blue.svg?labelColor=4F43DD&color=163B36&logo=twitter" />
  </a>
  <a target="_blank" href="https://dagster.io/slack" style="background:none">
    <img src="https://img.shields.io/badge/slack-dagster-blue.svg?labelColor=4F43DD&color=163B36&logo=slack" />
  </a>
  <a target="_blank" href="https://linkedin.com/showcase/dagster" style="background:none">
    <img src="https://img.shields.io/badge/linkedin-dagster-blue.svg?labelColor=4F43DD&color=163B36&logo=linkedin" />
  </a>
</div>

**Dagster is a cloud-native data pipeline orchestrator for the whole development lifecycle, with integrated lineage and observability, a declarative programming model, and best-in-class testability.**

It is designed for **developing and maintaining data assets**, such as tables, data sets, machine learning models, and reports.

With Dagster, you declare—as Python functions—the data assets that you want to build. Dagster then helps you run your functions at the right time and keep your assets up-to-date.

Here is an example of a graph of three assets defined in Python:

```python
import dagster as dg
import pandas as pd

from sklearn.linear_model import LinearRegression


@dg.asset
def country_populations() -> pd.DataFrame:
    df = pd.read_html("https://tinyurl.com/mry64ebh")[0]
    df.columns = ["country", "pop2022", "pop2023", "change", "continent", "region"]
    df["change"] = df["change"].str.rstrip("%").astype("float")
    return df


@dg.asset
def continent_change_model(country_populations: pd.DataFrame) -> LinearRegression:
    data = country_populations.dropna(subset=["change"])
    return LinearRegression().fit(pd.get_dummies(data[["continent"]]),