We consider it a glorified cron replacement. The main selling point is its scheduling feature and the ability to view logs via the web UI it provides.
You write DAGs in Python to do 'stuff', schedule it to run, say, every hour. You can then get a history of its runs, failures, what went wrong. Rerun things if needed.
Those are the pros.
Cons - when new devs try to treat it as a programming paradigm, things can get difficult to work with. Some aspects aren't easily automatable - eg creating users. Needs to make its authentication options obvious and would be good to have some finer grain control over who can do what in the Airflow UI.
Overall we're quite happy with it and also using it for datascience as well as data feeds, data workflows, ETL processes.
There is an attempt to create a role based access control by the guys at WePay according to the shared slide "RBAC talk" below [1].
Don't know why their repo [2] can't be accessed now, though.
You write DAGs in Python to do 'stuff', schedule it to run, say, every hour. You can then get a history of its runs, failures, what went wrong. Rerun things if needed.
Those are the pros.
Cons - when new devs try to treat it as a programming paradigm, things can get difficult to work with. Some aspects aren't easily automatable - eg creating users. Needs to make its authentication options obvious and would be good to have some finer grain control over who can do what in the Airflow UI.
Overall we're quite happy with it and also using it for datascience as well as data feeds, data workflows, ETL processes.