Apache Airflow as an external task scheduler for Qlik

Not so long ago, at my new workplace, we asked the question of moving away from old developments.We had self-written services for windows in c#, incomprehensible php scripts, Microsoft iis was used as a web server.

Around this time, I came across an article about apache airflow on the site habr.com. I found the solution interesting, especially since it is written in a simple (I don’t think so now) python programming language.

We did not think long, and I took up the study of apache airflow and at the same time the study of the python programming language.

The first result was obtained after about 2 weeks. I was able to install airflow on debian os, create the first dag that could run tasks in QlikView server.

Within a few months of development, I reached the level of publishing the project on github.com and pipy.org so that anyone can view the code and use my development.

Now you can get acquainted with the project at the link : https://github.com/bintocher/qlikflow, the documentation is located there.

The name “qlikflow” is currently being actively discussed with the legal department of qliktech, because at the first approximation, they forbade me to use this name. I hope that everything will turn out in favor of open source, and I will not have to change the name.

On my own behalf, I want to add the current state of our airflow server.uptime : 47+ days, completed — 27k tasks

Now, the most complex dag we have performs daily more than 100 tasks on 3 servers, sequentially