Here is the English translation:
I have some code, and it contains Russian words. If you do not understand them, use a translator. The code is written in Qlik/SQL. The required code is Qlik
The task is simply to rewrite this code in PySpark. If there are any Russian words or Cyrillic text anywhere, they must also be preserved in the code. The results after Qlik and Spark must be identical. Just transfer all the logic to Spark.
If in Qlik some tables are written somewhere and then read back from there again in order not to overload the kernel in Jupyter Notebook and to prevent the server from crashing, do the same thing in Spark as well. For example, you can write to HDFS under a path like /tmp/promo_fixed/ and then whatever is needed there.
Do not use any AI, because it still will not write the code completely correctly I already tried. I cannot send you the tables that exist here, so you can just use some large sample data of your own.
Before sending me the code, run both the Qlik and Spark code on your own data, and if the outputs are identical, the kernel does not die, and everything works correctly, then send it to me. Because with the code I wrote, the kernel died and the server crashed.
This is shown in the photos. You can open and look at the photos there is a report there explaining what the problem was with my Spark code, and there are photos from Open Spark UI.
If you want, I can also make it sound more natural and professional in English, as if you are giving this as a task specification to a developer.

Leave a Reply
You must be logged in to post a comment.