Tuesday, May 3, 2016

Luigi fails to finish all tasks listed in the require method

By Hường Hana 7:30 PM Leave a Comment

Say I have a task with the following dependency structure

class ParentTask(luigi.Task):     def requires(self):         return [ChildTask(classLevel=x) for x in self.class_level_list]     def run(self):         yadayda

The child task runs fine on it own. The parent correctly checks all the children tasks for finish status. Yet when the first child task finishes, the scheduler mark the parent task as finished. with the following message:

   Scheduled 15 tasks of which: * 3 ran successfully:     - 1 CleanRecord(...)     - 1 EstimateQuestionParameter(classLevel=6, qdt=2016-04-19, subject=english)     - 1 GetLog(classLevel=6, qdt=2016-04-19, subject=english) * 12 were left pending, among these:     * 12 were left pending because of unknown reason:         - 5 EstimateQuestionParameter(classLevel=1...5, qdt=2016-04-19, subject=english)         - 5 GetLog(pool=None, classLevel=1...5, qdt=2016-04-19, subject=english)         - 1 UpdateQuestionParameter(qdt=2016-04-19, lastQdt=2016-03-23, subject=english, isInit=False)         - 1 UpdateQuestionParameterBuffer(qdt=2016-04-19, subject=english, src_table=edw.edw_behavior_question_record_exam_new)  This progress looks :) because there were no failed tasks or missing external dependencies

0 Answers

If You Enjoyed This, Take 5 Seconds To Share It

0 comments:

Post a Comment