One technique to control an errant pipeline is to count the number of pipeline processes after a while. If they haven't finished, then zap them and restart.
In this example code we use ask Bash for its process id (the
$$variable), then use pgrep to find all sub-processes of the script. That is,
pgreptells us all the workflow process IDs. If there are any, then we use the corresponding pkill command to squash them before a restart.
In addition, we use the date command to output a pretty log file, showing what the system is doing at what time.
full post on SO: How to send SIGKILLs to entire pipeline?