Old job files shouldn't be left on nodes removed from a cluster.

Reviewed-by: iustinp