Fix error handling and add a test.

3 jobs for master in 5 minutes and 2 seconds (queued for 2 seconds)