It would be beneficial to introduce more flexible and granular error handling options for executions and data sources.
Current Limitations
For Executions:
Retry
Fail — causes all subsequent steps to fail after the first failure.
For Data Sources:
Fail — stops all processing after the failure.
Continue without data — proceeds with the execution, but incorrectly marks it as successful.
Continue with old data — same as above; the execution is marked successful despite a failure.
The available options would not handle certain scenarios, for instance, a client has 20 different data sources, they need to load all valid sources while being notified about any that fail. However, with the current settings:
If we choose "Continue without data", the execution completes successfully and no alert is triggered.
If we choose "Fail", the failure of one source stops the remaining from loading, which is also undesirable.
Suggested Enhancement:
We propose expanding the failure handling options for Executions, Data Sources, or at least for Data Sources to include:
Fail entire execution if any step fails (current behavior, but configurable)
Retry failed steps
Skip failed step but send error notification, with the choice to:
Continue without data
Continue using old data
This would allow more flexibility for users who need a middle ground between full failure and silent success.
