To homepage To homepage
Solved

Alerts on validation rules

  • 20 July 2023
  • 5 replies
  • 80 views
devin.tiemens
Rank
Userlevel 2
Badge +2

Hi,

To my knowledge it is not possible to setup alerts for errors or warnings on validation rules.

However, I read the following on the TimeXtender Sales Hub:

"Set up automated alerts to notify you when data quality rules are violated, data quality issues are detected, or executions fail to complete.” (https://coda.io/d/TimeXtender-Sales-Hub_d3yl25w6y3Y/Data-Quality_sueAz#_luZ9c).

Does someone know if this is possible, and if so, how? If not this would be a nice idea though!

 

Kind regards,

Devin

icon

Best answer by daniel 20 July 2023, 12:05

View original

5 replies

daniel
Rank
Userlevel 4
Badge +5

Hey @devin.tiemens ,

To my knowledge this is not possible. What I do is I have a script shih scans all het validation rule tables and takes all the data from those tables. This table I send to Power BI or Qlik and have Power BI or Qlik send me an alert. I use this table as a general Data Quality reporting table, so my end users know which records to correct.

 

Would be nice to have TX send it directly.

rory.smith
Rank
Userlevel 5
Badge +7

Hi,

if you have your _L and _M tables loaded into TX you could set up checkpoints and use execution packages to notify you, but that is a rather technical approach.

It is usually better to approach validity checks as a flag, either in a separate layer if you don't want the quality assessment to be part of the main load or included in the normal flow.

devin.tiemens
Rank
Userlevel 2
Badge +2

Thanks for the ideas and workaround guys! 

Though, wouldn't you agree that, as the text implies, it would be nice to be able to sent messages directly from timextender when a validation rule is violated?

rory.smith
Rank
Userlevel 5
Badge +7

Hi,

while the idea seems intuitive I think it is usually better to treat data quality auditing data just like any other data. Typically you will have many records failing to meet one or more quality checks, any you probably want all of them processed instead of the first record causing an Execution failure.

I would solve Master Data Management issues - which is what these usually are - in solutions that are aimed at solving that. Otherwise you also need to take all the governance and other aspects in, and then you are seeing nails everywhere because you are holding a hammer.

There are other tools that can be integrated into your Data Platform (Exmon, CluedIn, etc.) that can deal with the flow of notifying users, approvals and writing back to line-of-business tools without needing to deal with those aspects in TimeXtender. All you do there is implement the business rules and derive flag values in your dimensions and facts to be used by your MDM tooling in addition to the other features they bring.

<rant> I would much prefer the sales material to not imply things that are not a good fit at the moment. Or to have the functionality and focus actually there. </rant>

daniel
Rank
Userlevel 4
Badge +5

Thanks for the ideas and workaround guys! 

Though, wouldn't you agree that, as the text implies, it would be nice to be able to sent messages directly from timextender when a validation rule is violated?

To be honest, I think people would go absolutely crazy with the amount of notifications one will get…
I'm all for better data quality, except I would like the business users to still be my friends or at least the good neighbour. 
 

I will create the insights for the business so they can monitor the data quality themselfs, but they have to be the change they want to see in their system.

@rory.smith I've found that I have so many data quality rules that my tables blow up. So I'd rather have a seperate script which I run after the DWH has run to check all the _L and _M tables for violations and report these seperately.

= Daniel

Reply


Terms and ConditionsCookie settings