Filter tables in Ingest only on non-production

Hi ​@wouter.goslinga we’ve solve that problem using views in the data source:

	WITH Parameters
		AS (SELECT 
				 CutYear
			    , OffsetYear
		    FROM   aux.DateFilter
		    WHERE  TableName = 'Albaranes Ventas'
				AND Environment='prod'),
.... Rest of query

Filter:
	WHERE  UCC@_RecordYear > CutYear + OffsetYear

In the DateFilter table we decide what records filter:

Hope this helps

Hi ​@wouter.goslinga ,

Could it work if you clone the data source? 

One version to use in the production environment without row filtering, and the clone with a row filter on a suitable column for the dev and test systems.

BR
Anders

Thanks for the suggestions ​@rvgfox and ​@anders.e.jonsson ! Unfortunately, we are unable / not allowed to create tables in the data source; and cloning the datasource would require us to remap every single table on every production release which is too labour intensive.

The search continues!

Hi ​@wouter.goslinga ,

As I understood it you have different ingest instances for each environment. When you migrate one prepare instance to another environment and instance you select ingest instance for the destination.

Would it be possible to use a mapping set? Both data sources exist in each ingest instance and are included in the mapping set in the prepare instance, but you only transfer data to one of the data sources in each ingest instance. To dev and test instances transfer is done with the filtered version of the data source, and in production you use the unfiltered version.

I haven’t tried it, but it seems possible 🤓.

BR
Anders

I have converted this to a product idea