Skip to main content
Solved

How to add Query table for Mysql data source


Hi Team,

We have a data provider for which we need to connect to MySQL. We want to incrementally bring data from the source, but the problem is that all the columns in the table are of the string data type. This prevents us from setting any incremental logic on the date column, which is currently stored as a VARCHAR(8). I tried casting the column to a date format, which works in the query tool, but when I try to add the query table, the option is disabled.

 

I am using Mysql 5.1 client data source.

Could you please suggest how I can achieve this?

Best answer by Dushyant

Unfortunately i was not able to fix the issue. So with the help of source partner team, added a new datetime column with which i am able to select data incrementally.

View original
Did this topic help you find an answer to your question?

8 replies

rory.smith
TimeXtender Xpert
Forum|alt.badge.img+7
  • TimeXtender Xpert
  • 649 replies
  • August 21, 2024

Hi @Dushyant ,

if you have a query table that outputs a field that can be used in an incremental selection rules like an int or date(time) you can make that query table load incrementally.

There are two prerequisites:

  • set a primary key for your Query Table (you cannot load incrementally without one). You can use the Override function to do this
  • set up an incremental selection rule that picks up the field

Primary key on SalesOrderDetail Query Table:

Incremental selection rule picks up the query table:

In the DWH you will see an Incremental mapping and the second reload will show an empty Raw table (static data in Adventureworks):

 


  • Author
  • Starter
  • 2 replies
  • August 21, 2024

Thanks @rory.smith , But the problem for me is that the query table option inside the “data source setting” is disable. Hence i am unable to add query table. What i mentioned earlier was in query “tool” i am able to case the column which is varchar to date.

 


daniel
TimeXtender Xpert
Forum|alt.badge.img+7
  • TimeXtender Xpert
  • 188 replies
  • August 21, 2024

Dear @Dushyant ,

I tink you don't need these option because you can put the casts / converts in the Query Table.

you do need to write this in mysql instead of T-SQL.

 

Hope this helps.

= Daniel


rory.smith
TimeXtender Xpert
Forum|alt.badge.img+7
  • TimeXtender Xpert
  • 649 replies
  • August 21, 2024

Hi @Dushyant ,

given that that screenshot says “Discovery Hub”, that would be TimeXtender 17.x? Given that that is many years old, I can't really remember the exact possibilities.

I suspect in those cases you cannot set Primary Keys on query table fields though. To get query tables working Perhaps you should then approach the MySQL database through ODBC, OleDB or Anysource - one of will probably work.

You may want to focus on getting closer to a recent release though.


daniel
TimeXtender Xpert
Forum|alt.badge.img+7
  • TimeXtender Xpert
  • 188 replies
  • August 21, 2024

Dear @Dushyant ,

In this version you need to set up the query table first. Cast the varchar to date(time). Synchronize the data source and select the created table to be loaded. In the ODX, set the Primary Key on a field by right clicking on the field and selecting add to primary key.

Then, in the data source table, select the query table - right click - add incremental selection rule. Select the datetime.

This should work, also in your version. I do agreee with Rory that it might be a good thing to upgrade to a more recent version.

 

= Daniel


Thomas Lind
Community Manager
Forum|alt.badge.img+5
  • Community Manager
  • 1017 replies
  • August 21, 2024

Hi @Dushyant 

It would seem like you are using our MySQL provider.

Years ago I wrote a guide about how to convert this to a AnySource ADO.Net version instead.

The benefit being that you can do queries like this in that.

https://legacysupport.timextender.com/hc/en-us/articles/115005949443-How-to-change-a-MySQL-ODBC-type-data-source-to-a-Any-Source-ADO-MySQL-data-source


Christian Hauggaard
Community Manager
Forum|alt.badge.img+5

Hi @Dushyant is the issue resolved? If so please help us by marking a best answer above. Please let us know if you have any follow up questions


  • Author
  • Starter
  • 2 replies
  • Answer
  • September 3, 2024

Unfortunately i was not able to fix the issue. So with the help of source partner team, added a new datetime column with which i am able to select data incrementally.


Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings