To homepage To homepage
Solved

Rest API rate limit

  • 3 May 2023
  • 8 replies
  • 259 views
N
Rank

Hi,

I'm currently connecting to a new api.
https://ads-api.autotelex.com/swagger/index.html?urls.primaryName=Autotelex%20%7C%20Advertisement%20API%20V2%20(Beta)
In my RSD file I call /api/v2/stock to get the ids for a stock, then I call /api/v2/stock/{adsStockId}/summary using the ids from the first call.
The problem I run in to is that i can only make 300 calls per minute for the second call. Whenever I run into this limit my execute fails with the message: "Too many requests".
Is there any way to limit or reduce the amount of calls I make per minute?

Kind regards,
Niek

icon

Best answer by Christian Hauggaard 22 May 2023, 09:53

View original

8 replies

Christian Hauggaard
Rank
Userlevel 6
Badge +5

HI @Niek.Guijt 

  1. Can you please share the rsd file which was generated.
  2. Please also share the log file which can be setup under logging in the rest data source by specifying a file path for the txt log file and set the verbosity to 3. If you would prefer to share the log file in a private message to me then please feel free to do so.
  3. Please let us know if you have any API call limitation on your application.
N
Rank

Hi Christian,

 

Here is my RSD file:

<api:script xmlns:api="http://apiscript.com/ns?v1" xmlns:xs="http://www.w3.org/2001/XMLSchema">

  <!-- See Column Definitions to specify column behavior and use XPaths to extract column values from JSON. -->
 <api:info title="JSONData" desc="Generated schema file." xmlns:other="http://apiscript.com/ns?v1">
  <attr name="id"                xs:type="integer"  readonly="false"              other:xPath="/json/id"                />
  <attr name="adsStockId"        xs:type="integer"  readonly="false"              other:xPath="/json/adsStockId"        />
  <attr name="advertisementTitle" xs:type="string"   readonly="false"              other:xPath="/json/advertisementTitle" />
</api:info>



<api:set attr="urlbase" value="https://ads-api.autotelex.com"/>
  <api:set attr="filein.DataModel" value="DOCUMENT" />
  <api:set attr="filein.JSONPath" value="$." /> 
  <api:set attr="filein.URITemplate" value="https://ads-api.autotelex.com/api/v2/stock?modifiedSinceUnixMs=1683031500000&amp;stockStatus=InStock&amp;includeDeleted=false" />
  <api:set attr="filein.ElementMapPath#" value="json/id" />
  <api:set attr="filein.ElementMapName#" value="file_id" />

  <api:set attr="stopin.DataModel" value="DOCUMENT" />
  <api:set attr="stopin.EnablePaging" value="TRUE" />
  <api:set attr="stopin.JSONPath" value="$." />
  <api:set attr="stopin.URITemplate" value="https://ads-api.autotelex.com/api/v2/stock/{file_id}/summary" />

  <!-- The GET method corresponds to SELECT. Here you can override the default processing of the SELECT statement. The results of processing are pushed to the schema's output. See SELECT Execution for more information. -->
  <api:script method="GET">
    <api:set attr="filein.URI" value="[filein.URITemplate]"/>
    <api:call op="jsonproviderGet" in="filein" out="fileout">
      <api:set attr="filein.file_id" value="[fileout.file_id]" />
      <api:set attr="stopin.URI" value="[stopin.URITemplate | replace('{file_id}', [filein.file_id])]"/>
      <api:call op="jsonproviderGet" in="stopin" out="stopout">

	  <api:set attr="out.adsStockId" value="[stopout.adsStockId | allownull()]" />
	<api:set attr="out.advertisementTitle" value="[stopout.advertisementTitle | allownull()]" />

	<api:push item="out"/>
      </api:call>
    </api:call>
  </api:script>
</api:script>



I'll send you the log in a pm. 

 

Kind regards,

Niek

Christian Hauggaard
Rank
Userlevel 6
Badge +5

Hi @Niek.Guijt 

Thanks for the rsd and log file. I have passed these onto Cdata and will let you know once I receive a response. In the meantime, could you please investigate if you have any API call limitation on your application?

N
Rank

Hi Christian,

I can make 60 calls per minute for the first call and 300 per minute for the second call. 

Christian Hauggaard
Rank
Userlevel 6
Badge +5

Hi @Niek.Guijt I received the following response from Cdata:

The API limit is default 300 as per the Autotelex ADS API documentation. 


So, we couldn't change the API limit but there is a way to avoid the 429 - Too many request by adding the following connection string into the CData driver.

MaximumRequestRetries - This connection string helps to retries the API request after it fails. We recommend to use MaximumReuqestRetries = 5;

RetryWaitTime - This connection String used to how long the connection can wait for next API request retry. We recommend to use RetryWaitTime=10000;

So, kindly use the above connection property into your driver and try once. Please note that above connection properties are not a direct property, so used through Other connection property such as 

Other = MaximumReuqestRetries = 5;RetryWaitTime=10000; 
 

N
Rank

Hi @Christian Hauggaard,

I tried your suggestion, however i haven't been able to get it to work. Is this the correct place for the connection property?

Judging from the log it doesn’t appear to actually apply the retries and/or wait-time. I'll send you the new log in a PM.

Kind regards,

Niek

Christian Hauggaard
Rank
Userlevel 6
Badge +5

Hi @Niek.Guijt 

Please update the REST provider to the latest version (22.0.8539.0)

Then edit your data source and select OK for the warning dialog and OK for the data source.

Edit the data source again and set the maximum request retries to 40. i.e. Other = MaximumReuqestRetries = 40;RetryWaitTime=10000; 

Then execute the synchronization task and then the transfer task. If you still encounter issues, please send an updated log file. 

J
Rank

Hi all,

I ran into the same issue of having a restriction to the number of calls per minute. Unfortunately the suggested settings above did not work for me.

However, I did find another solution that can be applied in the rsd-file and may help other users. 

I added a piece of code to handle error messages and when it occurs it will have a waiting time (in this case 30 seconds) before it proceeds with the call. 

 

 <api:catch code="*">
       <api:call op="utiladoSleep?timeout=30" />
 </api:catch>

 

More information on this: CData ADO.NET Provider for REST - utiladoSleep

Kind regards,

Jacqueline

Reply


Terms and ConditionsCookie settings