@p.ruijter It can be that another package is still using all your cores so it isn't able to run this package. It can also be that your package aren't enabled on running concurrent, this means that only 1 of the packages can run at the time. If they aren't able to run concurrent they either fail (with unknown/running) or they run later when the other packages are done.
This delay happens because the scheduler service needs to start a TimeXtender application instance (timextender.exe) and then execute the selected execution package. Doing this takes some time (depending on the work load on the machine running TimeXtender).
@Bernarddb
We have 4 environments in TX.
For instance: a TX Acceptance (night) job is running from 5 am till 7 am. A job from TX Production environment is running at 6 am. I cannot select a concuurent packages from another environment. How to solve that?
@peter.jensen Even if there isn't a workload it is still running later than scheduled.
Let's get the picture:
- Windows Server 2019
- Memory 71680 Mb
- Cores: 6, Logical Processors: 6
- CPU 3,4Ghz4
- TX environments all on the same server
- services: see attached screenshot
For example: we have a job at 1 am who starts 1 minute later as scheduled (1:01 am). no job, in any environment, is running at the same time.
Can it be the multiple services (see screenshot) we have on the same server? This is to handle our 4 environments.
Or do you have other suggestions?
Hi,
triggering Windows executions is not an exact thing as @peter.jensen stated. If you look in Windows logging or attach a debugger to the service, you will probably see that they do start at almost the same time. Depending on the load of your system, it will take longer or shorter for the whole process to reach the point where TimeXtender generates the starting timestamp.
This is why Windows Services have the Automatic (Delayed) startup type, for instance, this makes it so that a service is only triggered to start after a certain period (controlled by the registry) to ensure that certain required services (networking) are all-but guaranteed to have started. If you have a short delay specified, you might still have service-related errors if your networking stack is slow to start.
As some added “fun”: if your machine is not properly receiving time from an NTP server, the clock will drift and the actual time of starting will deviate more or less from real time.
Note that concurrent running is not an issue across environment only within, if your infrastructure is separate for each environment. If, as per your latest post, you are hosting all environments on one machine then there will likely be resource contention or differences in RAM / CPU availability which will all affect the time it takes to open a timextender processs. If you are hosting SQL on that, it will likely be swapping out data from buffer pool as well which will affect the runtime. Any users leaving sessions open will also consume resources and change the speed.
Is it actually a problem if an execution starts 60 seconds later on one day vs another?
@rory.smith But why if I start the job manually it starts immediately?
If it is by schedule then the Event viewer Windows Log says: TimeXtender started, using configuration file C:\Users\te_service\AppData\Roaming\TimeXtender\TimeXtender\20.10.45.64\Config.xml.
This happens for example at 15:01:18 (the schedule is at 15:00:00). Why does it start more than one minute later?
The service has not the option Delayed. All other users are logged out.
When you manually run a Job you do this from the context of an already running timextender.exe process that already has the instances loaded. If you schedule a Job, or Task this sends a command to the required service, which then cause a TimeXtender.exe process or other child process to be started, which needs to load things and then start running.
This is similar to using ADF with an AzureRuntime or Autoresolver runtime: this queues spinning up a VM to handle your compute and incurs a delay (that you may be paying for). If you have your own self-hosted integration runtime, it immediately triggers the run without needing to spin up any VMs on your infrastructure.
tl;dr: if you double-click the timextender icon, right-click your Job or open the Ingest instance and execute a task your execution also does not start from the moment you double-clicked the timextender icon.
@rory.smith Yeah I figured out that it first starts something before executing the job. But starting the application/project is less than 10 seconds. So why is starting a scheduled job 1,5 minute before starting the job? I still think it is too long. Is there a way to speed the proces of starting timextender?
Hi @p.ruijter as Peter mentions above, this happens because the scheduler service needs to start a TimeXtender application instance (timextender.exe) and then execute the selected execution package. Perhaps scaling up the application VM will reduce the time slightly but there will be some delay before the execution package is started
@Thomas Lind Can you maybe look also at this issue.
The execution package starts after more than one minute later as scheduled.
The scheduler service starts timextender, this causes a delay. But when we start timextender manually and open a project en than manually start an execution package it takes about 20 seconds. So I don't understand why it takes so long before a scheduled package really starts.
Why does it take so long before the scheduler service starts timextender or why does it take so long starting timextender? The Event Viewer indicates that after more than one minute timextender has started (TimeXtender started, using configuration file C:\Users\te_service\AppData\Roaming\TimeXtender\TimeXtender\20.10.45.64\Config.xml)
PS: No other packages are running