Solved

Job stuck on Active


Userlevel 5
Badge +7

I have a job that reloads a DWH instance that seems to be stuck on ‘Active’. This job is months old and may have been killed by a VM being shut down automatically.  This job blocks executions with a ‘Job already queued’ error.

Is there any way to remove this from the queue?

 

 

icon

Best answer by rory.smith 4 April 2023, 12:01

View original

15 replies

Userlevel 6
Badge +5

Hi @rory.smith have you tried to delete and recreate this job?

Userlevel 5
Badge +7

Is that the only way? That seems rather destructive

Userlevel 5
Badge +7

I deleted the job and recreated it, now I have two:

 

Userlevel 6
Badge +5

I tried on my end to delete a job and then recreate it with the same name, but am unable to reproduce the issue. Have you tried refreshing/restarting the desktop app?

If you try deleting both and creating a new job with a new name, and refresh and then execute the new job, does it run?

Userlevel 5
Badge +7

Refreshing the jobs/interface removes the duplicate (which triggered an ‘Object not set to reference’ error when trying to execute). I tried deleting the 2. DWH… job. This removes the queue entry from the Jobs > Monitor list. I subsequently refreshed the interface and closed and reopened the desktop app.

If I recreate the Job (same name, same items) the queue record reappears in the Monitor with the ‘Idle’ State and can now be executed.which sets the Jobs > Monitor State to ‘Active’. But, there is no entry in the Execution Log for the Job. I can also see no evidence for it being run.

Creating a new job with a different name and the same content results in the same. I then tried deleting all jobs and choosing Jobs > Refresh and now get the following error:

 

Error in command: 'Refresh'.

An error occurred while sending the request.
The remote name could not be resolved: 'app-encryption-prod-001.azurewebsites.net'

Details:

The remote name could not be resolved: 'app-encryption-prod-001.azurewebsites.net'
Module: System
System.Net.WebException
   at System.Net.HttpWebRequest.EndGetRequestStream(IAsyncResult asyncResult, TransportContext& context)
   at System.Net.Http.HttpClientHandler.GetRequestStreamCallback(IAsyncResult ar)

An error occurred while sending the request.
Module: mscorlib
System.Net.Http.HttpRequestException
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TimeXtender.Portal.API.ApiHelper.<PostAsJsonAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TimeXtender.Portal.API.ApiHelper.<>c__DisplayClass5_0.<<PostWithSecretAsync>b__0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at TimeXtender.Portal.API.ApiHelper.<ManageExceptionsWithRetry>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TimeXtender.Portal.API.ApiHelper.<PostWithSecretAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TimeXtender.Portal.API.Encryption.TimeXtenderEncryptionAPI.<DecryptAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TimeXtender.Portal.API.Encryption.TimeXtenderEncryptionAPI.<DecryptAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TimeXtender.Portal.API.Encryption.TimeXtenderEncryptionAPI.<DecryptAsStringAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at TimeXtender.Utility.AsyncHelper.RunSync[TResult](Func`1 func)
   at TimeXtender.DataManager.MDWInstance.CreateEndpointModel(IEnumerable`1 properties)
   at TimeXtender.DataManager.MDWInstance.UpdateInstance(InstanceDTO instanceDataTransferObject)
   at TimeXtender.DataManager.TimeXtenderInstanceController.<>c.<LoadInstances>b__19_2(InstanceDTO instanceDataTransferObject)
   at System.Collections.Generic.List`1.ForEach(Action`1 action)
   at TimeXtender.DataManager.TimeXtenderInstanceController.LoadInstances(ConnectingThread connectingThread)
   at TimeXtender.DataManager.RefreshInstancesCommand.<>c__DisplayClass2_0.<Execute>b__0()
   at TimeXtender.DataManager.ConnectingThread.ExecuteConnectingThread(Object dummy)

An error occurred while sending the request.
Module: timeXtender
TXModelInterface.ExceptionWrapperException
   at TimeXtender.DataManager.ConnectingThread.HandleError()
   at TimeXtender.DataManager.ConnectingThread.Execute(String title, Int32 progressSteps, List`1 actions)
   at TimeXtender.DataManager.RefreshInstancesCommand.Execute()
   at TimeXtender.DataManager.MainForm.ExecuteTXCommand(ITXCommand command, Boolean checkForOpenWindows, Object owner)

Time: 2023-04-04 11:42:51
UTC: 2023-04-04 09:42:51
Title: Demo DWH - TimeXtender
Application: 6221.1
User: TXUser1
Domain: tx-as-base
Login user: Ruairidh Smith (rory.smith@e-mergo.nl)
OS: Microsoft Windows 10 Pro
OS version: Microsoft Windows NT 6.2.9200.0
Machine name: tx-as-base
CPU count: 2
Build: 64 bit
 

This seems to have been a transient error, but may be useful to know

Userlevel 5
Badge +7

After stopping and starting the TimeXtender Execution service it now seems to be running executions properly again.

Userlevel 1

Hi have a similar issue. A job is stuck on running:

 

Everything seems to work as it should, but it’s incredible annoying (not to mention confusing) to have wrong information in the monitor.

In the old version of TX I experienced similar things where an execution package would be lost, perhaps to a TX crash or other issues. What I’d do then is to go to the “repository manager” or whatever it was called in the TX desktop and manually delete the log.

It would be great to have some sort of insight into the repo data in cloud to do a similar operation.

As Rory said, deleting and recreating a job to remove the erroneous status isn’t a great solution.

 

 

@joelpettersson: I know i'm a bit late to the party, but i also see this problem with a few of our jobs. I already tried deleting the job and reconfiguring it, but the Execution State will stay in running however all tasks have completed successfully.

 

Did you find any solution to this problem?

Userlevel 5
Badge +7

Hi @JogchumSiR ,

 

after deleting the job, did you restart the Execution Engine and perhaps try refreshing the TX UI?

@rory.smith no i didn't. I will test that and let you know

@rory.smith yesterday i've recreated these 2 jobs. One i did already run as a test, the second one didn't until yesterday evening. I've restarted the service as you suggested. Unfortunately the execution state remains incorrect after everything was executed...

However, yesterday we also configured and scheduled the same job for production (we are in the implementation phase) and the same job, with the same configuration, just different time also has the same symptom. Can something in the job configuration cause this?

I have the same issue with ODX sync and clean-up jobs. 
All tasks will run fine, but state will not change

If I delete the jobs and recreate the new jobs will run into the exact same issues

Userlevel 5
Badge +7

Hi,

these are both ODX related tasks/jobs? I assume that your ODX shows nothing in the queue? If noone else chimes in, it may be time to open a ticket.

On a separate note: scheduling synchronization tasks might result in unwanted situations for data being loaded from file share or similar. You may want to manually run synchronizations in those cases.

@rory.smith : Both related to ODX indeed. I will raise a ticket about this.
Thanks for your help so far!

Userlevel 6
Badge +5

Hi all

This seems to be a bug, I made an internal case for this. It should be reset when it is not running.

Reply