Failed Jobs

Enqueued 0 / 1 Scheduled 0 Processing 14 Succeeded 3,084,166 Failed 855 Deleted 499 Awaiting 0

Failed jobs do not become expired to allow you to re-queue them without any time pressure. You should re-queue or delete them manually, or apply AutomaticRetry(OnAttemptsExceeded = AttemptsExceededAction.Delete) attribute to delete them automatically.

Id	Job	Failed
#213075	RetrieveCancellationRequestsTask.ProcessRecurringAsync An exception occurred during performance of the job. Fewer details...	07/20/2025 11:48:06
#213075	System.InvalidOperationException A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection() at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(StateManager stateManager, Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at EkolPY.Integration.Service.IntegrationTaskService.AddTaskLog(TaskLogLevel logLevel, String[] messages) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 154 at EkolPY.Integration.Service.IntegrationTaskService.AddTaskError(Exception ex, String[] msgs) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 146 at EkolPY.Integration.Service.IntegrationTaskService.SetFailedWithException(String message, Exception ex) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 246 at EkolPY.Integration.Manager.TaskBase`1.ProcessRecurringAsync(PerformContext context, String queueName, TaskType taskType) in /src/Integration/EkolPY.Integration.Manager/TaskBase.cs:line 162 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
#213051	RetrieveCancellationRequestsTask.ProcessRecurringAsync An exception occurred during processing of a background job. More details...	07/20/2025 11:46:04
#213051	Npgsql.NpgsqlOperationInProgressException The connection is already in state 'Executing' Npgsql.NpgsqlOperationInProgressException (0x80004005): The connection is already in state 'Executing' at Npgsql.Internal.NpgsqlConnector.<StartUserAction>g__DoStartUserAction\|259_0(<>c__DisplayClass259_0& ) at Npgsql.Internal.NpgsqlConnector.StartUserAction(ConnectorState newState, NpgsqlCommand command, CancellationToken cancellationToken, Boolean attemptPgCancellation) at Npgsql.Internal.NpgsqlConnector.Reset(Boolean async) at Npgsql.NpgsqlConnection.CloseAsync(Boolean async) at Npgsql.NpgsqlConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.ClearTransaction() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.Dispose() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.ResetState(Boolean disposeDbConnection) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Microsoft.EntityFrameworkCore.DbContext.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Hangfire.JobActivatorScope.Dispose() at Hangfire.Server.CoreBackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_0.<PerformJobWithFilters>b__0() at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation) at Hangfire.Server.BackgroundJobPerformer.PerformJobWithFilters(PerformContext context, IEnumerable`1 filters) at Hangfire.Server.BackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)
#213048	RetrieveOrdersTask.ProcessRecurringAsync An exception occurred during processing of a background job. More details...	07/20/2025 11:46:14
#213048	System.NullReferenceException Object reference not set to an instance of an object. System.NullReferenceException: Object reference not set to an instance of an object. at Npgsql.NpgsqlTransaction.Dispose(Boolean disposing) at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.Dispose() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.ResetState(Boolean disposeDbConnection) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Microsoft.EntityFrameworkCore.DbContext.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Hangfire.JobActivatorScope.Dispose() at Hangfire.Server.CoreBackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_0.<PerformJobWithFilters>b__0() at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation) at Hangfire.Server.BackgroundJobPerformer.PerformJobWithFilters(PerformContext context, IEnumerable`1 filters) at Hangfire.Server.BackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)
#213036	RetrieveOrdersTask.ProcessRecurringAsync An exception occurred during processing of a background job. More details...	07/20/2025 11:45:03
#213036	System.NullReferenceException Object reference not set to an instance of an object. System.NullReferenceException: Object reference not set to an instance of an object. at Npgsql.NpgsqlTransaction.Dispose(Boolean disposing) at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.Dispose() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.ResetState(Boolean disposeDbConnection) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Microsoft.EntityFrameworkCore.DbContext.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Hangfire.JobActivatorScope.Dispose() at Hangfire.Server.CoreBackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_0.<PerformJobWithFilters>b__0() at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation) at Hangfire.Server.BackgroundJobPerformer.PerformJobWithFilters(PerformContext context, IEnumerable`1 filters) at Hangfire.Server.BackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)
#212995	RetrieveCancellationRequestsTask.ProcessRecurringAsync An exception occurred during performance of the job. More details...	07/20/2025 11:42:16
#212995	System.InvalidOperationException A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection() at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(StateManager stateManager, Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at EkolPY.Integration.Service.IntegrationTaskService.AddTaskLog(TaskLogLevel logLevel, String[] messages) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 154 at EkolPY.Integration.Service.IntegrationTaskService.AddTaskError(Exception ex, String[] msgs) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 146 at EkolPY.Integration.Service.IntegrationTaskService.SetFailedWithException(String message, Exception ex) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 246 at EkolPY.Integration.Manager.TaskBase`1.ProcessRecurringAsync(PerformContext context, String queueName, TaskType taskType) in /src/Integration/EkolPY.Integration.Manager/TaskBase.cs:line 162 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
#212975	RetrieveOrdersTask.ProcessRecurringAsync An exception occurred during performance of the job. More details...	07/20/2025 11:40:15
#212975	System.InvalidOperationException A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection() at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(StateManager stateManager, Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at EkolPY.Integration.Service.IntegrationTaskService.AddTaskLog(TaskLogLevel logLevel, String[] messages) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 154 at EkolPY.Integration.Service.IntegrationTaskService.AddTaskError(Exception ex, String[] msgs) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 144 at EkolPY.Integration.Service.IntegrationTaskService.SetFailedWithException(String message, Exception ex) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 244 at EkolPY.Integration.Manager.TaskBase`1.ProcessRecurringAsync(PerformContext context, String queueName, TaskType taskType) in /src/Integration/EkolPY.Integration.Manager/TaskBase.cs:line 162 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
#212966	RetrieveCancellationRequestsTask.ProcessRecurringAsync An exception occurred during performance of the job. More details...	07/20/2025 11:40:15
#212966	System.InvalidOperationException A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection() at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(StateManager stateManager, Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at EkolPY.Integration.Service.IntegrationTaskService.AddTaskLog(TaskLogLevel logLevel, String[] messages) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 154 at EkolPY.Integration.Service.IntegrationTaskService.AddTaskError(Exception ex, String[] msgs) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 144 at EkolPY.Integration.Service.IntegrationTaskService.SetFailedWithException(String message, Exception ex) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 244 at EkolPY.Integration.Manager.TaskBase`1.ProcessRecurringAsync(PerformContext context, String queueName, TaskType taskType) in /src/Integration/EkolPY.Integration.Manager/TaskBase.cs:line 162 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
#175883	RetrieveCancellationRequestsTask.ProcessRecurringAsync An exception occurred during processing of a background job. More details...	07/18/2025 13:30:04
#175883	Npgsql.NpgsqlOperationInProgressException The connection is already in state 'Executing' Npgsql.NpgsqlOperationInProgressException (0x80004005): The connection is already in state 'Executing' at Npgsql.Internal.NpgsqlConnector.<StartUserAction>g__DoStartUserAction\|259_0(<>c__DisplayClass259_0& ) at Npgsql.Internal.NpgsqlConnector.StartUserAction(ConnectorState newState, NpgsqlCommand command, CancellationToken cancellationToken, Boolean attemptPgCancellation) at Npgsql.Internal.NpgsqlConnector.Reset(Boolean async) at Npgsql.NpgsqlConnection.CloseAsync(Boolean async) at Npgsql.NpgsqlConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.ClearTransaction() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.Dispose() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.ResetState(Boolean disposeDbConnection) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Microsoft.EntityFrameworkCore.DbContext.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Hangfire.JobActivatorScope.Dispose() at Hangfire.Server.CoreBackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_0.<PerformJobWithFilters>b__0() at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation) at Hangfire.Server.BackgroundJobPerformer.PerformJobWithFilters(PerformContext context, IEnumerable`1 filters) at Hangfire.Server.BackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)
#175879	RetrieveOrdersTask.ProcessRecurringAsync An exception occurred during processing of a background job. More details...	07/18/2025 13:29:11
#175879	Npgsql.NpgsqlOperationInProgressException The connection is already in state 'Executing' Npgsql.NpgsqlOperationInProgressException (0x80004005): The connection is already in state 'Executing' at Npgsql.Internal.NpgsqlConnector.<StartUserAction>g__DoStartUserAction\|259_0(<>c__DisplayClass259_0& ) at Npgsql.Internal.NpgsqlConnector.StartUserAction(ConnectorState newState, NpgsqlCommand command, CancellationToken cancellationToken, Boolean attemptPgCancellation) at Npgsql.Internal.NpgsqlConnector.Reset(Boolean async) at Npgsql.NpgsqlConnection.CloseAsync(Boolean async) at Npgsql.NpgsqlConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.ClearTransaction() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.Dispose() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.ResetState(Boolean disposeDbConnection) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Microsoft.EntityFrameworkCore.DbContext.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Hangfire.JobActivatorScope.Dispose() at Hangfire.Server.CoreBackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_0.<PerformJobWithFilters>b__0() at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation) at Hangfire.Server.BackgroundJobPerformer.PerformJobWithFilters(PerformContext context, IEnumerable`1 filters) at Hangfire.Server.BackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)
#175842	RetrieveOrdersTask.ProcessRecurringAsync An exception occurred during processing of a background job. More details...	07/18/2025 13:26:03
#175842	Npgsql.NpgsqlOperationInProgressException The connection is already in state 'Executing' Npgsql.NpgsqlOperationInProgressException (0x80004005): The connection is already in state 'Executing' at Npgsql.Internal.NpgsqlConnector.<StartUserAction>g__DoStartUserAction\|259_0(<>c__DisplayClass259_0& ) at Npgsql.Internal.NpgsqlConnector.StartUserAction(ConnectorState newState, NpgsqlCommand command, CancellationToken cancellationToken, Boolean attemptPgCancellation) at Npgsql.Internal.NpgsqlConnector.Reset(Boolean async) at Npgsql.NpgsqlConnection.CloseAsync(Boolean async) at Npgsql.NpgsqlConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.ClearTransaction() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.Dispose() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.ResetState(Boolean disposeDbConnection) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Microsoft.EntityFrameworkCore.DbContext.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Hangfire.JobActivatorScope.Dispose() at Hangfire.Server.CoreBackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_0.<PerformJobWithFilters>b__0() at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation) at Hangfire.Server.BackgroundJobPerformer.PerformJobWithFilters(PerformContext context, IEnumerable`1 filters) at Hangfire.Server.BackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)

System.InvalidOperationException

Npgsql.NpgsqlOperationInProgressException

System.NullReferenceException

System.NullReferenceException

System.InvalidOperationException

System.InvalidOperationException

System.InvalidOperationException

Npgsql.NpgsqlOperationInProgressException

Npgsql.NpgsqlOperationInProgressException

Npgsql.NpgsqlOperationInProgressException