Failed Jobs

Enqueued 0 / 1 Scheduled 0 Processing 8 Succeeded 3,144,214 Failed 855 Deleted 499 Awaiting 0

Failed jobs do not become expired to allow you to re-queue them without any time pressure. You should re-queue or delete them manually, or apply AutomaticRetry(OnAttemptsExceeded = AttemptsExceededAction.Delete) attribute to delete them automatically.

Id	Job	Failed
#771166	RetrieveCancellationsTask.ProcessRecurringAsync An exception occurred during processing of a background job. Fewer details...	08/18/2025 08:08:17
#771166	Npgsql.NpgsqlOperationInProgressException The connection is already in state 'Executing' Npgsql.NpgsqlOperationInProgressException (0x80004005): The connection is already in state 'Executing' at Npgsql.Internal.NpgsqlConnector.<StartUserAction>g__DoStartUserAction\|259_0(<>c__DisplayClass259_0& ) at Npgsql.Internal.NpgsqlConnector.StartUserAction(ConnectorState newState, NpgsqlCommand command, CancellationToken cancellationToken, Boolean attemptPgCancellation) at Npgsql.Internal.NpgsqlConnector.Reset(Boolean async) at Npgsql.NpgsqlConnection.CloseAsync(Boolean async) at Npgsql.NpgsqlConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.CloseDbConnection() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.ClearTransaction() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.Dispose() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.ResetState(Boolean disposeDbConnection) at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Microsoft.EntityFrameworkCore.DbContext.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Hangfire.AspNetCore.AspNetCoreJobActivatorScope.DisposeScope() at Hangfire.JobActivatorScope.Dispose() at Hangfire.Server.CoreBackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_0.<PerformJobWithFilters>b__0() at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_1.<PerformJobWithFilters>b__2() at Hangfire.Server.BackgroundJobPerformer.PerformJobWithFilters(PerformContext context, IEnumerable`1 filters) at Hangfire.Server.BackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)
#771138	RetrieveCancellationRequestsTask.ProcessRecurringAsync An exception occurred during performance of the job. More details...	08/18/2025 08:06:03
#771138	System.InvalidOperationException A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection() at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(StateManager stateManager, Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at EkolPY.Integration.Service.IntegrationTaskService.AddTaskLog(TaskLogLevel logLevel, String[] messages) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 154 at EkolPY.Integration.Service.IntegrationTaskService.AddTaskError(Exception ex, String[] msgs) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 146 at EkolPY.Integration.Service.IntegrationTaskService.SetFailedWithException(String message, Exception ex) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 247 at EkolPY.Integration.Manager.TaskBase`1.ProcessRecurringAsync(PerformContext context, String queueName, TaskType taskType) in /src/Integration/EkolPY.Integration.Manager/TaskBase.cs:line 162 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
#771128	RetrieveCancellationsTask.ProcessRecurringAsync An exception occurred during performance of the job. More details...	08/18/2025 08:06:04
#771128	System.InvalidOperationException A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection() at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(StateManager stateManager, Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at EkolPY.Integration.Service.IntegrationTaskService.AddTaskLog(TaskLogLevel logLevel, String[] messages) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 154 at EkolPY.Integration.Service.IntegrationTaskService.AddTaskError(Exception ex, String[] msgs) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 146 at EkolPY.Integration.Service.IntegrationTaskService.SetFailedWithException(String message, Exception ex) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 247 at EkolPY.Integration.Manager.TaskBase`1.ProcessRecurringAsync(PerformContext context, String queueName, TaskType taskType) in /src/Integration/EkolPY.Integration.Manager/TaskBase.cs:line 162 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
#771127	RetrieveOrdersCreatedPickingTask.ProcessRecurringAsync An exception occurred during processing of a background job. More details...	08/18/2025 08:06:15
#771127	System.NullReferenceException Object reference not set to an instance of an object. System.NullReferenceException: Object reference not set to an instance of an object. at Npgsql.NpgsqlTransaction.Dispose(Boolean disposing) at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.Dispose() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.ResetState(Boolean disposeDbConnection) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Microsoft.EntityFrameworkCore.DbContext.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Hangfire.JobActivatorScope.Dispose() at Hangfire.Server.CoreBackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_0.<PerformJobWithFilters>b__0() at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation) at Hangfire.Server.BackgroundJobPerformer.PerformJobWithFilters(PerformContext context, IEnumerable`1 filters) at Hangfire.Server.BackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)
#771126	SendCancellationsTask.ProcessRecurringAsync An exception occurred during processing of a background job. More details...	08/18/2025 08:06:01
#771126	System.NullReferenceException Object reference not set to an instance of an object. System.NullReferenceException: Object reference not set to an instance of an object. at Npgsql.NpgsqlTransaction.Dispose(Boolean disposing) at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.Dispose() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.ResetState(Boolean disposeDbConnection) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Microsoft.EntityFrameworkCore.DbContext.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Hangfire.JobActivatorScope.Dispose() at Hangfire.Server.CoreBackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_0.<PerformJobWithFilters>b__0() at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation) at Hangfire.Server.BackgroundJobPerformer.PerformJobWithFilters(PerformContext context, IEnumerable`1 filters) at Hangfire.Server.BackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)
#716897	SendOrderItemIdentityTasks.ProcessRecurringAsync An exception occurred during performance of the job. More details...	08/15/2025 15:15:03
#716897	System.InvalidOperationException A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection() at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(StateManager stateManager, Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at EkolPY.Integration.Service.IntegrationTaskService.AddTaskLog(TaskLogLevel logLevel, String[] messages) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 154 at EkolPY.Integration.Service.IntegrationTaskService.AddTaskError(Exception ex, String[] msgs) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 146 at EkolPY.Integration.Service.IntegrationTaskService.SetFailedWithException(String message, Exception ex) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 247 at EkolPY.Integration.Manager.TaskBase`1.ProcessRecurringAsync(PerformContext context, String queueName, TaskType taskType) in /src/Integration/EkolPY.Integration.Manager/TaskBase.cs:line 162 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
#716507	SendCancellationsTask.ProcessRecurringAsync An exception occurred during processing of a background job. More details...	08/15/2025 14:46:08
#716507	Npgsql.NpgsqlOperationInProgressException The connection is already in state 'Executing' Npgsql.NpgsqlOperationInProgressException (0x80004005): The connection is already in state 'Executing' at Npgsql.Internal.NpgsqlConnector.<StartUserAction>g__DoStartUserAction\|259_0(<>c__DisplayClass259_0& ) at Npgsql.Internal.NpgsqlConnector.StartUserAction(ConnectorState newState, NpgsqlCommand command, CancellationToken cancellationToken, Boolean attemptPgCancellation) at Npgsql.Internal.NpgsqlConnector.Reset(Boolean async) at Npgsql.NpgsqlConnection.CloseAsync(Boolean async) at Npgsql.NpgsqlConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.ClearTransaction() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.Dispose() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.ResetState(Boolean disposeDbConnection) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Microsoft.EntityFrameworkCore.DbContext.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Hangfire.JobActivatorScope.Dispose() at Hangfire.Server.CoreBackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_0.<PerformJobWithFilters>b__0() at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation) at Hangfire.Server.BackgroundJobPerformer.PerformJobWithFilters(PerformContext context, IEnumerable`1 filters) at Hangfire.Server.BackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)
#716480	SendOrderItemIdentityTasks.ProcessRecurringAsync An exception occurred during processing of a background job. More details...	08/15/2025 14:45:06
#716480	Npgsql.NpgsqlOperationInProgressException The connection is already in state 'Executing' Npgsql.NpgsqlOperationInProgressException (0x80004005): The connection is already in state 'Executing' at Npgsql.Internal.NpgsqlConnector.<StartUserAction>g__DoStartUserAction\|259_0(<>c__DisplayClass259_0& ) at Npgsql.Internal.NpgsqlConnector.StartUserAction(ConnectorState newState, NpgsqlCommand command, CancellationToken cancellationToken, Boolean attemptPgCancellation) at Npgsql.Internal.NpgsqlConnector.Reset(Boolean async) at Npgsql.NpgsqlConnection.CloseAsync(Boolean async) at Npgsql.NpgsqlConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.Close() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.ClearTransaction() at Microsoft.EntityFrameworkCore.Storage.RelationalTransaction.Dispose() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.ResetState(Boolean disposeDbConnection) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Microsoft.EntityFrameworkCore.DbContext.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Hangfire.JobActivatorScope.Dispose() at Hangfire.Server.CoreBackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_0.<PerformJobWithFilters>b__0() at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation) at Hangfire.Server.BackgroundJobPerformer.PerformJobWithFilters(PerformContext context, IEnumerable`1 filters) at Hangfire.Server.BackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)
#716464	RetrieveOrdersTask.ProcessRecurringAsync An exception occurred during performance of the job. More details...	08/15/2025 14:44:07
#716464	System.InvalidOperationException A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. System.InvalidOperationException: A second operation was started on this context instance before a previous operation completed. This is usually caused by different threads concurrently using the same instance of DbContext. For more information on how to avoid threading issues with DbContext, see https://go.microsoft.com/fwlink/?linkid=2097913. at Microsoft.EntityFrameworkCore.Infrastructure.Internal.ConcurrencyDetector.EnterCriticalSection() at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(IList`1 entriesToSave, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.ChangeTracking.Internal.StateManager.SaveChangesAsync(StateManager stateManager, Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Npgsql.EntityFrameworkCore.PostgreSQL.Storage.Internal.NpgsqlExecutionStrategy.ExecuteAsync[TState,TResult](TState state, Func`4 operation, Func`4 verifySucceeded, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at Microsoft.EntityFrameworkCore.DbContext.SaveChangesAsync(Boolean acceptAllChangesOnSuccess, CancellationToken cancellationToken) at EkolPY.Integration.Service.IntegrationTaskService.AddTaskLog(TaskLogLevel logLevel, String[] messages) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 154 at EkolPY.Integration.Service.IntegrationTaskService.AddTaskError(Exception ex, String[] msgs) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 146 at EkolPY.Integration.Service.IntegrationTaskService.SetFailedWithException(String message, Exception ex) in /src/Integration/EkolPY.Integration.Service/IntegrationService.cs:line 247 at EkolPY.Integration.Manager.TaskBase`1.ProcessRecurringAsync(PerformContext context, String queueName, TaskType taskType) in /src/Integration/EkolPY.Integration.Manager/TaskBase.cs:line 162 at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
#716462	RetrieveOrdersCreatedPickingTask.ProcessRecurringAsync An exception occurred during processing of a background job. More details...	08/15/2025 14:44:07
#716462	Npgsql.NpgsqlOperationInProgressException The connection is already in state 'Executing' Npgsql.NpgsqlOperationInProgressException (0x80004005): The connection is already in state 'Executing' at Npgsql.Internal.NpgsqlConnector.<StartUserAction>g__DoStartUserAction\|259_0(<>c__DisplayClass259_0& ) at Npgsql.Internal.NpgsqlConnector.StartUserAction(ConnectorState newState, NpgsqlCommand command, CancellationToken cancellationToken, Boolean attemptPgCancellation) at Npgsql.Internal.NpgsqlConnector.Reset(Boolean async) at Npgsql.NpgsqlConnection.CloseAsync(Boolean async) at Npgsql.NpgsqlConnection.Close() at Npgsql.NpgsqlConnection.Dispose(Boolean disposing) at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.DisposeDbConnection() at Microsoft.EntityFrameworkCore.Storage.RelationalConnection.ResetState(Boolean disposeDbConnection) at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Microsoft.EntityFrameworkCore.DbContext.Dispose() at Microsoft.Extensions.DependencyInjection.ServiceLookup.ServiceProviderEngineScope.Dispose() at Hangfire.JobActivatorScope.Dispose() at Hangfire.Server.CoreBackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.BackgroundJobPerformer.<>c__DisplayClass9_0.<PerformJobWithFilters>b__0() at Hangfire.Server.BackgroundJobPerformer.InvokePerformFilter(IServerFilter filter, PerformingContext preContext, Func`1 continuation) at Hangfire.Server.BackgroundJobPerformer.PerformJobWithFilters(PerformContext context, IEnumerable`1 filters) at Hangfire.Server.BackgroundJobPerformer.Perform(PerformContext context) at Hangfire.Server.Worker.PerformJob(BackgroundProcessContext context, IStorageConnection connection, String jobId)

Npgsql.NpgsqlOperationInProgressException

System.InvalidOperationException

System.InvalidOperationException

System.NullReferenceException

System.NullReferenceException

System.InvalidOperationException

Npgsql.NpgsqlOperationInProgressException

Npgsql.NpgsqlOperationInProgressException

System.InvalidOperationException

Npgsql.NpgsqlOperationInProgressException