Instance version: 2021.1.0.2258
Review of production logs by the client revealed several queue entries that failed with following error:
Work queue condition evaluation failed. Command will not be processed.
Queue is setup in a way that there are automatically processed commands with conditions. Logs shown two types of errors without any apparent signs of cascade failure of whole application (they seem to be isolated incidents, but there are around 10 of them):
2022-03-03 09:45:49,061 [233] FATAL Origam.Workflow.WorkQueue.WorkQueueService [(null)] <NOT AVAILABLE> - Queue item processing failed. Id: 44cd47d9-7aa0-4230-be87-a02359938610, Queue: Orders Being Uploaded
System.Exception: Work queue condition evaluation failed. Command will not be processed.
---> System.Xml.XPath.XPathException: Function 'AS:LookupValue()' has failed.
---> Origam.OrigamException: Lookup-Text für Lookup nicht verfügbar: 3ed15824-43df-4e9d-9110-144eecee080b
---> System.Data.DataException: Beim Erhalt eines Einzelwerts ist ein Ausnahmefall aufgetreten.
There is already an open DataReader associated with this Command which must be closed first.
---> System.InvalidOperationException: There is already an open DataReader associated with this Command which must be closed first.
at System.Data.SqlClient.SqlInternalConnectionTds.ValidateConnectionForExecute(SqlCommand command)
at System.Data.SqlClient.SqlCommand.ValidateCommand(Boolean async, String method)
at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean asyncWrite, String method)
at System.Data.SqlClient.SqlCommand.ExecuteScalar()
at Origam.DA.Service.AbstractSqlDataService.GetScalarValue(DataStructureQuery query, ColumnsInfo columnsInfo, IPrincipal principal, String transactionId) in D:\a\1\s\origam-source\Origam.DA.Service\AbstractSqlDataService.cs:line 949
--- End of inner exception stack trace ---
at Origam.DA.Service.AbstractSqlDataService.GetScalarValue(DataStructureQuery query, ColumnsInfo columnsInfo, IPrincipal principal, String transactionId) in D:\a\1\s\origam-source\Origam.DA.Service\AbstractSqlDataService.cs:line 1011
at Origam.Workflow.DataServiceAgent.GetScalarValue(DataStructureQuery query, ColumnsInfo columnsInfo) in D:\a\1\s\origam-source\Origam.Workflow\Service Agents\DataServiceAgent.cs:line 176
at Origam.Workflow.DataServiceAgent.Run() in D:\a\1\s\origam-source\Origam.Workflow\Service Agents\DataServiceAgent.cs:line 608
at Origam.Workbench.Services.DataLookupService.GetDisplayText(Guid lookupId, Hashtable parameters, Boolean useCache, Boolean returnMessageIfNull, String transactionId) in D:\a\1\s\origam-source\Origam.Workbench.Services\DataLookupService.cs:line 349
--- End of inner exception stack trace ---
at Origam.Workbench.Services.DataLookupService.GetDisplayText(Guid lookupId, Hashtable parameters, Boolean useCache, Boolean returnMessageIfNull, String transactionId) in D:\a\1\s\origam-source\Origam.Workbench.Services\DataLookupService.cs:line 373
at Origam.Workbench.Services.DataLookupService.GetDisplayText(Guid lookupId, Object lookupValue, Boolean useCache, Boolean returnMessageIfNull, String transactionId) in D:\a\1\s\origam-source\Origam.Workbench.Services\DataLookupService.cs:line 222
at Origam.Rule.RuleEngine.LookupValue(String lookupId, String recordId) in D:\a\1\s\origam-source\Origam.Rule\RuleEngine.cs:line 1036
at MS.Internal.Xml.XPath.FunctionQuery.Evaluate(XPathNodeIterator nodeIterator)
--- End of inner exception stack trace ---
at MS.Internal.Xml.XPath.FunctionQuery.Evaluate(XPathNodeIterator nodeIterator)
at MS.Internal.Xml.XPath.LogicalExpr.Evaluate(XPathNodeIterator nodeIterator)
at System.Xml.XPath.XPathNavigator.Evaluate(XPathExpression expr, XPathNodeIterator context)
at Origam.Rule.RuleEngine.EvaluateXPath(String xpath, Boolean isPathRelative, OrigamDataType returnDataType, XPathNavigator nav, XPathNodeIterator contextPosition) in D:\a\1\s\origam-source\Origam.Rule\RuleEngine.cs:line 4331
at Origam.Rule.RuleEngine.EvaluateXPath(Object nodeset, String xpath) in D:\a\1\s\origam-source\Origam.Rule\RuleEngine.cs:line 2144
at Origam.Workflow.WorkQueue.WorkQueueService.EvaluateWorkQueueCondition(DataRow queueRow, String condition, String queueName, String transactionId) in D:\a\1\s\origam-source\Origam.Workflow\WorkQueue\WorkQueueService.cs:line 1686
--- End of inner exception stack trace ---
at Origam.Workflow.WorkQueue.WorkQueueService.EvaluateWorkQueueCondition(DataRow queueRow, String condition, String queueName, String transactionId) in D:\a\1\s\origam-source\Origam.Workflow\WorkQueue\WorkQueueService.cs:line 1686
at Origam.Workflow.WorkQueue.WorkQueueService.IsAutoProcessed(WorkQueueCommandRow cmd, WorkQueueRow q, DataRow queueRow, String transactionId) in D:\a\1\s\origam-source\Origam.Workflow\WorkQueue\WorkQueueService.cs:line 1643
at Origam.Workflow.WorkQueue.WorkQueueService.ProcessQueueItem(WorkQueueRow q, DataRow queueItemRow, IParameterService ps, WorkQueueClass wqc) in D:\a\1\s\origam-source\Origam.Workflow\WorkQueue\WorkQueueService.cs:line 1526
2022-03-10 11:00:50,389 [331] FATAL Origam.Workflow.WorkQueue.WorkQueueService [(null)] <NOT AVAILABLE> - Queue item processing failed. Id: f5b52d5b-cd35-4ff3-a4ea-c4f1cc21cb00, Queue: Orders Being Uploaded
System.Exception: Work queue condition evaluation failed. Command will not be processed.
---> System.Xml.XPath.XPathException: Function 'AS:LookupValue()' has failed.
---> Origam.OrigamException: Lookup-Text für Lookup nicht verfügbar: 3ed15824-43df-4e9d-9110-144eecee080b
---> System.Exception: Transaction is already registered.
at Origam.ResourceMonitor.RegisterTransaction(String transactionId, String resourceManagerId, OrigamTransaction transaction) in D:\a\1\s\origam-source\Origam.Common\ResourceMonitor.cs:line 48
at Origam.DA.Service.AbstractSqlDataService.GetTransaction(String transactionId, IsolationLevel isolationLevel) in D:\a\1\s\origam-source\Origam.DA.Service\AbstractSqlDataService.cs:line 271
at Origam.DA.Service.AbstractSqlDataService.GetScalarValue(DataStructureQuery query, ColumnsInfo columnsInfo, IPrincipal principal, String transactionId) in D:\a\1\s\origam-source\Origam.DA.Service\AbstractSqlDataService.cs:line 936
at Origam.Workflow.DataServiceAgent.GetScalarValue(DataStructureQuery query, ColumnsInfo columnsInfo) in D:\a\1\s\origam-source\Origam.Workflow\Service Agents\DataServiceAgent.cs:line 176
at Origam.Workflow.DataServiceAgent.Run() in D:\a\1\s\origam-source\Origam.Workflow\Service Agents\DataServiceAgent.cs:line 608
at Origam.Workbench.Services.DataLookupService.GetDisplayText(Guid lookupId, Hashtable parameters, Boolean useCache, Boolean returnMessageIfNull, String transactionId) in D:\a\1\s\origam-source\Origam.Workbench.Services\DataLookupService.cs:line 349
--- End of inner exception stack trace ---
at Origam.Workbench.Services.DataLookupService.GetDisplayText(Guid lookupId, Hashtable parameters, Boolean useCache, Boolean returnMessageIfNull, String transactionId) in D:\a\1\s\origam-source\Origam.Workbench.Services\DataLookupService.cs:line 373
at Origam.Workbench.Services.DataLookupService.GetDisplayText(Guid lookupId, Object lookupValue, Boolean useCache, Boolean returnMessageIfNull, String transactionId) in D:\a\1\s\origam-source\Origam.Workbench.Services\DataLookupService.cs:line 222
at Origam.Rule.RuleEngine.LookupValue(String lookupId, String recordId) in D:\a\1\s\origam-source\Origam.Rule\RuleEngine.cs:line 1036
at MS.Internal.Xml.XPath.FunctionQuery.Evaluate(XPathNodeIterator nodeIterator)
--- End of inner exception stack trace ---
at MS.Internal.Xml.XPath.FunctionQuery.Evaluate(XPathNodeIterator nodeIterator)
at MS.Internal.Xml.XPath.LogicalExpr.Evaluate(XPathNodeIterator nodeIterator)
at System.Xml.XPath.XPathNavigator.Evaluate(XPathExpression expr, XPathNodeIterator context)
at Origam.Rule.RuleEngine.EvaluateXPath(String xpath, Boolean isPathRelative, OrigamDataType returnDataType, XPathNavigator nav, XPathNodeIterator contextPosition) in D:\a\1\s\origam-source\Origam.Rule\RuleEngine.cs:line 4331
at Origam.Rule.RuleEngine.EvaluateXPath(Object nodeset, String xpath) in D:\a\1\s\origam-source\Origam.Rule\RuleEngine.cs:line 2144
at Origam.Workflow.WorkQueue.WorkQueueService.EvaluateWorkQueueCondition(DataRow queueRow, String condition, String queueName, String transactionId) in D:\a\1\s\origam-source\Origam.Workflow\WorkQueue\WorkQueueService.cs:line 1686
--- End of inner exception stack trace ---
at Origam.Workflow.WorkQueue.WorkQueueService.EvaluateWorkQueueCondition(DataRow queueRow, String condition, String queueName, String transactionId) in D:\a\1\s\origam-source\Origam.Workflow\WorkQueue\WorkQueueService.cs:line 1686
at Origam.Workflow.WorkQueue.WorkQueueService.IsAutoProcessed(WorkQueueCommandRow cmd, WorkQueueRow q, DataRow queueRow, String transactionId) in D:\a\1\s\origam-source\Origam.Workflow\WorkQueue\WorkQueueService.cs:line 1643
at Origam.Workflow.WorkQueue.WorkQueueService.ProcessQueueItem(WorkQueueRow q, DataRow queueItemRow, IParameterService ps, WorkQueueClass wqc) in D:\a\1\s\origam-source\Origam.Workflow\WorkQueue\WorkQueueService.cs:line 1526