Work Queue Improvements

ideas
1

UI improvements

  1. WorkQueue commands - add validation (e.g. command field is mandatory when command type is ‘Queue Command’)
  2. describe fields by tooltip messages (especially in ‘Creation’ tab)
  3. Improve screen loading performance by making them lazy loaded (solved in Performance Improvement for TextFile Work Queue Screen)
  4. Add loader/processor history table so we know when and with what results they were run
  5. Work Queue Groups

Backend

  1. Workqueue - implement round-robin processing
  2. Work Queues - Support for Retry
  3. Implement scheduling (cron) mechanism for each loader (today there is one setting for all the queues), e.g. “At midnight look in the folder and load all the files”
  4. Implement scheduling (cron) mechanism for each queue auto processing (today it is fixed every 10 seconds checking and then processing until done), e.g. “Start processing this queue at midnight”
  5. Consider Hangfire to replace work queue/scheduler
  6. Implement throttling for queue processing
  7. Implement individual queue entry scheduling (e. g. “Process this entry next week”)
  8. WorkQueue LoadExternalWorkQueues vs. autoprocessing step
  9. Make it possible to easily see the state of external loaders – is it in errror? When did it run last time, how long it took, is it running just now? Maybe even users should see a message in the frontend, that the current queue has problems and they should not expect anything to arrive and contact an admin.
  10. Use case - there are 10 queues which time out in 1 minute because of an error. One round of loading external queues will take 10 minutes. Some queues do not time out and those could run more often (in the right interval). Scheduling of loaders (cron) might also be a solution, so it would be clear at which time it will run next and so it would be possible to “skip” other queues by retrying (we have a queue command for manually executing a loader).
2

Let’s put all the work queue improvement ideas here please so we can plan one major release with all of them.

3

If the Hangfire’s licensing policy is not suitable for Origam policies, you can consider similar component https://www.quartz-scheduler.net/ which is under Apache License v2.0.

I believe it would be easier to maintain application if there would not be necessary to configure scheduler out of the application.

1 Like