Stream Data Page

ideas
1

There is a requirement to export large dataset via API. The current Data Page is not suitable because it loads the entire dataset into memory before sending it to the client. For large datasets, this can put significant pressure on server resources.

I suggest implementing a new type of API Page - Stream Data Page. It should stream data directly to the client without loading the entire dataset into memory first. It also needs to handle client disconnection gracefully.

This means introducing a new model element StreamDataPage and its corresponding API handler.

The model element is similar to Data Page with following differences:

  • Paging is not supported.
  • Custom filters are not supported
  • The underlying dataset should have only one output entity (filter parent, join relations should be supported).
  • Transformation are not supported.
  • Only the GET method is supported.
  • utput format can be XML or JSON.
  • JSON output should always use OmitJsonRootElement = true and OmitJsonMainElement = true. There won’t be configuration options.