Recently there have been some improvements with logging on the Ingest service. Some analysis are still hard to do, for instance to measure how long it takes to load a particular table/API endpoint if there are multiple tables within one Transfer Task.
Prepare instances have detailled logging per table and per step, including a gantt chart. This way you can easily identify tables/steps with bad performance.
Currently it can be quite time-consuming to identify slow tables/steps in an Ingest instance. A more detailled logging per step (data transfer, build parquet etc) with start- and endtimestamps, a gantt chart etc. would be a huge time saver.
Please also see
