Uploaded image for project: 'StreamSets Data Collector'
  1. StreamSets Data Collector
  2. SDC-13261

When processing subdirectories in ADLS Gen2, pipeline finisher throws false errors.

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: P4 (Minor Impact)
    • Resolution: Not an Issue
    • Affects Version/s: 3.12.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Testing Status:
      STF Testing Required
    • Team:
      Data Plane

      Description

      When processing subdirectories in ADLS Gen2, pipeline finisher throws false errors stating "COMMON_0001 - Stage precondition: CONTAINER_0051 - Unsatisfied precondition(s) '${record:eventType() == 'no-more-data'}'".

      The pipeline finisher stops the pipeline but we see a large number of these false errors stating the pipeline finisher will not trigger. 

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            nathan Nathan Le
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: