Uploaded image for project: 'StreamSets Data Collector'
  1. StreamSets Data Collector
  2. SDC-13107

Allow "exclusion list" field in SFTP origin stage

    XMLWordPrintable

    Details

    • Type: New Feature
    • Status: Open
    • Priority: P3 (Limited Impact)
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: sftp
    • Labels:
    • Testing Status:
      STF Testing Required
    • Team:
      Data Plane
    • Stage:
      SFTP/FTP Client origin, SFTP/FTP/FTPS Client origin

      Description

      At present, there are only two methods for inclusion/exclusion of certain files in an SFTP origin stage: configuring the glob expression to only match desired files, or utilizing the "first file to process" property so that any files older than the one specified are ignored.

      It would be useful to be able to set an explicit "exclusion" blacklist, similar to the glob expression for file name pattern matches, that could be used to identify files that should not be processed (even if they match the file name pattern for files to process).

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            mitch Mitch Barnett
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: