Uploaded image for project: 'StreamSets Data Collector'
  1. StreamSets Data Collector
  2. SDC-14852

LocalFS event isn't consistent

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: P3 (Limited Impact)
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.19.0
    • Component/s: None
    • Labels:
    • Testing Status:
      STF Testing Required
    • Team:
      Data Plane
    • STF Test:

      Description

      Normally the filepath attribute is just a directory, but on certain condition it can include full URL:

      {"filename":"sdc-06251c85-6a73-11ea-9de0-6949e631745f_4e8d9d1d-298a-49cc-a903-5b9a06274c97","filepath":"file:/tmp/out/input/sdc-06251c85-6a73-11ea-9de0-6949e631745f_4e8d9d1d-298a-49cc-a903-5b9a06274c97","length":2415}
      

      This is inconsistent behavior that requires downstream users to know what to expect. Way to repro the full URL:

      1) Import the attached pipeline and start the SDC
      2) In the customer environment, due to some reason, SDC got crashed (for local testing, we can kill the SDC by using kill -9 command)
      3) Start the SDC again
      4) Now, in the event file, we can see the entry which contains _*file:*_ in filepath attribute

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              konstantin Konstantin Golub
              Reporter:
              jarcec Jarcec
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: