Uploaded image for project: 'StreamSets Data Collector'
  1. StreamSets Data Collector
  2. SDC-4793

Ability to pass parameters to pipeline start CLI/API/UI

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: P2 (Critical with Workaround)
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.5.0.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Sprint:
      Sprint 51
    • Doc Impact:
      Complete
    • Doc Impact Details:
      Hide
      Sample CLI Command:
      bin/streamsets cli -U http://localhost:18630 manager start -n
      60947387-ae9a-4a6c-aa33-ae4cc832e9dd --runtimeParameters '{"fields":
      "x", "fromField": "/x", "toField": "/changedField"}'

      Sample CURL Command:
      curl
      'http://localhost:18630/rest/v1/pipeline/ae18bc20-54cd-4036-acf9-3424dda
      4d394/start?rev=0' --user admin:admin -H 'X-Requested-By:
      DataCollector' -H 'Content-Type: application/json;charset=UTF-8'
      --data-binary '{"fields": "x", "fromField": "/x", "toField":
      "/changedField"}'


      Also new panel added in summary view for viewing Runtime Constants passed along with start command
      Show
      Sample CLI Command: bin/streamsets cli -U http://localhost:18630 manager start -n 60947387-ae9a-4a6c-aa33-ae4cc832e9dd --runtimeParameters '{"fields": "x", "fromField": "/x", "toField": "/changedField"}' Sample CURL Command: curl ' http://localhost:18630/rest/v1/pipeline/ae18bc20-54cd-4036-acf9-3424dda 4d394/start?rev=0' --user admin:admin -H 'X-Requested-By: DataCollector' -H 'Content-Type: application/json;charset=UTF-8' --data-binary '{"fields": "x", "fromField": "/x", "toField": "/changedField"}' Also new panel added in summary view for viewing Runtime Constants passed along with start command

      Description

      For example, if a user needs to parameterize an origin with a directory on each run, or with a date string to indicate the proper directory to pull from. Ideally, we would be able to pass in parameters like you could into a bash script with an array of arguments ${pipeline:argv(0)} or something like that.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                madhukard Madhukar Devaraju
                Reporter:
                natty Jon Natkins
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: