Spring Cloud Data Flow Server - Local

The Data Flow Server implementation for deploying Spring Cloud Stream and Spring Cloud Task apps locally by spawning a new JVM process on the local machine.

Note	The local version of the Data Flow server should only be used for development and in environments where production grade resilience is provided via means other than Data Flow itself.

Getting Started

This shows how to start the Data Flow Server and Shell to create the time | log stream.

Start the Data Flow Server and Shell

1. Start Kafka locally (e.g. kafka-server-start.sh /usr/local/etc/kafka/server.properties)

2. Start Redis locally via redis-server

3. Build from the spring-cloud-dataflow root directory:

   ./mvnw clean install

4. Start the Local Data Flow Server application:

   java -jar spring-cloud-dataflow-server-local/target/spring-cloud-dataflow-server-local-<version>.jar

5. Start the shell

   $ java -jar spring-cloud-dataflow-shell/target/spring-cloud-dataflow-shell-<version>.jar

Creating the time | log stream:

1. Create the 'ticktock' stream:

   dataflow:>stream create --name ticktock --definition "time | log"
   Created new stream 'ticktock'

   This is equivalent to the following HTTP POST request

   $ curl -X POST -d "name=ticktock&definition=time | log" http://localhost:9393/streams/definitions?deploy=false

2. List all streams available in the repository:

   dataflow:>stream list
   ╔═══════════╤═════════════════╤══════════╗
   ║Stream Name│Stream Definition│  Status  ║
   ╠═══════════╪═════════════════╪══════════╣
   ║ticktock   │time | log       │undeployed║
   ╚═══════════╧═════════════════╧══════════╝

   This is equivalent to the following HTTP get request

   $ curl http://localhost:9393/streams/definitions

3. Deploy the 'ticktock' stream:

   dataflow:>stream deploy --name ticktock
   Deployed stream 'ticktock'

   This is equivalent to the following HTTP get request

   $ curl -X POST http://localhost:9393/streams/deployments/ticktock

If successful you should see output similar to the following in the Data Flow Server console:

...o.s.c.d.spi.local.LocalAppDeployer    : deploying app ticktock.log instance 0
   Logs will be in /some/path/ticktock.log

If you tail the stdout_0.log file from the directory mentioned, you should see output similar to the following:

2016-04-26 15:10:18.320  INFO 59890 --- [pool-1-thread-1] log.sink    : 04/26/16 15:10:18
2016-04-26 15:10:19.322  INFO 59890 --- [pool-1-thread-1] log.sink    : 04/26/16 15:10:19
2016-04-26 15:10:20.322  INFO 59890 --- [pool-1-thread-1] log.sink    : 04/26/16 15:10:20

Configuration

Default

To configure the Data Flow Server you can follow the configuration setup guidelines specified in the boot documentation found here

Note: The dataflow-server.yml containing the defaults can be found here

Spring Cloud Configuration

The Spring Cloud Data Flow Server offers the user the ability to configure properties via spring-cloud-config. All configurations retrieved from the cloud config will take precedence over Boot’s defaults enumerated above. The Spring Cloud Data Flow Server will look for the server at localhost:8888, however this can be overwritten by setting the spring.cloud.config.uri property to the desired url.

Cloud-Config-Server configuration

To specify a repository in the cloud config server configuration.yml for the Data Flow Server, setup a repo profile with the pattern spring-cloud-dataflow-server. For example:

spring:
  cloud:
     config:
       server:
         git:
           uri: https://github.com/myrepo/configurations
           repos:
            spring-cloud-dataflow-server:
              pattern: spring-cloud-dataflow-server
              uri: https://github.com/myrepo/configurations
              searchPaths: dataFlowServer

Fail Fast

In some cases, it may be desirable to fail startup of a service if it cannot connect to the Config Server. If this is the desired behavior, set the bootstrap configuration property spring.cloud.config.failFast=true and the client will halt with an Exception.

Note

If the Data Flow Server cannot connect to the cloud config server, the following warning message will be logged:

`WARN 42924 --- [           main] c.c.c.ConfigServicePropertySourceLocator : Could not locate PropertySource: I/O error on GET request for "http://localhost:8888/spring-cloud-dataflow-server/default":Connection refused; nested exception is java.net.ConnectException: Connection refused`

To disable the cloud config server set the spring.cloud.config.enabled property to false.