Externalizing configuration

You may have noticed that over time the configuration of the services becoming richer and more complex. Deploying hundreds of services into production environments increases such complexity 100 times fold.

This means that we need to provide a consistent way of handling service configuration.

You have to work out a convention that works for your organization and stick to it. Inconsistency and "snow flakes" increase complexity, introduce points of failure and increase troubleshooting time.

Spring Boot uses a very particular PropertySource order that is designed to allow sensible overriding of values, properties are considered in the following order:

Command line arguments.
Properties from SPRING_APPLICATION_JSON (inline JSON embedded in an environment variable or system property)
JNDI attributes from java:comp/env.
Java System properties (System.getProperties()).
OS environment variables.
A RandomValuePropertySource that only has properties in random.*.
Profile-specific application properties outside of your packaged jar (application-{profile}.properties and YAML variants)
Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)
Application properties outside of your packaged jar (application.properties and YAML variants).
Application properties packaged inside your jar (application.properties and YAML variants).
@PropertySource annotations on your @Configuration classes.
Default properties (specified using SpringApplication.setDefaultProperties).

Extracting configuration from code

The solution to this exercise part can be found in solution1 folder.

Let’s start with the basic controller:

src/main/java/msvcdojo/AccountsServiceApplication.java

@RestController
class HomeController {

    private String name = "World";

    @RequestMapping("/")
    String home() {
        return "Hello, " + name + "!";
    }
}

The variable name has a hard-coded value. We’re going to externalize it into an application.yml file.

Create the src/main/resouces/application.yml file add the following code:

src/main/resouces/application.yml

server:
  port: ${PORT:8100}

name: World

Annotate the variable name with the @Value attribute. Your controller will look like this:

src/main/java/msvcdojo/AccountsServiceApplication.java

@RestController
class HomeController {

    @Value("${name}")
    private String name;

    @RequestMapping("/")
    String home() {
        return "Hello, " + name + "!";
    }
}

Build and start the service:

$ gradlew assemble
$ java -jar build\libs\mysvc-0.0.1.jar

Play time

Can you guess what will be the result value when we’re going to hit the controller with the basic request?

$ curl http://localhost:8100
Hello, World!

That was simple, right?

Now let’s up the game, shall we?

Stop the service, set environment variable and start the service again:

Windows

$ set name=John
$ java -jar build/libs/mysvc-0.0.1.jar

Mac

$ export name=John
$ java -jar build/libs/mysvc-0.0.1.jar

Start the service and validate the result:

$ curl http://localhost:8100
Hello, John!

The OS environment variable takes the precedent over the value provided in application.yml file.

Let’s override the environment variables as well with the command-line parameters:

$ java -DPORT=8222 -Dname=Alice -jar build/libs/mysvc-0.0.1.jar

Note that the service will start listening on the port 8222.

$ curl http://localhost:8222
Hello, Alice!

The -D parameter will override the OS environment variables values according to the order in which the the values are resolved (see the list at the top of this guide).

Note the value of server.port setting:

src/main/resouces/application.yml

server:
  port: ${PORT:8100}

The notation of ${PORT:8100} expands another variable PORT but, if the value was not resolved, falls over to the default value, provided after : sign.

Resolution troubleshooting

While the service running hit the actuator endpoint /env:

$ curl http://localhost:8222/env

The result will look like the following:

{
  "profiles": [],
  "server.ports": {
    "local.server.port": 8222
  },
  "servletContextInitParams": {},
  "systemProperties": {
    "java.runtime.name": "Java(TM) SE Runtime Environment",
    "PORT": "8222",
    "name": "Alice",
...
  },
  "systemEnvironment": {
    "NUMBER_OF_PROCESSORS": "4",
    "name": "John",
...
  },
  "applicationConfig: [classpath:/application.yml]": {
    "server.port": "${PORT:8100}",
    "name": "World",
    "endpoints.health.sensitive": false,
    "endpoints.shutdown.enabled": true
  }
}

If you’ll analyze the file, you may notice the 3 different values of name variable, i.e. systemProperties, systemEnvironment and applicationConfig. The configuration resolution will pick the first one it’ll find, i.e. the one in systemProperties.

This allows you to see and understand where the values are coming from.

Configuration Server

In this section we’re going to go one step further. We’re going to start preparing our service for production deployment.

In production environment, where you’re going to have more than one instance of each service for scale, there will be a challenge to manage multiple settings across a range of hosts for each service. While the settings are not static it’ll become increasingly complex to keep all the settings consistent across all of the instances of the service.

To solve this problem we’re going to build a configuration server.

Building Configuration Server

Create an empty Spring Boot application.

In the build.gradle file add the following project variables:

config-service/build.gradle

project.ext {
    springBootVersion = '1.3.1.RELEASE'
    springCloudVersion = '1.0.3.RELEASE'
}

Update the spring-boot-starter-actuator dependency and add the spring-cloud-config-server dependency, so we get:

config-service/build.gradle

compile("org.springframework.boot:spring-boot-starter-actuator:$springBootVersion")
compile("org.springframework.cloud:spring-cloud-config-server:$springCloudVersion")

Add @EnableConfigServer annotation for the application class:

src/main/java/msvcdojo/ConfigurationServerApplication.java

@EnableConfigServer
public class ConfigurationServerApplication {

    public static void main(String[] args) throws Exception {
        SpringApplication.run(ConfigurationServerApplication.class, args);
    }
}

Configuration Location (local file system)

Create application.yml file where you can specify location where the service configurations can be found:

src/main/resources/application.yml

spring:
  profiles:
    active: native
  cloud:
    config:
      server:
        native:
          searchLocations: file:./config/

Operating Configuration Server (local file system)

Create a config folder and move there the application.yml configuration file from mysvc service with the name my-service.yml:

config-service/config/my-service.yml

server:
  port: ${PORT:8100}

name: World

endpoints:
  health:
    sensitive: false
  shutdown:
    enabled: true

Launch the config server and try to get the service configuration:

$ java -DPORT=8888 -jar build\libs\config-service-0.0.1.jar
$ curl http://localhost:8888/my-service.yml

If everything configured properly, you’re going to see the contents of the my-service.yml file.

Configuration Location (on Git)

Create application.yml file where you can specify location where the service configurations can be found:

config-service/src/main/resources/application.yml

spring:
  cloud:
    config:
      server:
        git:
          uri: ${conf:https://github.com/Accordance/microservice-dojo-configuration}

Operating Configuration Server (with Git)

For this exercise the uri in the application.yml file above has to be replaced with your GitHub repo’s uri.

Create an empty GitHub repo. Clone it locally, and move there the application.yml configuration file from mysvc service with the name my-service.yml:

config-service/src/config/my-service.yml

server:
  port: ${PORT:8100}

name: World

endpoints:
  health:
    sensitive: false
  shutdown:
    enabled: true

Commit and push to the GitHub.

Remember to set the uri in the config-service’s application.yml to your GitHub repo’s uri.

Launch the config server and try to get the service configuration:

$ java -DPORT=8888 -jar build\libs\config-service-0.0.1.jar
$ curl http://localhost:8888/my-service.yml

If everything configured properly, you’re going to see the contents of the my-service.yml file.

Using Config Server on the Client Side

Prepare the Configuration Service

Update the config-service application.yml.

First set the default port:

config-service/src/main/resources/application.yml

port: ${PORT:8888}

Then revert the uri to the standard GitHub location:

config-service/src/main/resources/application.yml

spring:
  cloud:
    config:
      server:
        git:
          uri: ${conf:https://github.com/Accordance/microservice-dojo-configuration}

And launch the config server.

Prepare MySvc

Delete the mysvc/src/main/resources/application.yml file (it has been moved to the Config Server).

Create a mysvc/src/main/resources/bootstrap.yml file, where you specify a service ID as well as the location of the config server URI:

mysvc/src/main/resources/bootstrap.yml

spring:
  application:
    name: my-service
  cloud:
    config:
      uri: http://localhost:8888

Build and start the service:

$ gradlew build
$ java -jar build\libs\mysvc-0.0.1.jar

If everything was done properly, when we hit the REST endpoint, you’re going to see the following:

$ curl http://localhost:8100
Hello, World!

Running Config Server in Docker

Follow the instructions for building docker image for Config Server:

$ gradlew clean prepDocker
$ docker build -t msvcdojo/config-service:0.0.1 build/docker

Launch Config Server as docker container and see if you get MySvc configuraiton:

$ docker run -d --name=config-service -p 8888:8888 msvcdojo/config-service:0.0.1
$ curl http://dockerhost:8888/my-service.yml

Remember that on Windows, the mysvc/src/main/resources/bootstrap.yml file has to be updated, so the URI now points to http://dockerhost:8888.

Optional: Push Config Server image to your Docker Registry and now you can always get it with the docker run command.