Introduction
In the first part of this article, talking about Microservice Configuration, we have seen how to set up a simple scenario with a Spring Cloud Config server and the client counterpart. The server was set with a native profile, and the configuration was stored in the classpath, in a subfolder named config. The demo was made of a single instance of a server and client.
In this part, we will show how to configure Spring Cloud Config to connect and use an external Git repository. Then, in the next sections, we will talk about refreshing the configuration data without restarting the services.
Microservice Configuration by Git Repository: Server Side
Spring Cloud Config uses a Git repository by default. In part I of this article we have seen how to switch on a repository based on filesystem/classpath, setting the property spring.profiles.active to the native value. To use a Git repository we have to set the following instead:
spring:
cloud:
config:
server:
git:
uri: https://github.com/mcasari/spring-cloud-config-git-server-repo.git
username: ${github.username}
password: ${github.password}
cloneOnStart: true
We have set a URI to a sample public repository in the example above. Even if not required in this case, we have also put a username and password just to show how we would configure access to a protected private repository. For a private repository, we can pass the credentials of a user who has access to that repository and as a password, we can set a security token expressly generated using the GitHub administration panel. In the example, we have used placeholders for both username and password values, which can be provided as Java arguments in the startup command:
java -jar -Dgithub.username=<myusername> -Dgithub.password=<mygithubtoken> spring-cloud-config-git-server-1.0-SNAPSHOT.jar
In the case of a public repository, username and password will be simply ignored.
The cloneOnStart property allows us to configure when the server is supposed to clone the repository. By default, the server does this when the configuration data is requested for the first time. If we set it to true, it will be triggered in the startup phase instead.
Microservice Configuration by Git Repository: Client Side
From the standpoint of the client side, nothing changes compared to the discussion in the first part of this article. We can use the following piece of configuration in the application.yml file if we want to contact the server using the default host and port (localhost:8888):
config:
import: "optional:configserver:"
or, if we have to set a more specific address:
config:
import: "optional:configserver:http://configserverhost:8888"
And if the server is protected by a username and password, the following is also needed:
cloud:
config:
username: myusername
password: mypassword
Running Server and Client
We can run the server and client in the same way we did for the native profile scenario:
java -jar spring-cloud-config-git-server-1.0-SNAPSHOT.jar
...
java -jar spring-cloud-config-git-client-1.0-SNAPSHOT.jar
Microservice Configuration by Git Repository: Reloading Configuration
Let’s turn back a while to the configuration described in the previous sections. If we clone the Git repository locally, make some changes to the properties, commit, push them, and check the config server by its endpoints, we will see the updated values. But, if we call the client REST services, we will still see the old values. This is because the client needs its Spring context to be reloaded in some way, to refresh the bean properties.
Reloading Spring Beans Manually
Spring Cloud provides a specific actuator ‘refresh‘ endpoint, that we can call by HTTP POST request. This endpoint has the effect of reloading the beans marked with the @RefreshScope annotation. In order to build a simplified demo we can avoid introducing a security layer in the client application. We can achieve this by simply not setting a security starter in the Maven POM. This way we will be able to call the refresh endpoint freely.
Let’s suppose we have a bean defined like this:
@Component
@RefreshScope
@ConfigurationProperties(prefix = "myproperties")
public class DemoClient {
private List<String> properties = new ArrayList<String>();
public List<String> getProperties() {
return properties;
}
@Value("${myproperty}")
private String myproperty;
public String getMyproperty() {
return myproperty;
}
}
And a controller class with two REST services:
@RestController
public class ConfigClientController {
private static final Logger LOG = LoggerFactory.getLogger(ConfigClientController.class);
@Autowired
private DemoClient demoClient;
@GetMapping("/getProperties")
public List<String> getProperties() {
LOG.info("Properties: " + demoClient.getProperties().toString());
return demoClient.getProperties();
}
@GetMapping("/getProperty")
public String getProperty() {
LOG.info("Property: " + demoClient.getMyproperty().toString());
return demoClient.getMyproperty();
}
}
Then, let’s make some changes to the properties and myProperty fields, commit and push them, and finally call the refresh endpoint by an HTTP POST request:
curl -X POST http://localhost:8081/actuator/refresh
If we then call the two client REST services, we will see the updated values.
Reloading Spring Beans Automatically
Clearly, the option of manually reloading the configuration properties is not the best solution. We would prefer a scenario in which after a Git push operation the system is automatically updated to the new configuration state, without any restart and without the need to manually execute a refresh on specific beans. The picture below describes an available architecture implementing such a feature.
In the above diagram, we can see a remote Git repository, that can send notifications to the config server as some commit is pushed on the main branch. The config server then dispatches those notifications to a message broker, and the clients can connect to the message broker to consume them. For simplicity, we can limit ourselves to considering just one client.
To make the first part of the above architecture work we have to make some configurations. GitHub, like other Version Control Software tools, provides a feature named Webhook. A Webhook is a definition of an endpoint that GitHub can invoke passing the information involved in a push to the main Git branch.
A /monitor endpoint is available in Spring Cloud Config and can be activated by the spring-cloud-config-monitor dependency:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-monitor</artifactId>
</dependency>
and adding a piece of configuration in the application.yml file:
spring:
cloud:
config:
server:
monitor:
github:
enabled: true
We can configure the /monitor endpoint as a Webhook in the GitHub repository, to make GitHub call it to notify any changes to the config server.
Once the above pieces are put in place, the next piece for the architecture is a message broker that acts as a funnel for events coming from the config server and originating from a Webhook notification and as a source for the clients that are meant to consume those notifications (as events of type RefreshRemoteApplicationEvent). We can choose between Kafka and RabbitMq as message brokers. In this article, we will use RabbitMq.
To run a demo of the above architecture in a local environment, the easiest way would be to run a docker image of RabbitMq:
docker run -d --name rabbit -p 5672:5672 -p 15672:15672 rabbitmq:management
Both client and server need the following dependency to be able to connect to the RabbitMq message broker:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bus-amqp</artifactId>
</dependency>
And the following configuration in the application.yml file:
rabbitmq:
host: localhost
port: 5672
username: guest
password: guest
Once we run all the pieces, the config server and the client will be bound to the message broker by a RabbitMQ exchange named SpringCloudBus, as we can see in the following screenshot.
Testing the above architecture would be problematic in a local environment: we can connect our config server to the remote Git repository but not vice-versa. To overcome this situation, we can simulate the Webhook feature by making a dummy HTTP POST pretending it is a real Git push notification. We can use any tool we want to make the POST call, here we use a curl command:
curl -H "X-Github-Event: push" -H "Content-Type: application/json" -X POST -d '{"commits": [{"modified": ["client-service.yaml"]}]}' http://localhost:8888/monitor
In this call, we pass the type of event as a specific HTTP header, and in the JSON body of the POST request, we specify what file has been modified.
Note: In a windows system we will have problems with the single quote character, so we have to replace them with double quotes and escape those inside the JSON content, like this: curl -H “X-Github-Event: push” -H “Content-Type: application/json” -X POST -d “{\”commits\”: [{\”modified\”: [\”client-service.yaml\”]}]}” http://localhost:8888/monitor
Quick Recap
To summarize, to run a full example test we can do the following steps:
- Run the RabbitMq instance by docker.
- Run the config server and client applications by using their executable jars with the java -jar command.
- Clone the remote repository, make some changes to the configuration properties, commit, and push them.
- Simulate a Webhook interaction, making an HTTP POST call to the config server /monitor endpoint by curl or any other tool.
- Call the client REST services exposing the configuration properties: if all works as expected we should see the updated values printed on the screen.
Conclusion
We have seen how to serve the microservice system configuration out of a Git repository. This is the default and the most common way. We have also seen how to refresh the configuration properties without the need to restart the application.
You can find the server and side parts used as samples in this article on the following GitHub projects:
The repository is also available on GitHub, as already mentioned in this post: