Custom highly scalable solution for contract testing
Introduction
In the previous story, I explained what is contract testing, why, when and how you can use it in your environment. In this article, I want to focus on how to build your own custom solution. What constraints do both tools have and how to solve them.
Why do you consider and need your own custom solution?
I’ll explain in a short what other tools such as Pact and Spring Cloud Contract are missing or lacking in their implementation.
In general, both tools
- don’t use bi-directional flows as a producer/consumer-driven approach. With Spring Cloud Contract it’s possible with Pact it’ll be way harder to achieve producer-driven flow.
- has its own language for writing the tests. Spring Cloud Contract can read Pact contracts but not the other way around.
- it’s not a language-agnostic approach. You can achieve it by containerizing it but this is an extra layer of complexity
- changes within the contract don’t trigger builds automatically
Specifically for Pact
- requires a self-hosted broker that should be available and accessible for both sides. Could be a problem if you have an external party that requires access to a broker from a Security perspective.
- implements only a consumer-driven approach
- there is no official support for async messages in Python. There is only a community-based solution that helps to make it work.
Particularly Spring Cloud Contract
- requires artifactory for storing the contracts in jar format. The same applies here based on sharing with external parties.
- Java-based and producer-driven approach by default.
- under the hood, it wraps the Wiremock instance
Apart from all downsides of both tools, the custom solution is also built-in mind to solve other bottlenecks that might arise in the future
- store contracts in JSON Wiremock representation for ease of migrating and transitioning to a new tool in the future
- using JSON format will help to generalize contracts for all systems and make contracts language-agnostic with an easy learning curve for all teams
- using custom stages and triggers of pipelines based on the changes
Idea
The goal of the project is to build a highly scalable bi-directional approach with a language-agnostic solution in mind and a flexible transition to a new tool in the future.
Architecture overview
A more in-depth description of the new approach
As you can see on the image above you have a Git repository that orchestrates the contracts within the users. Ideally, it’s good to have a shared Git repository per producer. In this way, you won’t have a monolith and it’s a good separation of concerns and responsibilities.
Let’s walk through the new approach starting from the consumer side.
The consumer should make code changes for receiving messages from the producer. The next step is to create a new contract in a shared Git repository. The shared Git repository will trigger the appropriate producer to validate the contract and if the pipeline is green then we are good to go.
The same applies to the producer but the only difference is after pushing new changes to a contract it’ll trigger pipelines of all consumers.
Let’s dive deep with some code examples
As with any custom solution, it’s good to have some guidelines to follow to set up your first working project and see it in action.
The folder structure should look similar to
In this way, you can fine-tune the triggers and stages within the pipeline per team and per business flow.
The content of the 200.json file might look as simple as that
To distinguish async communication you can add a metadata object
With async messages, you just need to fetch the right contract and test it against your own testing tool which doesn’t require any extra setup.
Producer side you can easily automate because the producer will have multiple consumers and creating a test for each of them are time-consuming and redundant.
Let’s take a look at a simple wrapper that will take all consumer tests and iterate through all of them at once without any code changes on the producer side.
As you can see in the picture above it iterates through all consumer contracts and perform basic validation which you can fine-tune later on. I made the code in Kotlin, but it’s possible to make the same code in any other language of your preference because it fetches the response from the Wiremock stub local server.
Git pipeline integration
Let’s start with a shared Git repository that contains all producer’s contracts.
Create a shared template that everyone can use
And this is how the .gitlab-ci.yml file will look with the template above for every producer/consumer of this Git repository
Now let’s create a template for producer/consumer projects to simplify the integration
As you already can see that I’m using sparse-checkout which gives me the ability to take only files that I need. You require only files that were added or changed from a consumer perspective and all consumer files from a producer perspective. In this way the amount of data that you need to fetch is small and you perform contract testing against only specific contracts.
To finalize the build pipeline on a producer/consumer side you need to include a template for fetching the contracts that was discussed earlier and add a dependency before your code starts performing integration tests. To make it even more separated you can also separate unit tests and integration tests per stage and run them in parallel.
Result with Git pipelines
When the contract was changed on a consumer side
And here is when the contract was changed on a producer side
With Git pipelines you can also set up an alert system if your pipeline failed you can notify a specific team about it.
Conclusion
In this article, I described a highly scalable custom solution that will use a bi-direction approach of contract testing. There are lots of things that you can add to it without increasing the complexity by just adding extra stages into the Git pipeline.
