CI/CD - Continuous integration and continuous delivery

 Old school fashion Integration

Let us look into the detail how things are happening before CI/CD process, the old school way.

If we take an example of a E-Learning application, there will have product management who creates the vision and defines the requirements for the application according to the market needs.

There will have an IT depart who develops, deploys and maintains the application according to the requirements from product team.

If we do a closer look into the IT department, there will have programers or different team of programmers who develops the feature according to the instructions from product team.

Once it is developed, the source code is passed to build and integration team who then builds these different source code and compile it.

Once it is integrated and compiled, it is passed to operations team who take care the deployments. They deploy first in test environment and then passed to Quality Assurance team.

Quality assurance team who work closely with product management team ensures it is defect free.

If any defect found, the same cycle is again repeated since programmers needs to fix it first.

Integration effort becomes more painful if it is a large team or multiple programmers are working on the same product. Merging the multiple source code branches into a master branch is a tedious work.

Pain points of Old fashion deployment and testing process

Integration is painful and Effort Consuming

Merging the control from source branches to main branched happens only once in an iteration. The integration team sits with one representing developer from each module and they figure out how to integrate it into single one to release. It is a huge task, consume a lot of time and error-prone manual process

Fixing issues at the end of iterations

 Integration happens at the end of iteration only. After integration, it is passed to QA and they check whether it has any defects. If defects found, it is again passed to the developers to fix it and the same process repeated again. It will consume a lot of time to go back to the piece of code they have written earlier and fix it since defects are found at end of the iteration and developer may have moved to other feature development in the same or different project.

Immediate merge issues can hold up teams

When there is an inter-module dependencies, developers try to merge these batches earlier without waiting for at the end of the cycle. This can lead to hold up the development team to solve the errors in merging the branches intermediate. This might take hours or even days.

Long feedback cycle for functional defects

Real functional test happens at end of the iteration and this leads to a long feedback cycle for developers. Functional defects won’t get identified during the compilation and developers may not be able to find out it during their unit tests, it may have dependencies with other module too. So, they see the real working of the entire feature or functionality at end during the QA process only. Ideally developers would like to get feedback during the development stage itself.

Long iterations

Long iteration is a cumulative effect of above all which included huge time of manual integration, time spend to fix the compilation errors, late defect feedback which will initiate the iteration again. 

Introduction to Continuous Integration

Continuous Integration is a development practice the requires developers to integrate code in the shared repository several times a day. This helps to avoid the integration by a different team, development team itself who has clear picture about their code integrates it.

For this all developers should use same source control repository. They can use different branches, but they should frequently merge merge to the mainline branch.

To make sure that the code actually fits together and works, it should be compiled regularly. It should compile automatically after each and every code commit by developer at any time. We will use build server instead of computer used by build and integration team.

So, whenever there is a code commit, it should be compiled automatically in this build server and developer. If there is any error, the failure details will be automatically mailed to the developer and build and compilation result will posted in the portal where developers can refer and fix it.

Apart from compilation of code, the build should also have automated tests that do various checks to ensure the correctness and quality of the code. These tests will run against the latest version of the code.

An important requirement for continuous integration is that the whole build process which includes compilation and running the tests should be quick, not more than a few minutes.

This way the results of the tests can be communicated to the developers quickly, who can then fix any issues that pop up.

The automated test will include both unit test and automated UI test. Automated UI test will find out the functionality defects. This way developer is early notified with short feedback cycle through automated unit and UI test.

Organizations that adhere to these principles for development and deployment are adopted the continuous integration. Continuous integration doesn’t mean simply automating the build process. The developer should follow the above working style as well, checking code frequently at least once in a day, fixing the errors immediately.

 Following, we can consider as a golden rule for this CI/CD integration

- A single central repository where the code lives. 

- Developers check in their code frequently, typically every few hours, but atleast once a day.

- Build should be triggered every time a developer checks in code.

- Build should be automated and should be fast - it can be executed many times and developer get immediate feedback

- Build should compile the code as well as run automated tests on the compiled code.

- Fixing a failed build should be top priority for the concerned developers

- Build results including automated test results should always be visible to all developers. Emails should be sent out in case of failures.

Comparison

CI/CD Pipelines

CI/CD pipelines are series of steps to take the source code to production. It has different steps such as building, packaging, testing, validating, verifying infrastructure and deploying into the production. We can divide into integration pipeline and release pipeline.

The integration pipeline can be divided into 5 stages - Code Integration, Compilation, Packaging, Running Automated Unit tests and Automated UI tests.  When there is a multiple build process triggered, it will go in sequence to each steps. ie, one build process will be in packing stage and another will be in testing stage. This improves the efficiency of the build process.

 Integration pipeline

 

Continuous Delivery
Before Continuous delivery - The old school model
Once the software is integrated and packaged, it is provided to operations team who then deploys the packages in different environments.
Operations team will receive the set of instructions to be followed while deploying in different environments.
Using these instructions, they prepare environments needed and deploy in this different environments such as test, production, etc

Pain points of old school fashion continuous delivery
- First one is the correctness of instructions
- Difference in instructions across environments
- Error prone manual tasks
- Deployments are sophisticated, high down time

Continuous Delivery in Action
Continuous delivery is a software development practice where software can be released to production at any time.Continuous integration is a per-requisite for Continuous delivery. 

Once the code is checked in by developers, it gets compiled and built automatically at least once every day. 

The unit testing and automated UI testing also will be completed during the continuous integration process.

Once these steps are completed in Continuous integration, these packages should be deployed in test and production environments.

Instead of manual instructions to create and deploy the environments, we will create the scripts to this process. This part also automated.

It will be deployed in test environments and there will have manual process by QA team to verify.

Once it is passed, it will be deployed in production environment using the scrips prepared for the same.

Pain points solved by Continuous Delivery

 

The Complete View of Continuous Integration and Deployment

 

Towards Continuous Deployment 

Continuous deployment and continuous delivery is always a confusing terms.

In Continuous delivery, there will have a human intervention in between the steps. For example, there will have a User Acceptance Testing before pushing the data code changes to production.

On other hand, continuous deployments is a process where there will not have any human intervention to push the code changes to live once code is pushed/committed from developer side. To achieve this, we may need to bring automated User acceptance testing.

There will not have any human intervention between code commit and release to production. So, it is better to provide only few developers should have access to commit the final code in this case. Only enterprises that reached a maturity level should go for Continuous Deployment because of risks involved in that

CD stands for. People used to tell different terms.

CD - Continuous Delivery

CD - Continuous Deployment

CD - Continuous Delivery/Deployment

 All the above are correct. Just understand the difference, that is the key

Continuous Deployment - Phased Rollout

In phased rollout, the release will be done in phase by phase. Initially, we release in the few regional IPs. If things are going well, we do it another set of region and then later release to all regions completely.