How to Configure Gitlab CI for Android and Firebase App Distribution to Deliver Builds

1. To prepare builds automatically for every push and Merge request;
2. To perform tests on builds and not to allow merges of the branches which have not passed tests;
3. To deliver builds to interested parties (testers, managers, clients, other developers, etc.).

This article may be useful both for people with experience in CI and for Android developers who are new to CI and DevOps in general. That is why in addition to the description of Gitlab CI and Firebase App Distribution configurations we will also discuss what CI/CD and Docker are. The article presumes that your repository is located in Gitlab. As a computing landscape for creating the builds we will use Gitlab itself, which allows us to use 2000 free minutes a month for building (you can purchase more minutes, but for a small team free limit should be just enough. You can also easily set the building processes on your computing landscapes).

To put it in a complex way - CI/CD (Continuous Integration, Continuous Delivery) is a technology of testing automation and delivery of new modules of a developed project to interested parties (developers, analysts, quality engineers, end users, etc.).

To put it simple:

• CI means that when you perform a push to a remote repository -> a trigger activates and the app builds remotely, the tests are performed, APK is created – this is called Continuous Integration.
• CD means that after the previous step this build is hidden somewhere – it would be tiresome to look for it, download it, explain to others where it is located – that is why the build is sent automatically to the place where it could be easily received – this is called Continuous Delivery.

The builds will take place on Gitlab resources, with the use of free demo (2000 minutes).

To start building you have to put a special file .gitlab-ci.yml into the repository root. We will talk about it later.

1. The CI server sees that something needs to be built (a push occured or another trigger activated, for example, during a manual start) and starts the pipeline. The pipeline is a top-level component which consists of several jobs. For instance, building a dev build is one job, building a prod build is a second job, deploying is the third job.
2. The CI server takes the first job from the pipeline and puts it in the execution queue; 
3. The available agent takes the job and downloads the repository
4. The agent starts Docker:
   • Simply put, Docker is a lightweight virtual machine. It deploys linux-based images; for example, it can deploy Ubuntu. The CI system will put our repository into this machine. After the deployment VM can start working according to the prepared script (it can be described in .gitlab-ci.yml). For example, in this script you can write a set of commands, which will download required packages, Android-sdk, command-line-tools, etc. After you've downloaded them you can execute the usual ./gradlew assembleDebug (build, in other words) and get the required APK.
   • But there is one important moment! You can skip the download of Android-sdk, command-line-tools and everything else at every start by simply embedding them into the image. In other words, you can take an Ubuntu image and create a new image based on it, it will be the same Ubuntu + all the required dependencies are built in. This is called building an image. After that you can push this image into docker hub – similar to github but for images. There everyone can download it. After that an agent will deploy this image, which will have all the dependencies, and start a build there.
   • What image to use, what are the initial commands and what are the commands for builds - all of that is mentioned in .gitlab-ci.yml.
5. The build is created in docker as a result of the CI work. It is called an artifact.
6. After that a script will be executed, which will send a build to Firebase App Distribution – a system allowing testers to download apk from the installed app (otherwise they would have to download it from the gitlab website). The script will run in a separate image, but we will talk about it later.

First of all, let's have a look at the main terms.

As it was stated before, when CI has to build something, it starts the pipeline. The pipeline is its top-level component which consists of jobs and stages. We have already discussed jobs. Stages are the phases when jobs are performed. For instance, there can be 3 stages: build, test, deploy. During the first stage (Build) several jobs will create builds, several jobs will run tests during the Test stage and several jobs will upload the images during the Deploy stage.

It is also worth mentioning that in this example we do not want to start CI/CD on every push, but only on pushes in the branches, which are bound to merge requests or when the merge request is created itself. This also applies to all pushes (and merges) in master and release branches.

We will have a look at the example of the .gitlab-ci.yml file (reminder: it needs to be put in the root). In this file there will be builds – debug and release, and 2 flavors – dev for debug and prod for release. If you have only one flavor just remove the parts about the flavor (dev, prod) from the commands testDevDebug, assembleDevDebug, testProdRelease, assembleProdRelease. Release build will be signed. After the build the deployment to Firebase App Distribution will happen.

workflow: # Rules of the pipeline running are described here: 
automatic master and release/* branches building (e.x. release/2.1.0) merge requests and pushings existing merge requests. 
  rules:
    - if: '$CI_COMMIT_BRANCH =~ /release*/ || $CI_COMMIT_BRANCH == "master"' # Rule: if branch's name, where the commit occured, is master or release*, then always run pipeline. The next conditions are not important.

      when: always
    - if: '$CI_PIPELINE_SOURCE == "push"' # Rule: if we pushed, then it won't run.
      when: never
    - when: always # If previous conditions are false, then any other trigger will run assembly. E.x. if we made merge request - pipeline will be run.

stages: # Let's define two stages - build and deploy.
  - build
  - deploy

assembleDevDebug: # Define job for debug assembly with flavor. 
  image: lenant255/android:latest # Let's use image from my docker hub account -- in this image android-sdk for 29 version and nessecery required tools are installed. 
  stage: build # This job will run on build stage
  script: 
    - ./gradlew testDevDebug assembleDevDebug # Testing and assembly command. Type the command you need and insert correct flavor.
  artifacts:
    paths:
      - app/build/outputs/ # The path inside the docker container, where apk-files will be.

assembleProdRelease:  # Let's describe job for release assembly with production flavor - another one, but with additional steps for signature
  image: lenant255/android:latest
  stage: build
  script:
    - echo $KEYSTORE_FILE | base64 -d > my.keystore
    - ./gradlew testProdRelease assembleProdRelease # Let's pass parameters in gradlew for signature. They are defined in the gitlab ci interface. We will talk about them later.
      -Pandroid.injected.signing.store.file=$(pwd)/my.keystore
      -Pandroid.injected.signing.store.password=$KEYSTORE_PASSWORD
      -Pandroid.injected.signing.key.alias=$KEY_ALIAS
      -Pandroid.injected.signing.key.password=$KEY_PASSWORD
  artifacts:
    paths:
      - app/build/outputs/

deployDev: # The job for deploying of the dev assembly.
  image: node:latest # We need npm for installing firebase-tools, so we use image with node. 
  stage: deploy
  needs: [assembleDevDebug] # Let's define that we are waiting for assembleDevDebug job from the last stage. It means we haven't been waiting for asembleProdRelease job completing. Besides we have assembleDevDebug artifacts in this job.
  script:
    - npm install -g firebase-tools # Install firebase-tools
    - apkfile=$(find . -name "*.apk") # Let's find the apk file and write it to the variable apkfile
    - firebase appdistribution:distribute $apkfile --app $FIREBASE_APP_DEV_ID # we will execute a command that will send our assembly to firebase
      --release-notes "$CI_COMMIT_MESSAGE" # As release-notes we will use the message from commit, where task's key should be written.
      --groups "your-app-testers" # Specify the group of testers who will receive the assembly. About group settings below
      --token "$FIREBASE_TOKEN"

deployProd: # obj for deploy prod assembly, Everything is like in the previous job, but another application id and another job in needs
  image: node:latest
  stage: deploy
  needs: [assembleProdRelease]
  script:
    - npm install -g firebase-tools
    - apkfile=$(find . -name "*.apk")
    - firebase appdistribution:distribute $apkfile --app $FIREBASE_APP_PROD_ID
      --release-notes "$CI_COMMIT_MESSAGE" 
      --groups "your-app-testers"
      --token "$FIREBASE_TOKEN"

The notes in the file explain what's what.

Let's see how to set up environment variables KEYSTORE_PASSWORD, KEY_ALIAS, KEY_PASSWORD, KEYSTORE_FILE, FIREBASE_APP_DEV_ID, FIREBASE_APP_PROD_ID and FIREBASE_TOKEN.

Values of variables KEYSTORE_PASSWORD, KEY_ALIAS, KEY_PASSWORD are clear from the names: the password for keystore, alias of the key and the password for the key.

You can get the value for the variable KEYSTORE_FILE by doing the following:

base64 -i your_jks_file

After performing the command the required value will be shown on the terminal.

In your repository go to Settings CI/CD Variables. Enter the required variables.

Is it safe to keep the variables there? No, it is better to do signing on a separate server, where all the needed secrets will be located. But this place works as an example.

You can get FIREBASE_TOKEN using a method described at the link.

To get FIREBASE_APP_DEV_ID and FIREBASE_APP_PROD_ID go to project settings in Firebase and check the 'App id' field for your apps:

Next, at App Distribution page press 'start'.

It is necessary to add testers and create a group for them. You have to do this at the "Testers and groups" page.

Add the variables to the Gitlab interface similarly to the variables for signing the prod build.

Well done! Now push in the separate branch .gitlab-ci.yml and do a merge request. The build will start. You can see it by the newly appeared 'pipeline' field in the merge request.

Going into the pipeline you can see the jobs being processed.

After the successful build all users added to the 'testers' group will receive an email with a link to download the App Tester application, which will have the builds.

Screenshot of App Tester:

In order to simplify the identification of builds you can give them a unique version Code from CI. You can also take the unique ones to the pipeline, so that all builds from this pipeline have the same number (prod build, test build, etc.) To do so add to build.gradle:

def versionCodeFromCi = System.getenv('CI_PIPELINE_IID') as Integer ?: 999

android {
   defaultConfig {
       versionCode versionCodeFromCi
       ...
   }
}

Great! Now you can create builds, run tests on them and deliver them through App Tester. Before drawing the conclusion we will talk about what you should do if you need other versions of android-sdk and build-tools for a build.

First of all, let's understand why we need to create it. My image, which is used in above, uses android-sdk and build-tools version 29. You may need another version. Some of them you can find by googling. But sometimes that may not be enough.

To create your own image, install Docker on your system and sign up in Docker hub. After that, create a file named "Dockerfile" anywhere (but not in the repository, you do not need it there).

FROM openjdk:8-jdk

ENV ANDROID_COMPILE_SDK="29" \
  ANDROID_BUILD_TOOLS="29.0.3" \
  ANDROID_SDK_TOOLS="6858069" \
  ANDROID_SDK_ROOT="/android-sdk"

RUN apt-get --quiet update --yes
RUN apt-get --quiet install --yes wget tar unzip lib32stdc++6 lib32z1 file
RUN wget --quiet --output-document=command-line-tools.zip https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_SDK_TOOLS}_latest.zip
RUN unzip -d command-line-tools command-line-tools.zip
RUN mkdir ${ANDROID_SDK_ROOT}
RUN mkdir ${ANDROID_SDK_ROOT}/cmdline-tools
RUN mv command-line-tools/cmdline-tools/ ${ANDROID_SDK_ROOT}/cmdline-tools/tools
RUN echo y | ${ANDROID_SDK_ROOT}/cmdline-tools/tools/bin/sdkmanager "platforms;android-${ANDROID_COMPILE_SDK}" >/dev/null
RUN echo y | ${ANDROID_SDK_ROOT}/cmdline-tools/tools/bin/sdkmanager "platform-tools" >/dev/null
RUN echo y | ${ANDROID_SDK_ROOT}/cmdline-tools/tools/bin/sdkmanager "build-tools;${ANDROID_BUILD_TOOLS}" >/dev/null
RUN export PATH=$PATH:$ANDROID_SDK_ROOT/cmdline-tools/latest/bin:$ANDROID_SDK_ROOT/cmdline-tools/tools/bin
RUN yes | ${ANDROID_SDK_ROOT}/cmdline-tools/tools/bin/sdkmanager --licenses

Here the linux distributive with the jdk-8 installed is used as a base image. After that the environment variables are set up and the required commands are performed. Keep in mind that for older versions you should have downloaded android build tools instead of command line tools, so you may have to change the links and the number (but it would be simpler and better to update the project so that it will be built with the version 29).

After creating Dockerfile, execute the following in the folder where the file is:

docker build -t name_of_your_image

Your image will be built. After that you have to specify a tag. Look up IMAGE ID of your image – you can see it by performing the following command:

docker image ls

After that perform the command:

docker tag yourimageid yourhubusername/yourimage:tag

'yourimageid' is an id of your image from the previous step, 'yourhubusername' is your name in docker hub, 'yourimage:tag' is a name and a tag of your image, by which everyone can find it on docker hub.

After that perform:

docker push yourhubusername/yourimage

Now you can use your image in .gitlab-ci.yml.

The use of CI/CD makes the development process better in many ways: you can always be sure that your builds are automatically tested and delivered to the QA department. Developers do not have to think about sending APK files manually to QA or managers or about starting the autotests – everything is done automatically.