Firebase Test Lab Plugin for Android

Introduction

Firebase is actually the most popular developer tool platform, wchich handles almost every aspect of the app. It also gives possibility to run Android Tests on physical or virtual devices hosted in a Google data center through Firebase Test Lab. In order to fully exploit the potential of this tool I've created plugin to simplify process of creating tests configurations. It allows to run tests locally as well as on you CI server.

Contributors

• Jacek Marchwicki

Available features

• Automatic installation of gcloud command line tool
• Creating tasks for testable buildType[By default it is debug. If you want to change it use testBuildType "buildTypeName"]
• Creating tasks for every defined device and configuration separetly [ including Instrumented / Robo tests ]
• Creating tasks which runs all configurations at once
• Ability to download tests results to specific location
• Ability to clear directory inside bucket before test run

Benefits

• Readability
• Simplicity
• Remote and Local Testing
• Compatible with Gradle 3.0

Setup

1. If you don't have a Firebase project for your app, go to the Firebase console and click Create New Project to create one now. You will need ownership or edit permissions in your project.

2. Create a service account related with your firebase project with an Editor role in the Google Cloud Platform console - IAM/Service Accounts

3. Copy Project ID from Google Cloud Platform console - HOME

4. Add plugin to your root project build.gradle:

   buildscript {
       repositories {
           maven {
              url "https://plugins.gradle.org/m2/"
           }
       }
       dependencies {
           classpath "gradle.plugin.firebase.test.lab:plugin:1.1.2"
       }
   }
   

5. Add configuration in your project build.gradle:

   apply plugin: 'firebase.test.lab'
   
   firebaseTestLab {
       keyFile = file("test-lab-key.json")
       googleProjectId = "your-project-app-id"
       devices {
           nexusEmulator {
               deviceIds = ["hammerhead"]
               androidApiLevels = [23]
           }
       }
   }

   List of available devices

6. Run your instrumentations tests

   ./gradlew firebaseTestLabExecuteDebugInstrumentation

   Or run robo tests

   ./gradlew firebaseTestLabExecuteDebugRobo 

Advanced configuration

You can

// Setup firebase test lab plugin
firebaseTestLab {
    // REQUIRED obtain service key as described inside README
    keyFile = file("test-lab-key.json")
    // REQUIRED setup google project id ad described inside README
    googleProjectId = "your-project-app-id"

    // If you want you can ignore test failures
    // ignoreFailures = true

    // If you prefer you can use your custom google storage bucket for storing build sources and results
    // cloudBucketName = "your-custome-google-storage-bucket-name"
    // cloudDirectoryName = "your-custome-directory-name"

    // If you prefer to install gcloud tool manually you can set path by
    // cloudSdkPath = "/user/cloud-sdk/bin"

    // If you want to change default gcloud installation path (default is in build/gcloud directory)
    // you can set environment variable `export CLOUDSDK_INSTALL_DIR=`/cache/your_directory/`

    // REQUIRED
    devices {
        // REQUIRED add at least one device
        nexusEmulator {
            // REQUIRED Choose at least one device id
            // you can list all available via `gcloud firebase test android models list` or look on https://firebase.google.com/docs/test-lab/images/gcloud-device-list.png
            deviceIds = ["hammerhead"]

            // REQUIRED Choose at least one API level
            // you can list all available via `gcloud firebase test android models list` for your device model
            androidApiLevels = [23]

            // You can test app in landscape and portrait
            // screenOrientations = [com.appunite.firebasetestlabplugin.model.ScreenOrientation.PORTRAIT, com.appunite.firebasetestlabplugin.model.ScreenOrientation.LANDSCAPE]

            // Choose language (default is `en`)
            // you can list all available via `gcloud firebase test android locales list`
            // locales = ["en"]

            // If you are using ABI splits you can filter selected abi
            // filterAbiSplits = true
            // abiSplits = ["armeabi-v7a", "arm64-v8a", "x86", "x86_64"]

            // If you are using ABI splits you can remove testing universal APK
            // testUniversalApk = false

            // You can set timeout (in seconds) for test
            // timeout = 6000

            // Enable Android Test Orchestrator more info at: https://developer.android.com/training/testing/junit-runner
            // isUseOrchestrator = true // default false

            // A list of one or more test target filters to apply (default: run all test targets)
            // testTargets = ["size large"]

            // Environment variables are mirrored as extra options to the am instrument -e KEY1 VALUE1
            // environmentVariables = ["clearPackageData=true", "coverage=true"]

            // The fully-qualified Java class name of the instrumentation test runner
            // testRunnerClass = "com.my.package.MyRunner"

            // Pass any custom param for gcloud
            // customParamsForGCloudTool = --no-performance-metrics
        }
        // You can define more devices
        someOtherDevices {
            deviceIds = ["shamu", "flounder"]
            androidApiLevels = [21]
        }
    }
}

For more precise test selection run

./gradlew tasks 

to discover all available test options