Skip to content

Test artifacts

Test artifacts are precompiled binaries necessary for CI and local testing. They can change over time so developers should be able to maintain and share them easy as possible.

Overview

Storage

Local

Local working copy of test artifacts can be find in test_artifacts directory inside local flank repository. However, this directory MUST be added to .gitignore to prevent committing binaries to git. We accept few restrictions about storing test artifacts in test_artifacts directory. * test_artifacts directory CAN contain directories with binaries required for testing. * Names of directories inside test_artifacts SHOULD reflect names of working branches. * test_artifacts directory CAN contain test artifacts archives bundles. * The name of artifact archive should match following format branchName-unixTimestamp.zip

Remote

The remote copy of test artifacts can be find at test_artifacts/releases We accept few restrictions about using github releases as storage for test artifacts. * The release name and tag should reflect the name of related working branch. * Each release should contain only one asset with artifacts. * The name of test artifact assets archive should be [ ~~mdp5 checksum~~ | unix timestamp ] suffixed with zip extensions.

Linking artifacts

For convenient switching between test artifacts we are using symbolic links. Ensure you have symbolic link to correct directory inside test_artifacts directory, otherwise the unit tests will fail because of lack of binaries.

Test projects

All source code of test artifacts binaries can be find in test_projects directory.

Generating test artifacts

source .env
update_test_artifacts android ios  # [ android | go | ios | all ]

Working with artifacts

All testArtifacts subcommands can be configured using base options. * testArtifacts -b {name} used to specify branch name for test artifacts. For example if you want to run any subcommand on artifacts dedicated for your working branch feature123 run testArtifacts -b feature123 {subcommand}. By default, it is the name of current working git branch.

  • testArtifacts -p {path} used to specify the path to local flank repository. By default, the path is read from FLANK_ROOTenv variable. To export path to your local flank repository just source .env file.

Developer scenarios

As a developer I want to download test artifacts before test run.

  • Just run ./gradlew test command, this should trigger resolveArtifacts task which will update artifacts if needed.

As a developer I want to switch between local test artifacts

  • Run flankScripts testArtifacts link to create a symbolic link to test artifacts for the current branch.
  • Run flankScripts testArtifacts -b {name} link to create a symbolic link to test artifacts for the specific branch.

As a developer I want to edit existing test artifacts

  1. Edit required project in test_artifacts/releases directory.
  2. Ensure you have sourced .env file.
  3. Build required project and copy binaries using update_test_artifacts shell function or do it manually.
  4. Ensure you have linked a correct directory with artifacts.
  5. Your local tests now should you use updated artifacts.

As a developer I want to upload new test artifacts to remote repository.

  1. Make sure you have directory with artifacts in test_artifacts and the name same as working branch.
  2. Run flankScripts testArtifacts zip to create zip archive.
  3. Run flankScripts testArtifacts upload to upload zip as remote copy. The script will automatically resolve the latest archive and upload it.

As a developer I want to remove test artifacts

  • Run flankScripts testArtifacts remove_remote this will remove the remote copy of test artifacts for current working branch.

iOS test artifacts

Currently we have 4 different iOS test projects:

  1. EarlGreyExample
  2. FlankExample
  3. FlankGameLoopExample
  4. FlankTestPlansExample

Source code of each of them is located under: test_projects/ios

Test artifacts for each project contains: * build output in Debug-iphoneos * zipped build output in PROJECT_NAME.zip, * .xctestrun file for each test target

EarlGreyExample

This project is basically clone of EarlGrey. Source project contains two test targets: EarlGreyExampleSwiftTests, EarlGreyExampleTests.

Generate

Run: flankScripts assemble ios earl_grey.

Source Code

test_projects/ios/EarlGreyExample

FlankExample

Simple project with two test targets: FlankExampleTests, FlankExampleSecondTests.

Generate

Run: flankScripts assemble ios flank_example.

Source Code

test_projects/ios/EarlGreyExample

FlankGameLoopExample

Simple SpriteKit app to test gameloop mode. It doesn't contain any test target, so test artifacts contains only IPA file.

Generate

Run: flankScripts assemble ios game_loop.

Source Code

test_projects/ios/EarlGreyExample

⚠️ NOTE: Generating IPA requires Apple distribution certificate therefore for now it's not possible to generate it without correct Apple Developer Account. game_loop is excluded when building all iOS artifacts: 

update_test_artifacts ios

FlankTestPlansExample

iOS project with XCTestPlans. Contains AllTests test plan. Generated .xctestrun is using V2 format. More details about test plans: docs/feature/ios_test_plans.md

Generate

Run: flankScripts assemble ios test_plans.

Source Code

test_projects/ios/EarlGreyExample