Automated Testing and Continuous Integration/Delivery: Lessons Learned - Marcel Ziswiler, Toradex AG

Automated Testing and Continuous Integration/Delivery: Lessons Learned - Marcel Ziswiler, Toradex AG

Three years ago we set out to build a modern automated testing and continuous integration/delivery (CI/CD) pipeline with the goal to deliver more and better-tested software. Given our open source focus our entire source code was already managed in GitLab which is a great advancement from regular Git adding features like CI/CD. We put GitLab CI in place to check and build any major BSP source code we touch. Another building block is our Jenkins infrastructure able to do complete nightly builds of our entire Embedded Linux stacks (Yocto Project based). Artifactory manages the build artefacts with full over-the-air update integration. To allow testing on real hardware we set up LAVA infrastructure which manages an entire board farm of currently 47 devices and allows automatic regression testing of complete Embedded Linux images on real hardware. For this purpose, we built entire shelves of up to seven carrier boards each holding various different SoMs. Various interfaces either connect to real-life devices like SD cards and USB memory sticks or are looped back for testing. This talk introduces all the components and concepts both on the hardware as well as software side, the orchestration thereof, and discusses various things we learned on our journey to successful CI/CD.