Ihave traveled from Japan to Bangalore to attend Appium Conference 2019 for more than 8000 km for 24 hours on June 13th. Now that I’m back in Yokohama, I wanted to share what I learned with all of you. On June 14th and 15th, Appium Conference was hosted at Bangalore, India. The conference covered with inspiring and practical talks featuring the Appium team and the world’s top mobile testing and automation experts. It was a great two days filled with lively talks, India food and networking.
I’m not a newbie to Appium, so this recap will focus mostly on the new things in Appium talks, in addition to how to build an infrastructure for Appium. If that interests you, keep on reading.
[Day 1 (June 14th 2019)]
[KEYNOTE] Appium: The Next Generation by Jonathan Lipps
The keynote was presented by Jonathan Lipps, a project lead for @Appium, a former Director of Open Source at @Saucelabs and highlighted the realities and challenges that Automation Engineer face in a century where technology products change day by day. It was a great talk and blow my mind by show me how Appium is not only a mobile test tool, but also useful for other technical stuffs.
Recognition for the people who contribute code, documentation or just submit a bug, help someone else for using Appium.
The stakes are getting higher so the job will be getting harder and more complex (TVOS, IoT stuffs, Chatbots, etc…)
Beyond Functional Testing
- Visual Testing: make sure your app looks good and hasn’t some kind of visual regression and bug (one of my interesting talk, so talk about them in just a few moments)
- Performance Testing
- UX Testing
- Audio/Video Testing
- Visual Testing: My app’s visual appearance has changed?
- An intrusive or absent element
- Style/Design bugs
- Form factor/Layout bugs: running on a different kind of devices and device is totally broken
- Internationalization bug (I18n): different length of words, phrases direct to pretty nasty visual bugs
- Performance Testing: Is application responsive enough?
When users are using application and if app has some delay of even fraction of a second in some scenario. It may turn users off of application.
- Code blocking UI thread: App become no-responsive
- Slow network and poor network stack design：Type of problems that they see is network connection object not being reused, and all the network requests are continually recreated and you have overhead of new TCP connections. Use Network API for iOS, Android could fix this.
- Resource (CPU/Memories) overuse
- Old hardware or ignored locales: Suppose that your app under test is focusing on China market, but your service was banned in some case. Without verifying in focus locales will be hard to figure out bugs.
- Audio/Video Testing: Is my app’s media consumable? How would listener/viewer rate it?
- Dropped frames
- Audio/Video failing to play
- Compression artifacts
- Streaming service issues
Your job is getting complex? Are your tool is keeping up?
- Automation Tools
- Audio/Video Capture Tools
- CI/Build Tools
- API Testing Tools
- Image Comparison Tools
- Performance Monitoring
AI/ML for Testing
- Image analysis: Compare visual differences across every platform by screenshot daily is a fatigue. So, if Image analysis tools help me out with AI/ML, take a cup of tea and waiting for reports.
- UX Data analysis: Data bottle-neck & back-end server is misconfigured => help us figure out problem through logs
- Test metadata analysis
- Test authoring tool: Create your test without writing code
- Test discovery / generation tool: No need any code, help bots mimic exploratory testing, collect screenshot, CPU, metadata
I think it is worth to following this trend to solve different type of problems in the Era of new technology
Where does Appium fit?
Appium is a common interface for platform automation? Firstly, Appium let you remote control a mobile device, but beyond mobile devices it as a common interface for any type of platform (smart TV, IoT, etc). And Appium couldn’t support capabilities for any platform, so they need all of us contribute something like driver for other platform (we will dive in driver for Raspberry Pi or TV OS afterward, Samsung created driver for Tizen in next recap)
From Appium 2.0, We could write your own Appium driver starts with inheriting and extending Appium Base driver module. I ensure that Appium’s own ecosystem is growing more and more, base on Appium’s philosophy and development methodology are well-suited to adapting to whatever the testing ecosystem.
Appium won’t do everything, but Appium will keep up?
- Image recognition (find element by image, image comparison): Let’s take a deep-dive into how Appium communities have been working on to make Appium smarter.
- AI (test.AI classifier plugin): Appium team is working with Test.AI to bring some experimental find elements by AI. No-code, no record and playback, just pure reinforcement learning where the AI teaches itself to execute the test while we sleep. Cross-platform even. So, I can’t wait for that day come true.
- Batch command for faster cloud execution: group command together to send to cloud services, don’t deal with latency of command.
- Important platform updates and maintenances
Benefits of Appium’s Philosophy:
- Less cognitive overload: ONE API for everywhere
- Platform extensibility: new WebDriver-compatible drivers are trivial to build
- Feature extensibility: single-purpose plugin (Appium 2.0): using for single purpose
- Mix and match the drivers and plugins you need (Appium 2.0)
- Big umbrella, big community, open perspective
Fun Example: One of my favorite moment in this talk.
- Build up a customized Appium Driver in Raspberry PI
Open Source: https://github.com/jlipps/appium-raspi-driver
- Connect Raspberry Pi to Drum Machine
- Execute Appium code used in conjunction with the Appium Raspberry Pi Driver from Laptop. Drum Machine plays the song like a charm.Demo source: https://github.com/jlipps/appiumconf2019
Obviously for the real test, we need to capture audio and perform audio analytic to make sure that it actually played the right song. He said that he did recently write an article on how you can perform audio verification by using Audio Fingerprinting. If I have time, I will dive in this to share more my aware of Audio Fingerprinting in future, see you soon.
- WebDriver API is pretty flexible
- Appium can run in a lot of places, even on Raspberry PI or another stuffs like TV
- It was easy to create Raspberry PI GPIO header, (<250 LOC)
- Is IoT automation testing useful? Unclear, but it’s fun
- Get out of there and do creative a stuff with Appium
As the software industry grows everyday with a tidal wave of challenges and complexity. And to improve user experience, the new technologies, platforms will be launched. I predict that the power will shift towards the end-users, but there will be increasing pressure on QA and Automation testing companies to improve itself. Appium is not only making a powerful automation testing tool to companies reduce fatigue in Regression Testing for mobile application, but also baseline for developers do creative incredible things with helping from big community has open perspective.
What is next?
I will have recap of Visual Validation by AI/ML and Pixel by pixel libraries. It promises to be pretty interesting talks with some demo and open source. Stay tune.