Katalon Studio is one of the recognized tools for automated testing. Along with Selenium, Ranorex, TestComplete, and Apache JMeter, it’s worth discussing as more and more testing engineers opt for Katalon. The platform facilitates running automated tests for Web interfaces, APIs, Windows desktop and mobile apps (both iOS and Android), provides test recording, and allows for creating analysis reports.
In this article, we’ll look at the pros and cons of Katalon Studio. If you’re looking for a broad overview of the main test automation providers, have a look at our dedicated post.
What is Katalon Studio?
Katalon Studio is a robust automation tool initially released in January 2015 with a Selenium-based engine. Mostly, Katalon is designed to create and reuse automated test scripts for UI without coding. Katalon Studio allows running automated tests of UI elements, including pop-ups, iFrames, and wait-time. The tool can be launched on Microsoft Windows, macOS, and Linux.
Originally released as a free solution, in October 2019 a more feature-rich Katalon Studio Enterprise version as well as Katalon Runtime Engine were introduced to provide flexible options for different needs. However, the basic Katalon Studio intended for individual users still remains free of charge.
Katalon Studio key features
Katalon’s main advantage is its ease of deployment and a wider set of integrations compared to Selenium, the market leader. Katalon has dual scripting interfaces for users with different programming skills. This means that testers with limited technical knowledge can use a simpler user interface that doesn’t require writing code.
The mode for more proficient users has access to scripting with syntax highlighting, code suggestion, and debugging. Katalon Studio has a pre-defined artifact structure: a number of templates for test cases, test suites, test objects, and reports. Katalon Studio supports local and remote testing, as well as parallel and sequential executions. It runs on Groovy (Java) scripting language.
This solution has a quick setup and a number of pre-installed templates that allow repeating some testing patterns. Katalon Studio is a cross-browser tool that supports Web, mobile, Windows desktop, and API testing. These solutions go with analytics and recording modules. Let’s look closer at each Katalon product.
API testing
Unlike Selenium and Ranorex, with Katalon Studio you can test APIs without additional integrations. This inbuilt module allows testers to conduct end-to-end API testing, automate scripting, and maintain their tests. Main features of this module are code auto-completion, code inspection, code snippeting, references, and debugger. The software supports all types of REST, SOAP/1.1 and SOAP/1.2 requests and multiple data sources (XLS, XML, databases with dynamic mapping to maximize test coverage).
The API testing mode allows users to import tests from such API testing and editing tools as Swagger, Postman, and WSDL. Katalon API testing also has an inbuilt response viewer with auto-formatting and search for access to artifacts.
Web testing
Katalon offers a complete web testing solution with inbuilt Continuous Delivery/Continuous Integration and DevOps integrations. The tool is Selenium compliant and has code-assist utilities like built-in object spy, code refactoring, and in-context references. It also offers a smart XPath location mechanism and a range of reporting options. Besides, native plugins easily integrate with Jenkins, Git, JIRA, and qTest.
Katalon’s web-testing execution mechanism uses multiple configurations and datasets. It also allows for customizing execution workflow and running it automatically. The module features importing external libraries to improve the automation functions.
Mobile testing
This module allows users to test mobile Web, native iOS and Android apps, and hybrid apps. With native integration by means of Appium, a test automation tool for mobile applications, the module supports testing for the latest mobile platforms and devices without additional installations. Katalon speeds up mobile testing by detecting and storing the objects. With cross-environment execution capabilities, tests can be executed locally and remotely with real devices, simulators, or cloud-based services.
Desktop testing
Starting from Katalon Studio 7, Katalon supports automation testing (both locally and remotely) for desktop apps written on
- Universal Windows Platform (UWP),
- Windows Forms (WinForms),
- Windows Presentation Foundation (WPF), and
- Classic Windows (Win32).
Choosing a testing environment
Source: Katalon Studio Docs
The module has a rich set of built-in and custom Windows keywords to support the testing process. The main features are Spy Windows Objects (to locate, capture, and analyze all app objects) and Record Windows Action (to record test actions, inspect all running app objects, and present them in a tree view).
Katalon Runtime Engine
Katalon Runtime Engine (KRE) is the Katalon Studio add-on that allows users to schedule and execute automation tests in console mode and CLI (command-line interface) mode. Various scenarios exist for KRE implementation. It can be used for scheduling your tests, integrating with CI/CD system, or bundling your tests to execute in virtual containers like Docker.
KRE can be activated on a paid basis with either a node-locked or floating license model. For more details, see the pricing page.
Katalon TestOps (formerly Katalon Analytics)
TestOps is a web-based app designed to orchestrate tests and DevOps. Its key modules are test management, test planning, and test execution. It’s seamlessly integrated with other Katalon products, testing frameworks, and CI/CD tools.
Katalon TestOps has powerful reporting and analytics functionality. The tool provides insights in the testing activities, creating real-time test reports and allowing users to monitor test quality, coverage, flakiness, etc. Testing areas can be prioritized, highlighting the most crucial parts. All execution patterns are recorded on screenshots and videos, and test results are displayed as specific KPIs on dashboards. Customizable alerts can also be set up for full control over the testing process.
Katalon Recorder
Katalon is shipped with a Selenium-based recording module for Chrome, Edge, and Firefox. It records tests, allows testers to perform debugging, automate test management, and export testing scripts into C#, Java, Ruby, Python, Groovy, and Robot Framework. Katalon Recorder has a record and playback feature for testers without programming skills and saves the methods previously used in testing.
Katalium
Katalium is an open-source framework designed for users who still need to work with TestNG and Selenium. Basically, it’s a lightweight platform that provides a blueprint for test automation projects based on Selenium and TestNG.
A component of the Katalium framework, Katalium Server is a set of enhancements to improve the user experience with Selenium Grid. For more details, check the Katalium documentation and the related GitHub page.
Katalon Store
Katalon Store provides testers and developers with a variety of plugins grouped in 3 categories, i.e., Utilities, Integrations, and Custom Keywords. It’s an open-source platform where users can install, share, and review add-ons that extend Katalon Studio functionality and increase customization.
Having discussed all the Katalon Studio features and products, it’s time to look into its strong and weak sides.
Pros of using Katalon Studio
Even though Katalon Studio is a relatively new tool, it has already gained sizable popularity (Katalon claims that it’s used by 65K+ companies in 160+ countries.) and was even named 2020 Gartner Peer Insights Customers’ Choice -- and for good reason. Let’s see what its main advantages are.
+ Integrated with CI/DevOps workflow and other tools
Katalon doesn’t require additional extensions to run tests as it offers a wide range of handy integrations. For example, connecting to qTest, a centralized test management tool that facilitates the communication of a QA team, allows for controlling test requirements, test case planning, and defect management. Users can also easily integrate with other SDLC (software development lifecycle) management platforms such as JIRA, TestRail, and TestLink.
Connection between Katalon products and integrated tools
Source: katalon.com
Katalon is integrated with a number of tools used to support CI/CD and DevOps such as Jenkins, Bamboo, TeamCity, CircleCI, and Travis CI. Introduced in May 2021, Katalon Studio 8 includes the native integration with Azure DevOps Services, and namely, Azure Test Plans. Now, users can map Test Cases in Azure DevOps to automated Test Cases in Katalon Studio as well as automatically send Test Execution logs and reports from Katalon Studio to Test Run in Azure DevOps.
Katalon Studio also supports integrations with collaboration platforms (Git, Microsoft Teams, and Slack) and execution platforms (SauceLabs, BrowserStack, Selenium Grid, and Kobiton).
Katalon Studio integrations
+ Intuitive analytics dashboard and reports
An inbuilt, data-driven testing module allows users to analyze tests based on recorded scenarios. Katalon displays the analytic results in the form of built-in reports that can be exported in PDF, HTML, Excel, or CSV. The reports are visually intuitive and simple to work with.
Test Log Tab
Source: Katalon Studio Docs
Katalon Studio Enterprise offers more reporting options, i.e., test suite collection reports and report history, videos for headless browsers, etc.
Even more reporting and analytical capabilities are available through the TestOps tool integration.
Test runs in Katalon TestOps
Source: Katalon Studio Docs
+ Supports several types of testing
In Katalon Studio you can run the following test types:
Keywords-driven. This approach works for testers who aren’t proficient in scripting. Based on keywords that represent users’ actions on the AUT (Applications Under Test), a tester can run an automated test. Keywords can be customized.
Data-driven. A test execution strategy that features reading test scripts from various data-sources. The data is taken from data pools, Excel files, ADO objects, CSV files, and ODBC sources.
API-testing. It supports XLS, XML data sources, and databases with dynamic mapping for better test coverage. Also, the tests are compliant with Cucumber, a BDD-based (behavior driven development) framework for acceptance tests of web applications.
Cross-browser. Katalon supports all popular browsers and allows running automated tests in Chrome, Firefox, Internet Explorer, Safari, Edge, Remote, Headless, and custom environments.
+ User-friendly GUI
Katalon’s UI offers rich graphics with tree views, tables, and menus which allow a user to manage test artifacts easily. Testers find the tool’s interface convenient, user-friendly, clean, and easy to grasp even for non-technical users.
Katalon Studio interface
Source: Katalon Studio docs
+ Mild learning curve + learning materials
Katalon Studio is famous for its simplicity, both of installation and usage. It’s easy to set up thanks to its many integrated packages. It’s good both for beginners and advanced users, who can use the whole IDE and write advanced scripts. Developers have access to a number of training videos and documentation for users: Katalon has GitHub repositories, a list of online courses on Udemy, and video courses on YouTube.
Katalon Studio short guide
Source: Katalon Studio docs
+ Free version available
Katalon has a fully functional free version, which is considered an advantage by most of its users. However, the Enterprise version offers more features and access to all Katalon plugins and offline license, so it’s recommended for teams and scalable projects, while the free basic edition is offered for individual use.
Cons of using Katalon Studio
Every product has its good and bad sides. Let's take a look at the drawbacks of Katalon Studio.
- Lack of scripting languages
Unlike Selenium and TestComplete, the only scripting language supported by Katalon is Groovy. This scripting language is in the Java family so anyone who knows Java can use it. But, users would like to see more languages supported.
- Small community
Since Katalon was developed in 2015, the community is smaller than that of more mature competitors, and you may experience a lack of support from peer testers. So, just for comparison sake, as of the day of this update, there are 706 questions tagged Katalon Studio on Stack Overflow, while for Selenium there are 87,555 questions. However, according to Capterra, Gartner, and Stack Overflow, Katalon is way more popular than Ranorex and other test automation tools.
There’s a dedicated Katalon forum and a growing community on GitHub, so as the product matures and acquires more admirers, it becomes easier to find answers to arising questions.
- Not open source
The tool has a closed source code, resulting in a smaller number of developers in the community. Selenium, one of the main competitors to Katalon, is an open-source tool that allows engineers to customize it or use community-built packages.
However, Katalon Studio has some open components, an open-source Katalium framework, and an open-source Katalon Store with plugins that we’ve described above.
- Performance issues
Users report some bugs that hamper testing, slowing it down. For example, sometimes the tool freezes or may start lagging, the text and object verification within iframes is problematic. Mobile testing takes more time due to the necessity to capture and write code.
Who uses Katalon Studio?
Katalon Studio is used by companies from different industries.
Care Logistics, a company that provides healthcare logistics software and services to hospitals and health centers, faced the challenge of a quick verification of system stability. To verify the system’s functionality, the testing team had to create over 8,000 manual test cases and execute all of them before deployment. With Katalon Studio, the team managed to convert and automate these tests. As a result, the team could verify more issues and had 30 percent of products automated. Also, they achieved a 50 percent reduction in regression testing lifecycle.
Tunaiku, an Indonesian fintech lending venture, had to make sure that user data and transaction information was safe. This required a number of quality assurance tests in CI workflow, which involves Jenkins and Docker integration. Also, they needed to generate automation test scripts effectively. Tunaiku solved these quality assurance team problems by creating automation test scripts and customized code infrastructure in Katalon. This helped them reduce scripting time by 60 percent.
Angler, an Internet and technologies provider from India, used Katalon Studio for API testing. Their task was to test APIs and generate API scripts, as well as detect bugs in the development phase and prevent them from getting into APIs. As a result, Angler covered over 80 percent of functional testing with Katalon Studio.
Katalon vs Selenium
Comparison of Selenium and Katalon
Selenium is the most popular free open-source automation testing tool and that pretty much sums it all. Katalon is younger, has a smaller community, and only supports the Groovy language, while Selenium is a mature platform with an extensive fanbase and support for all of the most popular programming languages.
However, Katalon has its unbeatable advantages, among which are the mild learning curve, the ability to automate non web-based apps, and the versatility of reports -- which Selenium lacks. Katalon, in contrast to Selenium, has readily usable features and doesn’t require any programming knowledge, making it a great option for beginners or non-technical users.
Both products have their strengths and weaknesses, but in our opinion, both are very decent solutions to support automation testing activities. Both tools can be used to test apps across multiple platforms and environments and both offer wide integration options with CI/CD, DevOps, and other software development instruments.
For a more detailed comparison, read our separate article about these and other automated testing tools.
Getting started with Katalon Studio
If, after reading this article, you consider using Katalon Studio as your testing tool or just want to know more about this product, here are some resources that will come in handy.
This is the main Katalon website with its key offerings and download links: Katalon Studio, Katalon TestOps, and Katalon Recorder.
Check the documentation for all Katalon products on this page.
Review the license options on the pricing tab.
Here’s where you can find the docs for Katalium framework.
Explore the plugins in the Katalon store.
Communicate with the fellow developers on the Katalon forum or the GitHub fork.
Final word
Katalon Studio is a good choice for both individual and enterprise use as it provides a rich ecosystem of tools to choose from. It’s a constantly evolving solution with many integrations that allow you to cover a variety of testing types with a single tool. It comes with all the necessary installations out of the box to run various types of tests, including API testing. It’s easy to deploy, master, and use. Katalon can be used by specialists with different QA engineering roles and varied programming skills, making it an attractive solution for teams with testers of different levels.