Using GitHub Copilot for Software Testing

As software developers, we are always on the lookout for tools and technologies that can streamline our workflows and improve productivity. One such promising tool that has garnered significant attention recently is Copilot.

Developed by OpenAI and GiHtub, Copilot is an AI-powered code completion tool that promises to assist developers in writing code efficiently. In this article, I will share my firsthand experience of using Copilot to create a software test and provide an honest review of its capabilities.
Before diving into my experience with Copilot, let's briefly understand what it is and how it works. Copilot is an AI-powered code completion tool that is built on OpenAI's GPT-3.5 architecture. It works as a plugin for popular Integrated Development Environments (IDEs) like Visual Studio Code and helps developers by generating code suggestions as they type.

Copilot relies on a vast corpus of code from publicly available sources and is trained to predict the next lines of code based on the context and the code written so far. It aims to assist developers by automating repetitive tasks, reducing boilerplate code, and providing useful suggestions for coding challenges.

Why Efficient Software Testing Matters

As software systems grow in complexity, the importance of efficient and comprehensive testing becomes paramount. Developers face the challenge of ensuring thorough test coverage while optimizing their workflows to meet project deadlines. Copilot offers a promising solution to this challenge by providing relevant code suggestions. By using Copilot, one can focus more on higher-order testing activities and strategic decision-making, thereby enhancing the overall testing process.

The adoption of Agile and DevOps practices has further amplified the importance of efficient software testing. Frequent releases and shorter development cycles demand rapid and effective testing to maintain the application's quality. Test automation, continuous testing, and test-driven development (TDD) have become integral parts of Agile and DevOps workflows, allowing organizations to deliver value to users more frequently and consistently.

Copilot's Role in Shaping Quality Testing

Copilot, built on the GPT-3.5 architecture, is a powerful AI language model that has been trained on an extensive dataset of publicly available code. It integrates seamlessly into popular Integrated Development Environments (IDEs) like Visual Studio Code, assisting developers and testers by suggesting code snippets as they write. 

Copilot's capabilities extend to various programming languages, making it a versatile tool for a wide range of projects. Here are 8 more things you can do with Copilot. Thank me later! 

Setting Up Copilot: A Quick Overview

To harness the power of Copilot for software testing, you can easily set it up as a Visual Studio Code extension. Once installed, Copilot becomes an invaluable asset within your coding environment, offering real-time code completion suggestions tailored to your current context.
As one of the most popular IDEs in the developer community, VS Code provides excellent support for extensions, making it an ideal choice for integrating Copilot.

• Install VS Code: If you haven't already, download and install Visual Studio Code from the official website.

• Open Extensions View: Launch VS Code and go to the Extensions view by clicking on the Extensions icon in the Activity Bar on the side of the window.

• Search for Copilot: In the Extensions view, search for "Copilot" using the search bar on the top. Once found, click the "Install" button to add the Copilot extension to your IDE.

• Sign in with GitHub: After installing the Copilot extension, you will be prompted to sign in with your GitHub account. This step is necessary as Copilot is a GitHub Copilot and requires access to your GitHub repositories for code suggestions.

• API Access: During the setup process, you may be required to get access to the Copilot API. Follow the instructions provided during the setup process to complete the API access.

Once you have successfully completed these steps, Copilot will be integrated into your VS Code IDE, and you can start using it to generate code suggestions as you type.

Creating a Software Test With Copilot

Let's take a practical approach to understanding Copilot's impact on software testing. Consider a scenario where we need to write test cases for a web application registration form. We start by creating a new test file in our project and defining the initial structure of the test scenario, including test steps and expected outcomes.

// Test Scenario: User Registration Form

describe('User Registration', () => {
  it('should allow users to register with valid credentials', () => {
    // Test steps to be completed using Copilot's suggestions
  });

  it('should display an error message for invalid email addresses', () => {
    // Test steps to be completed using Copilot's suggestions
  });

  it('should require a password with a minimum length of 8 characters', () => {
    // Test steps to be completed using Copilot's suggestions
  });
});

As we begin writing the test scenarios for our user registration form, Copilot proves to be an indispensable assistant. As we type the test steps, Copilot instantly starts suggesting code snippets that match the context. Let's take a look at how Copilot aids us in completing some of the test steps.

it('should allow users to register with valid credentials', () => {
  // Copilot suggestion for form input and submission
  cy.visit('/register');
  cy.get('#username').type('testuser');
  cy.get('#email').type('testuser@example.com');
  cy.get('#password').type('secretpassword');
  cy.get('#submit').click();
  cy.url().should('include', '/dashboard');
});

Copilot provided an efficient code snippet for interacting with the registration form elements and simulating a user registration with valid credentials. This significantly reduced the time and effort required to write the test steps.

it('should display an error message for invalid email addresses', () => {
  // Copilot suggestion for entering an invalid email address
  cy.visit('/register');
  cy.get('#email').type('invalidemail');
  cy.get('#submit').click();
  cy.get('.error-message').should('be.visible').and('have.text', 'Invalid email address');
});

In this example, Copilot efficiently suggested a test case for validating the display of an error message when an invalid email address is entered in the registration form.

In the provided code context, cy refers to the function typically used in end-to-end (E2E) testing using Cypress, a popular JavaScript testing framework. Cypress is designed to facilitate E2E testing for web applications and provides an elegant and easy-to-use API for interacting with elements on a webpage.

In Cypress, cy.get() is a command used to locate and select DOM elements on the page. It allows testers to interact with these elements by performing actions such as clicking, typing, or validating their properties and contents. The cy.get() command uses a CSS selector or a jQuery-like expression to identify the element(s) to be selected.

For example, consider the following code snippet using cy.get() in a Cypress test:

cy.get('#username').type('testuser');

In this case, cy.get('#username') selects the input field with the ID attribute set to "username" on the webpage. The .type('testuser') method then simulates typing the text "testuser" into the selected input field.

[ Read Related: Using GitHub Copilot for Pull Request ]

Copilot's Challenges and Opportunities: A Personal Take

While Copilot undoubtedly offers valuable assistance, it does have its limitations. Copilot's suggestions may not always be accurate for complex or unconventional test scenarios. As developers, it is highly essential to carefully review and validate the code generated by Copilot to ensure its correctness.

Copilot's lack of customizability can pose challenges when working with projects that have unique testing requirements. It is crucial to strike a balance between leveraging Copilot's capabilities and applying human expertise to craft high-quality and comprehensive test scenarios.

During the test execution phase, I put Copilot to the test and observed its performance in generating test scenarios for the web application registration form. Overall, Copilot proved to be a valuable asset, significantly speeding up the process of writing initial test cases.

The test scenarios that I created using Copilot's suggestions covered the essential functionalities of the registration form, including successful user registration, error handling for invalid email addresses, and password validation.

However, I did encounter some limitations with Copilot during the testing process. While it excelled in generating straightforward test cases, it occasionally struggled with more complex or unconventional scenarios. For instance, when I needed to simulate test data with specific boundary conditions, Copilot's suggestions were not always precise or relevant. This limitation necessitated manual intervention and additional effort to refine and customize the generated code.

Additionally, Copilot's lack of customizability presented challenges when dealing with unique project requirements and domain-specific test cases. It sometimes led to code snippets that were not entirely aligned with the specific testing objectives of the application under test.

Despite these limitations, Copilot's assistance in writing boilerplate code and handling repetitive test steps was commendable. The time saved in setting up test scenarios allowed me to focus more on higher-level testing activities, such as exploratory testing and edge case analysis.

Although Copilot proved to be a valuable tool in creating initial test scenarios for the web application registration form. Its real-time code suggestions were particularly helpful in streamlining the testing process and reducing manual effort. However, the tool's limitations in handling complex scenarios and lack of customizability require testers to exercise caution and verify the generated code to ensure accuracy and relevancy.

While Copilot enhanced my productivity, it should be viewed as a supportive tool rather than a complete replacement for human intelligence in the testing process. Manual review and validation of test scenarios remain essential to maintain the quality and reliability of the tests.

Curious about your GitHub project status? Read our guide on how to track GitHub project status to keep your projects organized and on track.

Copilot: Your Partner in Crafting Superior Tests

Copilot is a remarkable tool that holds great promise in revolutionizing software testing workflows. Its AI-powered code completion capabilities can significantly improve productivity for testers and developers. By leveraging Copilot's suggestions effectively and combining them with human expertise, testers can craft comprehensive and high-quality test scenarios.

While Copilot has its limitations, it represents a significant step forward in harnessing the power of AI in software testing. As the technology continues to evolve, Copilot and similar AI tools are poised to become invaluable assets in the software testing arsenal. By striking the right balance between automation and human ingenuity, testers can unlock the full potential of Copilot to optimize their testing efforts and deliver software products of the highest quality.

Follow Hatica Blog for more amazing insights on AI, effective software development strategies and tools that make devs more efficient, productive, and future-ready!