In fast-paced projects, testing is often seen as a bottleneck. Skipping or rushing testing may seem like a time-saver, but the real cost comes later. Undetected issues can lead to major business disruptions - warehouse operations halting due to system failures, customer orders getting stuck, or critical data needing costly manual fixes. The effort saved in testing often multiplies as effort spent on incident resolution, emergency patches, and lost business trust. Proper testing isn’t just a phase; it’s risk prevention.

Raising awareness within project teams and leadership about the importance of structured testing with an emphasis on process validation, helps prevent last-minute surprises and ensures a smoother delivery. Behind the scenes, test management involves coordination across teams, environments, and evolving requirements.

Large-scale partner implementation test management

The program I’m using as an example is a large-scale partner implementation, involving technical and process changes, as well as over 1000 test cases across multiple markets (+20) and four sales channels. The department overseeing the project is relatively new, and this marks their first large testing initiative. This is an opportunity for growth and the introduction of new processes.

Managing over 1000 test cases across multiple testing phases simultaneously in different countries would have been nearly impossible using only Google Sheets or other existing ways of working. Drawing from my previous +20 years experience in testing, I created a proposal that leverages new processes and additional tools, such as Xray, for setting this project for success.

From the start, I made it clear to the program team, that setting up such new tools would be a lengthy and challenging process, as we need on one side support the learning curve for the team, as well as the definition and documentation of the hundreds of test case baselines needed for the each test cases we wanted to execute. Additionally, training 15 workstreams (+200 FTEs) on a new tool and driving new ways of working brought its own set of challenges.

Below, I am sharing the key aspects that my allocated testing team and stakeholders followed in the test management, from defining clear timelines and roles to fostering collaboration across teams. In this post, I break down how these strategies, along with tools like XRAY, helped streamline test case creation, improve reporting, and ensure better communication. These steps ultimately support the program by enabling smoother execution and higher-quality outcomes for the project.

Creating a test plan

Creating a comprehensive test plan is essential for ensuring all aspects of the testing process are thoroughly covered. The same needs to be agreed with the partner as well. This involves defining and aligning overall timelines for each testing phase, clearly outlining the objectives, and ensuring that resources are allocated effectively. A well-structured test plan also includes calculating the necessary capacity for each phase, taking into account team availability, skills, and required tools. By addressing these elements early on, teams can ensure the testing process runs smoothly, is properly resourced, and stays on track, minimizing and anticipating the risk of delays or resource shortages.

Resourcing

Effective test management requires having the right people in place at the right time, yet testing phases are often under-resourced. Teams may struggle with a lack of skilled testers, business experts for validation, or even sufficient time for proper execution. A well-planned testing phase addresses these challenges by clearly defining roles, creating a jointly aligned RACI matrix, ensuring realistic time allocation, and fostering cross-functional collaboration.

Key roles such as test leads, quality assurance (QA) specialists, business analysts, and subject matter experts (SMEs) are crucial for ensuring thorough validation and preventing last-minute chaos. It is also essential to book resources early enough to guarantee availability, ensuring the right people are engaged at critical times. Additionally, involving testers early in the process helps make sure that requirements are testable and potential risks are identified from the outset, leading to a more efficient and proactive testing phase.

Tooling

Moving from Google Sheets to a dedicated testing tool like XRay (Jira add-on) can greatly improve efficiency, but the transition requires careful planning. Choosing the right tooling is just as important as the transition itself. A proper tool should support test case management, execution tracking, reporting, and seamless collaboration across teams.

The onboarding process involves defining baselines utilizing subject matter experts’ knowledge, creating and uploading all test cases into the XRay tool, and configuring workflows to align with program requirements. Gaining stakeholder buy-in, providing structured training, and ensuring ongoing support are critical for successful adoption. A well-planned change management approach, combined with a phased implementation, helps prevent disruptions and promotes a smooth transition. By investing in the right tooling and a structured rollout, teams can enhance test visibility, improve collaboration, and streamline overall test management.

Leveraging AI for Test Case Drafting

Creating test case drafts from problem definition documents in a structured Google Sheet format before uploading them to XRay, can be significantly accelerated with AI. However, this approach is only feasible if the company has its own secure AI environment where confidential documents can be safely processed. Ensuring data security and compliance is crucial before integrating AI into test design.

When used within a controlled environment, AI-powered tools can analyze requirements and expected outcomes to generate structured test scenarios, reducing the manual effort required to create test cases from scratch. While human review and refinement remain essential, AI helps speed up the initial drafting process, ensuring broader test coverage and greater consistency. This allows testers to focus on validation, refining critical edge cases, and improving test quality rather than spending time on repetitive documentation. By integrating AI into test case design securely, teams can enhance efficiency while maintaining control over the accuracy and relevance of their test scenarios.

Workshopping and training

Organizing test planning workshops and training sessions on the joint testing process (including XRay tool) ensures that teams are well-prepared and aligned. The process begins with internal foundational workshops covering test planning basics, terminology, and an overview of test phases. This is followed by hands-on sessions where each workstream reviews and creates test cases, ensuring participants understand the full workflow from writing clear test cases to linking them with requirements and tracking progress.

The next step in strengthening collaboration is a joint workshop with the external implementation partner team, dedicated to aligning test management principles, terminology, test planning, and defect management, while also defining success criteria and escalation protocol. Creating a shared understanding across teams ensures consistency, enhances coordination, and facilitates efficient issue resolution.

Supporting this effort, comprehensive training of all needed tools is essential to familiarize teams with key features such as test case creation, execution, and reporting. Providing written guidelines and best practices enables teams to navigate the tool independently. A structured approach to training and collaboration ensures smooth adoption of testing processes, fosters alignment, and enhances overall efficiency in test management.

Reporting

Defining clear reporting and defect management processes with the partner ensures seamless tracking and timely updates, meeting partner’s reporting expectations. Transparent reporting provides all stakeholders with visibility into testing progress, risks, and defects, preventing critical issues from going unnoticed. Utilizing visualisation tools, such as XRay dashboards, enhances this visibility by offering real-time insights into test execution, defect trends, and overall progress. A well-structured reporting framework enables teams to make informed decisions and keeps leadership actively involved throughout the testing process.

The Power of Collaboration

Anne Schydlowski's blog article "How to Be an Inclusive Project Manager" highlights the importance of fostering an inclusive project team culture. This mindset not only strengthens collaboration across the project but also plays a crucial role in the testing phase. An inclusive team culture supports the testing core team and test lead by ensuring open communication, shared responsibility, and a collaborative problem-solving approach - following Zalando’s founding mindset. Additionally, it benefits the overall program manager by creating an environment where diverse perspectives contribute to more effective testing strategies and smoother project execution.

Effective testing thrives on collaboration, transparency, and strong communication. Since testing spans multiple teams—from development to business operations—unclear communication can result in delays, missed defects and almost always duplicated efforts.

A successful testing process depends on well-defined roles, clear expectations, and seamless teamwork. When all stakeholders are aligned, confusion is minimized, and last-minute firefighting is reduced. Strong communication bridges gaps between business, development, and operations teams, helping to craft meaningful test scenarios and proactively address risks.

Transparency is key—regular reporting and clear defect tracking keep stakeholders informed and aligned. Standardized workflows bring consistency to planning, execution, and reporting, reducing the likelihood of defects slipping through or work being repeated. Instead of a last-minute rush, testing becomes an integral part of the project’s workflow.

When testing is treated as a shared responsibility and a collaborative effort, rather than an isolated task, the entire project benefits. Open feedback, constructive discussions, and a culture of teamwork lead to better test coverage, faster issue resolution, and ultimately, higher-quality outcomes, driving streamlined implementations and more successful projects.

Challenges Behind the Scenes

Testing goes beyond running test cases – it requires alignment, ownership, and a shared understanding of its goals. To overcome this, clear definitions and alignment on terminology are essential early in the project as teams may use the same terms, like UAT (User Acceptance Testing) or E2E (End to End testing), but mean different things, leading to misaligned expectations and gaps in coverage.

Another hurdle is roles and responsibilities. Without clear ownership and a well-defined (and followed) RACI, testing efforts can become scattered, with no one accountable for ensuring completeness. This is especially critical when teams assume testing isn't necessary - "We don’t need to test or at least we don’t need to document it; it’s business as usual." - only to face unexpected failures in production.

While functional testing ensures individual features work, it’s process testing that validates everything works together in real-world scenarios. Skipping this step can lead to unnoticed failures in critical business workflows.

Additionally, limited test coverage and last-minute changes make test management even more complex. Success depends on strong coordination, early involvement, learning curve and structured execution - without it, testing remains an afterthought rather than a safeguard for business continuity.

Last but not least, collaboration in testing isn’t always smooth. Misaligned expectations and poor communication can lead to gaps in test coverage, delayed defect resolution, and last-minute surprises. When teams work in silos, critical information gets lost, and assumptions replace clear agreements on what needs to be tested and how.

Why Test Management Deserves More Attention

Testing is often seen as a necessary hurdle rather than a strategic enabler, but as we’ve explored, its impact goes far beyond finding bugs. Effective test management not only prevents costly issues but also strengthens business continuity and reduces unforeseen risks. From terminology alignment and resource planning to process validation and AI-assisted test case drafting, every aspect plays a role in delivering a successful implementation.

By prioritizing structured testing - not just functional checks but also process validation - we can reduce last-minute chaos, minimize business disruptions, and ultimately build stronger, more resilient systems. Testing isn’t just about quality assurance; it’s about business assurance. The more we recognize its value, the smoother and more predictable our projects will be.

Acknowledgments

A special thank you to our overall program manager Miguel Dupret for his invaluable support in aligning teams and driving the successful onboarding of new testing processes. His leadership has been instrumental in ensuring a smooth transition and fostering collaboration across multiple teams.

I’d also like to extend my appreciation to all the program workstreams that have embraced the new ways of working and taken the new XRay tool into use. Their commitment to adapting processes and improving collaboration has been key to making this program successful.