Quality Matters: The Benefits of QA-Focused Retros

Posted by on February 8, 2016

Retrospectives (retros) are meetings that take place after every major project milestone to help product teams at Etsy improve processes and outcomes in the next iteration of a project. The insights gained from project retros are invaluable to proactively mitigating problems in future projects while promoting continuous learning.

I am one of the managers on the Product Quality (PQ) team, which is Etsy’s centralized resource for manual QA. When manual QA was first introduced at Etsy, testers joined a team for the duration of the project but had limited opportunity  to get objective feedback. Consequently, testers were kept from seeing the importance of their contributions and from viewing themselves as true members of the product team they worked with. This lack of communication and continuous learning left our testers with less job satisfaction and feelings of empowerment than their peers in product engineering.

We decided to try QA-focused retros to surface feedback that would help us identify repeatable behaviors that contribute to consistently successful product launches. We were also interested in empowering Product Quality team members to understand how their specific contributions impact product launches and allow them to take more ownership of their responsibilities.

Regularly scheduled QA retros have helped to promote mindfulness and accountability on the PQ team. Over time, they have solidified relationships with product owners, designers, and engineers and reinforced the sense that we are all working toward the same goal.

Here’s some information to help you run a QA-focused retro:

Meeting Goals

Sample Agenda

How Does It Work?

The retro should be scheduled after the initial product launch or any milestone that required a significant QA effort. Participants should receive the agenda ahead of time and be asked to come prepared with their thoughts on the main questions.  The QA engineer who led the testing effort should facilitate the retro by prompting attendees, in a round robin fashion, to weigh in on each agenda item.

The facilitator also participates by giving insights to the product team they partnered with for launch. This 30-minute activity is best if scheduled near the end of the day and can be augmented with snacks and beverages to create a relaxed atmosphere. The facilitator should record insights and report any interesting findings back to the QA team and the product team they worked with on the project.

Who Should Attend?

Participants should be limited to those who directly interacted with QA during development. These are usually:

What Happens Next?

Everyone on the Product Quality team reviews the takeaways from the retro and clarifies any questions with those who participated in the retro. We then make the applicable adjustments to processes prior to the next iteration of a project. All changes made to the PQ process are communicated to team members and product teams as needed.

Conclusions

QA-focused retros have empowered our team to approach testing as a craft that is constantly honed and developed. The meetings help product managers learn how to get the most out of working with the QA team and provide opportunities for product teams to participate in streamlining the QA process.

Posted by on February 8, 2016
Category: operations, people, Uncategorized

8 Comments

Very insightful article. I’ve been on projects with and without post delivery lessons learned type gatherings. Projects with (retros) led towards future projects than ran smoother and quicker while those projects without (retros) seemed to always bog down over the same issues. This will not only increase employee moral but would also lead to more customer confidence as the customer will note the quality of the product delivered as well as shortened delivery times.

Couldn’t have said it better. Thanks Ken!

As outlined by Arylee, the presence of having QA-focused retros makes business sense. Constant constructive feedback during checkpoints throughout a project helps minimize costly errors in time and money. Always more inexpensive to correct an issue (if there is) in the early stages than later on.

In addition, one of the downsides of working in an unguided group is each individual is concerned with his/her objective. Very often the overarching objective ignored. Meeting that mission objective can better serve the long term benefits of a company and in turn, the clients.

Having a QA-focused retro can facilitate meeting both important objectives.

I love the idea of QA focused project retros. I’m a firm believer in the value of holding frequent retrospectives (we do them every two weeks on my team), but I had never thought about focusing them on a function (QA, design, engineering…etc). I think it especially makes sense because QA teams are often not centralized and that can make capturing learnings difficult. Those focused retros seem like a good way to address that.

Thank you for taking the time to share your thoughts and practices, Arylee. I’m the new QA manager at Idealist, and have been working on implementing great practices here to try make our work more efficient, more organized, and repeat mistakes less and less! The QA retro is a wonderful idea. We are very close to a launch now, which required weeks of QA, so next week is the perfect time to have our first meeting of this kind. Thanks again, and keep on being awesome!

    That’s awesome, Sara! Let me know how it works out :D.

I like the insights and recommended approach. We have a strong in-house IT presence and we have seen and experienced nearly all of the challenges mentioned in this article. I feel the focus on continuous improvement from a QA-centric perspective is an excellent way to approach advancing your team and organization.

I intend to try this with a couple recent projects that have had moderate challenges and a handful of setbacks (everything from team structure to overcoming late deliverables). As a team supervisor, my job is too make suggestions report observations, but more importantly facilitate and implement process improvements suggested by the team.

Thank you again for the great article.

    Thanks Gregor. I’d love a follow up to how things worked out!!