Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ


Choose your language

InfoQ Homepage Podcasts Eric Willeke & Ronica Roth on Leadership and Lisa Crispin on the State of Testing in Agile & DevOps

Eric Willeke & Ronica Roth on Leadership and Lisa Crispin on the State of Testing in Agile & DevOps

In this podcast recorded at Agile 2019, Shane Hastie, Lead Editor for Culture & Methods, spoke to Eric Willeke & Ronica Roth about supporting leadership through transformation and then he spoke to Lisa Crispin about the state testing in agile and DevOps

Key Takeaways

(Willeke & Roth):

  • Leadership teams are often not really teams at all – they are a collection of individuals looking out for different silos
  • Many leaders are actually trying to lead through change and they themselves don't know often how to begin to lead in the complex environment
  • One of the core shifts that allows continual change, is when the leadership team works very effectively together as a cross functional team of leaders in the organization
  • Facilitation and people skills are things that many managers have never been encouraged to pick up or rewarded for picking up


  • The value and importance of developers and testers learning from each other
  • The need to build truly cross-functional teams
  • Testers need to learn some consulting skills, coaching skills and facilitation skills
  • Testers need to educate themselves on new approaches and techniques, especially around continuous delivery and DevOps


00:00 Introducing Eric & Ronica

00:00 Shane: Good day folks. This is Shane Hastie for the InfoQ Engineering Culture Podcast. I'm at Agile 2019, and I'm here with Eric Willeke and Ronica Roth. Eric, Ronica, welcome. We've had you on the podcast before, but I gather that things have changed a bit. So, what's happened in the last year?

00:26 Eric: The fun news is that the three of us, along with Christine Hudson, have started a new business that we have named Elevate To, and perhaps a little differently than we've shown up in the past and talked about, we've taken a slightly different focus as we pursue helping companies achieve agility and achieve their potential.

00:44 Focusing on Supporting Leaders in Transformations

00:44 Yeah, we've really put the leadership alignment piece as the top priority. So, leadership teams are often not really teams, they're a collection of individuals heading up silos, we're starting there as the place to break down those silos, build true teams that are aligned in our work together.

01:03 Shane: You make a valid point and certainly it seems, from the outside, that a lot of those leadership teams really need help, or perhaps we're being a little bit uncharitable, but how can you help?

01:14 Ronica: They do need help, in the sense that they’re not like, Ooh, those terrible people over there need a thing. They are actually trying to lead through change and they themselves don't know often how to begin to lead.

01:27 Eric: Yeah. And I think, from the last time we talked, both Ronica and myself, we have a certain degree of empathy for those leaders, having in the meantime, become senior leaders in multiple organizations. And for me, at least personally, I look at it a little less as "they" these days. I have empathy for the role of leaders. I was influencing and at times inflicting work across the 3000 person organization in a role. And I was part of the problems for example, around work in process and creating change and hopefully part of the solution as well. But I gained a great deal of empathy for the leaders I'm working with and the efforts each individual wants to make. And the fact that each of their individual organizations is actually doing reasonably well, yet they, as leaders have to learn to collaborate with different functions, the same way individual contributors have to learn to contribute as part of a cross functional team.

02:17 Shane: So what does it mean to transform an organization?

02:21 Ronica: It's funny, I keep wanting to get rid of that word and yet that is so much what senior leaders are constantly doing. Circumstances change. The organization no longer has the capabilities it needs in some way to get done when it's trying to get done. And so that is usually most of the leader’s job is what new capabilities do we need? What new structures do we need that help work happen? And what new capabilities do we need in our organization? That's what they do. Sometimes. It's about agility. Sometimes it's about other stuff.

02:53 Eric: As we look at the world of business agility, these days, most of it is aligning around the overall value stream and that change, asking people to work together in fundamentally different ways is a transformation.Some are relatively small, we've got, were six scrum teams and we're going to blend some marketing in and help them communicate better. Some are relatively large. We want to take pieces of these eight top level organisations and invite them to work together as a whole end to end product. Both are transformations, they have significantly different change impact, and techniques and tactics you need to go through, but you're transforming your organization, ideally often and incrementally. And I think that's what gets lost is we assume transformation needs to be this huge thing, as opposed to a series of small, intentional changes.

03:35 Shane: Now you made an interesting statement there, invite them to work together a new way. Don't we just tell them?

03:42 Invite People on a Change Journey – don’t push them into it

03:42 You could.

03:43 Eric: Yeah, that is one way that is a leadership choice, I mean, sometimes it may even be the appropriate one, depending on the culture and the nature of your organization and the urgency of the situation. It could be a horrible decision that causes a lot of attrition and leaves, causes you to lose many of your best people.

03:59 Ronica: Yeah, I think of literally someone I'm mentoring right now, an architect in a large organization, and he's like, we're building a new team and we're trying to define what it is and I want to guide us to lean agile ways of thinking. And he's like, his former approach would be to tell, and he now knows that doesn't work. And he's like help me figure out how to invite so that the person he's trying to change their mind can step into rather than kind of be dragged into. And it's exactly what we're doing.

04:27 Shane: So what does that invite feel like? And look like?

04:30 Eric: For me, it's about allowing people the appropriate amount of space to explore how they want to fit into the new shape. In a large long term, whole company adoption that could be a year or two, six months of, Hey, this is, this is an experiment. Would you like to join the experiment, and if it doesn't work, we'll let you go back to your old role and not punish you or judge you in any way. We'll thank you for that. For the next year, it might be steadily increasing hey, you should probably join into this, it's kind of, exciting,  to we kind of expect this, too, towards the end, we'll get to the point of, you know what I hate to say this, but if you don't do this, your career is at risk and certainly your employment here is at risk, to maybe those final six months of the 24 are, well, if you're not joining, I'm sorry, but you're on a plan, taking it to that level.

05:13 Ronica: Or the opposite, right, I think of, to merge the two questions. Right? So, I worked with a 10-year-old startup and their transformation is they're shifting their business. They've been sort of B to C, they're going to go B to B after being kind of steady-state, they want to grow a lot and so, the CEO has been building a new leadership team, and the space for the invitation was let's have Ronica facilitate our executive offsite this quarter when we dream big. And I said, great, let's dream big. I'm going to spend a day doing team formation first and help create a space in which you're truly collaborative, in which you all have equal voices in which you all get to, to bring, to bear your expertise and your passion. So that in a very compressed time of like three days, we get to the place where we're aligned and working together and moving towards transformation.

06:06 We know how to help delivery teams become more effective, but struggle to achieve the same results with management teams

06:06 Eric: Yeah, that reminds me of one of the big concerns I have is, in the agile community, we spend a ton of time talking about team formation and how to create, fundamentally we talk about development teams, and we spend relatively little time talking about how to take groups of managers and turning them into teams of leaders. And that's one of the core shifts we see that allows continual change, is when you have a leadership team working very effectively together as a cross-functional team of leaders in the organization,

06:34 Shane: But isn't it, we want to get rid of all these managers, the layer in the middle, the frozen middle, the marshmallow layer? We hear that so often.

06:42 Eric: I think that attitude is one of the most damaging things to agility that I've seen in a long time. Leaders are the best-connected individuals in most companies. They usually have the most experience of how the company works, what its business models are, how they function. They're often the ones who designed the nervous systems and the DNA of the company at some point in the distant past, why would we be taking that experience and uninviting it in any way, shape or form?

07:05 Ronica: Well, and to go back to something you said in the beginning, that's where my personal empathy and growth in the last few years comes in. So, when CA Technology acquired Rally, I learned, I was a senior leader, I learned to be a leader in a much larger organization and we got to exert our influence over the whole corporation. And I met leaders at all levels throughout that organization and it was working with them as a peer that just taught me a lot about how to really move a company in the way that matters. Those are people, those are people with talents and influence, and I grew so much in those three years or whatever it was.

07:44 Shane: Ronica, you were talking about reforming to cross-functional value streams and so forth. What does that look like and feel like?

07:52 Ronica: What does that look like and feel like?  This is where I probably oversimplify, and Eric will give a much more interesting answer.

08:00 What does a cross-functional value stream look like?

08:00 To me, it's cross-functional teams all the way down and it's turtles all the way down. It looks and feels exactly as awkward as it felt when we first put developers and testers on a team together. It breaks the paradigm of how we think we need to operate as a company. It breaks the paradigm of, we centralize functions because we think putting all of the smart marketing people together is the right thing to do to make them effective and to ask them to work on a team with their sales colleagues or their product colleagues feels difficult. It feels just like forming scrum teams used to feel.

08:32 Eric: And I think mine, honestly, is about the same, because the idea and the concept should be about as obvious as listening to your customer and yet we've struggled with that for so long. So, if you think about taking that product-centric, customer-facing view and stretching it all the way through, then to me that's the end of value stream thinking. What're all the things you need in order to give your customer a phenomenal experience, one that delights them beyond their expectations.

08:58 Shane: Now, this can be a really scary shift for people who have traditionally worked and been measured in their silos. We've got the dichotomy that we've had for years, for decades, about business and IT, and that we're starting to break apart, or are we?

09:18 Eric: Well, there's a risk, and our community is loving to talk about business agility these days. And I'm worried we're starting to talk about it as if it's something fundamentally different, the same way that we tried to treat IT and the business, and I'm using air quotes here you can't see on the audio, as completely separate things. And if we allow agility itself to splinter into these two different domains, then we're going to have the same problem at a different level. When we started look at how to make it safe for people to adopt whatever this new form is, thinking of Joshua and safety as a prerequisite, and Modern Agile's mindset towards it. Three keys to meaningful change. That's where the invitation piece comes in and to create that invitation at a meaningful level. We look at three different aspects of how we engage the organization. At the first aspect, it is the value stream focus. It is let's create that product or that service-oriented mindset cantered on the customer and try to get all the functions necessary to collaborate in a positive, effective manner. The second place is to start looking at where we as a leadership team, and I always include myself in that when I'm speaking as the change agent, where have we created friction, where have we created impediments to people behaving in the way that we want them to behave and we start to identify and build initiatives, organizational initiatives, against those things impediments, like change our performance system, like global scale co-location initiatives, things like building job families and job architecture around the agile roles we're asking people to behave, so they can see their career progressions, things like changing our funding model, fundamentally, so that we can fund against those end to end value streams instead of having a cost conversation every time we make a decision, and we need to eliminate as many of those large sources of cross cutting friction as we can in the organization. And then the third aspect of it that we do, and this is kind of one of Ronica's big specialties, is inspiring the people through education and storytelling. You want to say a little bit more there?

11:18 Storytelling to help inspire culture change

11:18 Ronica: We talk about culture as a thing you can't change, you can't change culture and culture just is all blah. But if you want to envision a new way of working in your company and have that really be deeply instantiated, one of the biggest, the tools for doing that is storytelling. So, keep telling stories, wherever we see people in the company behaving in the way we want, then we tell stories about it. And I always think about our mentor, my mentor, Jean Tabaka, instantiated at Rally, you know, at the very beginning that we do appreciations. And everyone thinks of it as a culture of appreciating, being thankful, but it was really a culture of storytelling because it said whenever you've seen it, your colleague act according to our values, appreciate that colleague. Well, that always came with a story about someone living our values and that's what made them real. And that's one of the things we can bring into organizations, help leaders do, keep telling stories about the thing you want and it begins to change it. Another piece of that is, when you create a culture of collaboration when we teach, and actually this is where frontline managers absolutely are the most powerful vector for this, is teach frontline managers how to run meetings in a way that brings voices, all voices into the room, that makes space for contribution, that makes space for the diversity of thought, that is instantiating culture and helping create a company that every single person day in and day out is acting in the ways that we want, that will make our company awesome.

12:52 Shane: These are not skills that frontline managers are often trained in?

12:57 Ronica: Oh, good thing we know how to train them.

13:00 And the good news is, I think in the agile community we all believe in the growth mindset. They are learnable skills, they're practice-able skills. They're just simply skills that people have never been encouraged to pick up or rewarded for picking up for that matter, at least not visibly, although if you look higher up in organizations, the very senior leaders who know how to create good spaces do tend to get promoted, even though it's not for that reason typically.

13:25 Ronica: Good point.

13:26 Shane: What is this meeting culture thing, how does that work?

13:29 Meeting culture and psychological safety

13:29 Eric: For me, and I came from a slightly different background than the other two of my cofounders, but for me, the meeting culture really is an anchor for the research that came out of Google that talks about the best performing teams. And it's the teams that, all other factors put aside, have above average emotional IQ and have roughly equal talk time. And I look at that in the context of design thinking where we are trying to intentionally create divergence before we converge on what we want as a group. And yet, we have an alpha driven culture in many of our organizations where the first interesting idea that gets thrown out there, everybody piles on and we either explore it or we attack it or we debate it, but we only get one idea out there. And a lot of the meeting design that we're looking at is how do we baseline everybody on a common understanding of reality? Where are we as a business? What have we accomplished? What are our previous aspirations and how did we do against them? And then intentionally and with incredible discipline, create the space for many ideas to emerge into and diverge the space before allowing everybody to then pick on the ones that have the highest energy and converge them towards the ones that we want to do. And then once we kind of set as an organization, here's where we want to go, we then again, diverge back into how do we want to get there and look at many sets of tactics we could use to achieve those goals, before as a leadership team and perhaps as multiple levels of leadership teams ideally, bringing back together and here's how we will achieve those tactics, or which ones we will pursue.

15:04 Ronica: You just made me realize you, of course have famously given a talk on WIP for years, of which you have a new version tomorrow. We're very excited about, but you know, when most people hear that story of how meetings can be, then of course, they're like, that takes way more time and I have way too many meetings. So part of how you get there is ,there is a limiting of WIP in which we get rid of all the stupid meetings and the ones that don't need to be meetings, where that work can get done in different ways,.  So that when we do come together, it is because that is absolutely the best and only way to find the right strategy forward. And so we have fewer meetings and then make them richer and deeper.

15:46 Eric: Yeah. And I think we want to avoid overlooking the other aspect is that by investing the time into these meetings, you generate the alignment and the consensus that it's not false consensus, it is a true consensus where everybody contributed to this plan. And as a result, most everybody in the organization might be one at most, two degrees of separation from somebody who is part of creating the plan. And if you think about how many meetings that eliminates as part of decentralizing and allowing things to move forward, you find yourself where you suddenly don't have as many meetings to follow up on the meetings or plan the meetings or plan to plan the meetings. I think I got five degrees on meta on a recent meeting on that. And instead you're focusing more time on executing against the plans you collectively agreed to.

16:31 Shane: Thank you very much for that. Now, one of the things you mentioned earlier is when the formation of your company you've pledged one, two, three, Do you want to tell us a little bit about that?

16:41 Corporate social responsibility and giving

16:41 Eric: Absolutely. It's something we were really inspired by, and the pledge 1% movement, I think it was Mark Benioff at Salesforce that originally founded it. Rally was a proud member through Ryan and Tim and as they founded the company and the original version of that is the company would pledge 1% of its equity or 1% of its profits or 1% of its employees time. Towards doing good in the world.

17:03 Ronica: Which was a huge part of Rally, I remember the encouragement constantly to go volunteer and we would do it together, we do it separately, it became storytelling. We tell each other stories about the ways we went off as individuals to give our time. And I'll never forget when we IPO'd and the big check we gave to the community foundation in Colorado. So, this is the evolution.

17:23 Eric: So taking that theme of doing well by doing good and carrying it forward, we all want to create positive change in the world, both through our work but also through the benefits of our work, and we decided that at least for now we can do better. So we are pledging 1% of our equity, we have 1% of the company set aside to go into trust once we're large enough to actually have a valuation where that's meaningful, we're going to pledge 2% of our profits and for the founder time, we all have committed 3% of our time and we all are actively involved in a number of not-for-profit pursuits and we want to make that formal. And so it's 3% of founder time right now, we'll figure out what that means if we have more than just the founders, but that's one of the commitments we're making to the world as part of giving back what we get from helping people,

18:07 Shane: Eric Ronica. Thanks very much, good to catch up.  If people want to continue the conversation, where do they find you?

18:13 Ronica:

18:15 Eric: Yup. Elevate to greatness. And we look forward to hearing from you.

18:18 Thanks so much.

18:20 Lisa Crispin on Agile Testing

18:20 I'm at an agile 2019 in Washington, DC and I've got the privilege of sitting down with Lisa Crispin. Lisa. Welcome. Thanks so much for taking the time to talk to us.

18:29 Lisa:  Well, thank you very much, Shane. Your podcast is one of my favorites and I did get horribly behind on it, but I'm going to catch up because you just have so many awesome guests and so many awesome topics, so I get a lot of inspiration and knowledge from it.

18:43 Shane: Thank you for those kind words. You and I know each other well, and your name is well known in the testing community, but do you want to give me the two-minute overview? Some of our audience probably haven't come across you and your work. 

18:56 Testing Agile and DevOps

18:56 Lisa: My main thing is testing in agile teams and DevOps, we use the word DevOps a lot these days, so DevOps teams, and I've been a hands on tester on agile teams for most of the past 20 years. And currently I'm a testing advocate at Mabl. So my job is to, I try to educate the community, help people with things like continuous delivery testing and DevOps, test automation, and also just to go out to the community and what problems are we seeing that are difficult to solve? And if my company can't necessarily help solve them directly, but maybe we could as a community solve those problems. So, it's a really fun opportunity to just help everybody increase their education and make new connections.

19:38 Shane: At this conference, you've given a workshop on what testers and developers can learn from each other.So what can testers developers learn from each other? What testers and developers can learn from each other?

19:47 Lisa: This was a true workshop. So, Bill Wake and I facilitated it and we allowed the table groups to decide what topic do they want to talk about. So, it could be something that testers could help developers learn or vice versa. And so they each pick the topic and then we ask them to make a poster to talk about why that topic is important. What can they do to learn about it, how could a test or help a developer learn about a certain exploratory testing or some testing topic, or how could a developer help a test or learn about something like unit testing is more of a developer thing. And then, so they had some time to make a poster and then we asked them to send somebody over to another table, we paired up the table groups, and get feedback on their poster, get new ideas for the other team. And so just as a way to try to brainstorm more ideas. And then come back and then they could update their poster and finalize it. Then we put them up and had each team walk us through their ideas. So it was really interesting. It's hard to do that in 75 minutes, but we got some really interesting things out of it.  My favorite poster, they started brainstorming topics and they, then they say, let's dot vote to pick a topic and then they had a kind of, a lot of topics that were popular and they looked at it and I said, you know what this is about? This is about all communicating better together and working better together. So they made a poster with construction paper with half a house of quality and it had different little doors representing big topics. And then when you open the door, you could see resources where to learn about that topic. And then across the bottom, they had drawn little caricatures of each person at that table group, their faces, and it was across the bottom. And they said because the foundation of our house is the people on our team. And I was just, oh, I love that. Yeah, people were really creative and really pinpointed some areas that we do struggle, and we struggle to work together and that's something really important that we need to work on.

21:43 Shane: So what are some of those areas and how do we work on them?

21:46 Lisa: Some of it is just techniques. So as testers, we can help developers learn some testing techniques, such as exploratory testing or accessibility testing, security testing, whatever it is because, you know, in the modern world, doing continuous delivery, it doesn't scale very well, unless you have as many or more testers on your team as developers trying to get all the testing done, it would just slow things down too much. We need the developers to be building that quality in and thinking about testing and testability from the get-go. And if we transfer some of our knowledge to them, them and give them those skills, they can do that at the story level they can start doing these kinds of manual testing, for example, they're probably already pretty good at the test automation. So just building up those skills and vice versa. As testers, we need to learn, you know, some of the language, like for things like DevOps, we need to learn the terminology so that we can ask good questions about it. We need to learn how to use those tools. So there's a lot of back and forth that can go on and take that whole team approach.

22:45 Shane: Let's explore that whole team approach. How do we get our teams to collaborate cross-functionally and become that whole team? Encourage teams to collaborate cross-functionally

22:55 Lisa: Well, a lot of it has to do with the leadership and management support.So, for example, on my last team, the director of the development realized to go to continuous delivery we only had two testers for about 30 developers, that wasn't going to scale, and he knew how important exploratory testing was. And so, he made a well, not just him, but all the managers worked on it, kind of a career progression matrix for the developers and one of the sets of skills they needed to have was exploratory testing. So they had to achieve a certain level of competency to rise up the rankings in their job and their career.  And so that gave them motivation to do it right. And so then they asked the testers to do workshops that teach them exploratory testing and to pair with them a couple of days a week, and the developers were keen to learn. They saw it was really useful and valuable and they could do exploratory testing at the story level, before they checked in each story, before they said they were finished. We had a little checklist for them to go through. We had Elizabeth Hendrickson's testing heuristics cheat sheet. They knew how to write charters, and so that did work really well. It did enable us to start releasing twice a week, which we couldn't have done in our old way of doing things.

24:05 Shane: What would be in your checklist? Let's take exploratory testing. A lot of our audience are those developers. So what would you tell the developer in a couple of minutes? What is exploratory testing and what should they do?

24:16 Exploratory testing heuristics

24:16 Lisa: Well, since most of my recent experience is in web based applications, I would say maybe try a browser other than Chrome and just try some of the heuristics, you know, try boundary conditions, try filling up the field. If it's a numeric field, try a negative number, just these different, you know, don't just do the happy path, do the sad path. Try some edge cases, but also thinking about the value to the customer. How's the customer going to really use this and try to channel that customer. We can use personas and stuff because there might be some education things that we don't even care about because the customers don't care about them. But if there's the one really valuable capability of a feature and it breaks, we need to know. So teaching them that kind of focus.

25:00 Shane: And to the tester, how do they become better at mentoring and conveying these ideas? And also, what are some of the specifics they should be looking to learn?

25:10 Lisa: Well, we obviously have to learn some consulting skills, basically, coaching skills, facilitation skills, like can we facilitate a workshop to teach people some technique that we use in testing or any kind of technique because I bring a lot of things back from conferences like this, you know like I came to Agile, I think it was 2015. I learned about example mapping from Matt Wynne, and I was able to go back and do a workshop for my team and say, Hey, look at this, I think this would really help us with our cycle time and our rejection rate to do this example mapping to help better shared understanding up front and that worked out really well. So, that wasn't a testing technique per se, but it was something I could learn about and facilitate because I had facilitation skills. And one of the testing superpowers, I think many testers have is knowing the right people to get together, to talk about something. So just knowing who needs to know this, how can we help them learn it?

26:00 Modern Testing Principles

26:00 Alan Page and Britton Jensen have these modern testing principles they've come up with, which you can find that, I think. And it's the idea of, we should see it as part of our job to act as consultants to the team and help them learn these other skills. It isn't necessarily easy and I think it takes a lot of courage, but testers have a lot of courage because we're always having to deliver bad news. So we kind of have a lot of practice at that. But, you know, just bringing up these ideas as an experiment, let's try this, let's try it for two weeks and see if it works or not. Also observing, being able to step back and maybe see some patterns, some risks, we're good at identifying risks, and we're good at asking questions that maybe nobody else thought of. So don't be afraid to ask the questions and, you know, educate yourself on what the rest of the team is doing ,so that you can ask intelligent questions and collaborate with them.

26:55 Shane: I know that you and Janet Gregory have written one, actually it's more than one.

27:00 Lisa:  We are just about to finish up our third book.

27:02 Shane: Let's start with numbers one and two. What were they about? And then let's delve into book number three.

27:07 Book: Agile Testing

27:07 Lisa: Well, Agile Testing was our first book and it was about the whole team approach quality. Get your team together, decide what level of quality you want, that you can commit to and make it happen. And so we provide that a lot of models to help teams have those conversations. Like Brian Merrick's agile testing quadrants, things to help you with your test automation strategy, like Mike Cohn's test automation pyramid, which, you know, that was a while back, so nowadays there are a lot of really additional models for all of these things, because, you know, a model is just a thinking tool and different visualizations work for different teams. So whatever works in your context, try them out, but mainly have those conversations, get people drawing on the whiteboard, get people thinking up experiments. And we had a lot of stories from teams around the world in that book to say, here are the problems that a lot of agile teams are having with testing and here's what some teams have done to solve the problems. And we found that people have similar problems and solve them in similar ways. It doesn't work everywhere, there's no one best practice, but we can at least inspire people to say try this, if it doesn't work, maybe something similar will work for you.

28:16 Book: More Agile Testing

28:16 And then More Agile Testing, you know, time went by and we learned lots of new things about testing, about agile practices, technology and so we just went into more things that we didn't cover in the first book. Like how do you test in big enterprise companies? How do you test a big distributed team? What do you do with remote teams? More things on specialized testing like security testing and accessibility testing and testing embedded software, testing on mobile devices. And because Janet and I obviously are not experts in all of those things, we were able to get about 40 different leading practitioners in these different specialties and they contributed 70 sidebars to that book and people, we always get a lot of feedback that that's one of the things that people appreciate the most about the book, is all the stories from these other teams and how they had these problems and what they did to solve them. So it's really gratifying to be able to do that. 

29:05 Book: Agile Testing Condensed

29:05 But those are very large books are each about 550 pages and nobody reads the big books anymore, so we tried to write a small version of it. And so we have Agile Testing, Condensed a brief introduction, and it's a hundred pages and we're still working on it. There's a draft available on Leanpub, but we're working hard with, as we do copy editing and things like that, that we keep it under a hundred pages. But mainly it's just an overview of what's in the first two books, plus, again, new things have happened since we published the last book, especially a lot of the focus on continuous delivery, which we covered in the last book, but there's just much more focus on it, testing in production, there's a bigger focus on observability - that was not even a term we'd heard when we wrote our last book. So we wanted to add in these things and because it is a small book, the idea is linking off to other resources where people can learn more, because there are so many great resources out there. And also linking back to our first two books  for more details about things like Agile Testing Quadrants or visualizing your test automation strategy or any of those kinds of things, just different agile testing principles and key success factors and core practices that work for a lot of teams. So those are in more detail in the big books, but at least people who are new to it, or maybe a QA manager or a development manager who wants to just learn about testing and agile, they can get a good overview in that book and then wherever they need to learn more, they can go dig and find more.

30:32 Shane: You mentioned l LeanPub, is that where people can find the book?

30:35 Lisa: The book is available on Leanpub and like I say, right now it's just a draft, but we'll have the final one on Leanpub hopefully next month. And we will eventually also get it on Amazon and have it available as a print book too. So right now, it's just an ebook.

30:46 Shane: You also have a website where you're looking to build a community around testing and DevOps. Tell us about that.

30:53 Building a community around testing in continuous delivery and DevOps

30:53 Lisa: Well, I've been very interested in how can teams be successful with continuous delivery and with DevOps and how can testers contribute to that? Because it's obviously where the industry is headed. We need to have these frequent releases of value to production, small changes that are less risky. You know, as Jez Humble says it's just like, it's not dramatic. It's just routine. We're just releasing to production. It's not a big deal. And we have automated regression tests that give us the confidence to keep releasing and nowadays we have really great tools for monitoring and observability, all these operability areas so that we can see when there's a problem and respond really quickly. And so we know we can't test all the things. No test environments look exactly like production. We're never going to catch it all. We should try, but we can't and especially as we're releasing twice a week, every day, multiple times a day. And so I want testers to realize they need to get involved because as I see a lot of teams that call themselves DevOps teams or they're practicing DevOps, or they have a DevOps culture, but they're not involving testers, and I think they're missing. I think it's like the early days of extreme programming, they were like, we don't need any stinking testers. We'll do, the customers will write the tests and we'll automate them. And that was such a narrow part of testing. And like I say, testers are really good at identifying risks, seeing patterns, so things like observing production use, I think that when we learn to use these amazing tools that we have out there, that we can really start to spot, Oh, there's a problem area here. There's a fragile area here. Let's do some more testing on it. Or, you know, maybe let's revert that for now until we can do some more testing, or let's use feature toggling so we can do exploratory testing in production. There are just so many more opportunities and so many ways we can do more effective testing. I don't want people to be scared off by the thought of wow deployment pipeline and DevOps, it just sounds very strange and technical and I don't know anything about that and so I'm just going to shy away from it. And especially if they're not exactly inviting testers in, so let's learn about it. Let's learn the terminology. Let's learn the tools. Let's learn the practices, let's build those relationships. Mean. Katrina Clokie has a wonderful book A Practical Guide to Testing in DevOps, and she spends a lot of time in the book talking about identifying the relationships you need and building those relationships because you have to work with other people on your team and people outside your team, and that's such an important part of it. And so I and other people who've contributed to this website want to get testers excited about it as a career opportunity, we can really add a lot of value and will help our product and our team. That's the motivation behind it and I appreciate all the kind wonderful leading practitioners in testing and continuous delivery who've contributed guest blog posts.

33:36 Shane: So, what are some of your favourite of those posts? Some of the things that people should be looking for?

33:40 Lisa: So many, but one of my favourites was Abby Bangser, she did a post on how her platform team at Move tackle their problem of alert fatigue. So, you know, they have all this monitoring and they have all these alerts, but then if they get a lot of alerts and then by the time they responded, the problem fixed itself, or, you know, It makes you start ignoring the alerts, right? Because every time you look, it wasn't anything. You want to keep the alerts for really critical things. She describes the process, they went through to do that, and it was very interesting, and then she makes parallels between the failing automated tests, the same kind of thing happens.  If your tests are flaky or they fail a lot for no particular reason, you stop paying attention to those failures. And so as a team, we've got to tackle that problem. And so there are a lot of interesting parallels between testing and test automation and site reliability engineering and I thought that was interesting how she drew those parallels.

34:40 Shane: Anything else that you would point people to?

34:43 Lisa: Oh yeah. The most recent one by Rob Hornby is on the metrics that his team has used to measure their progress in moving to continuous delivery. And so he's on a platform team supporting all these different development teams and feature teams. And so they've been able to go from taking months to provision a test environment for them to, they can do it in hours, but they've got really great measurements that, you know, they set these goals and they have these measurements, so they can really track their progress and see how they're doing and getting that feedback from the measurements is really important. I think a lot of times we set goals and we kind of wave our hands and say, well, we'll feel better when we do this. They say, no, get some hard data. And there's a book. I can't remember the exact title, but there's a book on measurements for continuous delivery, which I have bought, but not read yet, and that's what they use to guide them. And I think that's super important, you know, we're monitoring in production and getting all that information. We need to measure our ability to build and support the infrastructure in order to do that continuous delivery. So that was really interesting and new to me.

35:51 Shane: Lisa, thank you so much for taking the time to talk to us today. It's been really good to catch up and to hear what's happening in the testing space. If people want to continue the conversation, where do they find you?

36:02 Lisa: Probably the easiest place to get me is on Twitter. I'm @LisaCrispin and you can also email me, that's pretty easy. I'm Lisa Crispin on LinkedIn as well, so pretty easy to find. And I love hearing other people's experiences. I love helping people out if I can or pointing them to somebody who can help them out that way. I'd love to hear from people.

36:23 Shane: Thank you so much.

36:23 Lisa: Thank you. 


More about our podcasts

You can keep up-to-date with the podcasts via our RSS Feed, and they are available via SoundCloud, Apple Podcasts, Spotify, Overcast and the Google Podcast. From this page you also have access to our recorded show notes. They all have clickable links that will take you directly to that part of the audio.

Previous podcasts

Rate this Article