The AI Revolution Fails Without Psychological Safety For Developers: A Conversation with Erin Doyle

Transcript

Michael Stiefel: Welcome to The Architects Podcast, where we discuss what it means to be an architect and how architects actually do their job. Today's guest is Erin Doyle, who is a founding engineer at Quotient, with over 20 years of experience building across the full stack. From mobile and web products to the platform architectures that power them. Throughout two decades in the trenches, she has developed a deep obsession with the human infrastructure of software, specifically how engineering culture and developer experience, commonly called DevEx, dictate technical success. A frequent speaker and writer, she's dedicated to helping teams navigate the psychological shifts required by modern engineering, moving past the hype to build high trust, high velocity organizations.

Becoming an Architect [01:32]

It's great to have you here on the podcast and I'd like to start out by asking you, were you trained as an architect? How did you become an architect? It's not something you decided, one morning you woke up, you got out of bed and said, "Today I'm going to be an architect".

Erin Doyle: Yes. Well, first of all, thanks for having me. I'm really excited to be here. I'd say I've never had a role with the title architect, but I feel like it's a hat I've worn for a long time now in my various staff, staff plus roles. And now that I'm a founding engineer on a relatively small team, I've got to put on the architect hat, amongst many others all the time. I wouldn't say it was something I was trained to do, but definitely have learned over time and have done all my system design and architecture, learning and reading. Yes, here we are and I have to make those decisions now.

With AI, Architecture Skills Become More Important [02:25]

Michael Stiefel: This would seem especially true as we start using more and more artificial intelligence in developing software, because as the coding and the development gets pushed to the agent and the humans have to both construct the requirements and do the testing, the architecture skills seem to come more into play.

Erin Doyle: Yes, I would absolutely agree. It's something I've really been feeling lately. It's like my day-to-day, I'm doing way less coding, maybe none some days, and it's all really high level. It's thinking holistically across systems, it's looking at the big picture, it's a lot of reviewing code, it's a lot of reviewing design. It's a lot of planning and gathering context, and then also taking outputs, whether it's code, whether it's documents, whatever it may be. And trying to translate those, so that those are easy for my teammates to understand and work with. I feel like I'm doing a lot of architecture and sort of tech lead style work these days.

Michael Stiefel: There's so much in what you just said that I want to unpack, but let me start with this. I've always viewed, and I know you agree with this. The human constraints, the team constraints, both for the team that's developing the software, however it's developed, and the customer provide just as much constraint in a technical sense as the actual development requirements. And as we use more and more AI, how does that change? How do you factor in those constraints?

Human Constraints on Software Development [04:18]

Erin Doyle: Yes, I think that's a really interesting question, and I've definitely been experiencing that. Now that AI coding tools are getting so good, if you're using the right skills, you're using the right prompts, whatever it may be. Now that the quality is getting to be really high and trustworthy, we're shifting our focus from, "Is the code correct? Does it do what it's supposed to? Is it safe? Is it reliable?" All those concerns that we used to really have to pay attention to, all those are being met and we don't have to spend as much time looking for those things.

So instead, we're kind of zooming out on bigger pictures. And a lot of it has to do with, how are the humans going to use this code? How are we going to continue to maintain and reason about these systems when we aren't necessarily writing these things from scratch ourselves anymore? How do we maintain context and understanding? Sometimes we might be spinning up code, a solution to something that we don't deeply understand right away. We can see that, "Yep, the test passed. Yes, it does what it's supposed to, it meets the requirements, but do I really thoroughly understand the solution?"

I think there's probably a big divide here right now, as to, "How important is it that we understand the solution? Should we just vibe code and let it do its thing? If we know it's working, that's good enough. Are our systems now black boxes that we just verify or do we still feel like humans need to really understand our systems holistically?" So try and find the balance, trying to find it for each team. It's going to be different probably for each team to decide, where is their comfort level. But I feel like a lot of my time now is getting that alignment with my team. What are our expectations of the output? What are our expectations of understanding?

And right now we still want to be a part of it. We still want to understand our outputs. So then how do I make sure I understand? And then how do I make sure that I can put my name behind that work? How do I make sure that I can clearly convey it to my teammates when they have questions? I can explain reasoning, I can explain decisions, and I'm still responsible for those decisions. And again, this can be code, this can be documentation, this can be architectural design and decisions. Whatever that product is. I may be using AI as the tool, but I still am responsible for it.

So we're putting a lot of work now into, "How do we now take all of this output that we didn't generate ourselves and how do we package it for quick, easy, and thorough human understanding?"

Corporate Liability For AI Code [07:16]

Michael Stiefel: There was an article in ... I think the Wall Street Journal a couple of days ago, where they talked about, if the AI makes a mistake, who is liable? And insurance companies now are starting to write into their policies that we're not going to cover problems that come from when the AI ... Let's for the sake of simplicity, makes a mistake. For example, if you're a financial services company and you have to make certain guarantees to your clients, whether people use your platform, or if you're a platform engineer and you have to make certain guarantees to the software that uses your platform. We do have to know what these systems are doing at some level. The AI can't be sued, but your company can and you can.

Erin Doyle: Right. And I think that's the line we need to continue to draw and hold. And that's something that we have talked on our team about. I think all teams should talk about this and make sure there's shared understanding there. When someone makes a mistake, when someone produces an output and others ask questions about it, they can't say, "I don't know, Claude wrote it". Or, "Yes, Claude missed that". We can't throw the AI under the bus. It is a tool. It is not a replacement.

Michael Stiefel: The AI doesn't care whether you throw it under the bus or not.

Erin Doyle: Right. It doesn't have repercussions. And so I think we need to continue to use it as a tool. It's a tool in our toolbox, just like our IDEs and our documentation and whatever it may be, our computers. If I make a mistake, I can't blame my laptop. That would be ridiculous. So it's up to us to continue to own the process, own the result, but we have these tools to help us do things faster and better, and that's where it needs to stay.

Michael Stiefel: I mean, this is sort of what happened, for a much larger scale now, but I'm old enough to remember when people started to use compilers instead of assembly language. I'm really that old. And we had these kinds of debates. Could you really understand what the compiler is producing? Because at some point these compilers were generating code, not for human reading, but for efficiency. And now when you have pre-pipeline stuff ... I mean it's all kinds of stuff that people have no idea what’s going on inside the compiler, pre-process in the instructions, and looking at the latencies, and looking at the least recently used. It's amazing the amount of stuff I've forgotten about how compilers were built, but there's all kinds of efficiencies going on that we have no idea of. And we somehow learned to take responsibility. We somehow learned to not blame the compiler or the assembler.

Agile and the SDLC in the Age of AI [10:15]

So I mean, there's precedent. We've gone across this boundary before, but the way we did that was having tests and developing a software development life cycle that reflects this. So what does the software development life cycle look like today? I mean, what does Agile mean today?

Erin Doyle: Well, I am definitely opinionated when it comes to Agile. I've been through the journey of when Agile first kind of hit the scene and before that we were all doing waterfall or maybe other really heavy-handed frameworks like CMMI or ... What's it called? Rational Reason. I don't know. Anyway,

Michael Stiefel: Rational Rose, I think it was.

Erin Doyle: Rational rose. Yep.

Michael Stiefel: I go back to case tools. I remember all of that.

Erin Doyle: Okay. So really heavy-handed processes and all the problems that we were discovering with them at the time that Agile came out of to solve. I experienced that and I went to my Scrum master training and got certified and all that, and I was super overly prescriptive and dogmatic about the methodology. We have to do all these ceremonies. They have to be this many hours for ... Whatever. And at the time, because we were in a transition, it made sense. We needed that discipline because ... It's like you either need process or people. And if you can trust people to do the right things at the right times, then I don't think you should weigh them down with process. But during a time of transition like that, we needed process. We needed guardrails. But as time went by, people got used to it, they were doing Scrum or Kanban for the most part, and we sort of got the hang of it.

I then started to see teams that continued to be overly prescriptive and it slowed them down and it caused stress and effort that gave us no benefit. I'd see teams killing themselves to try to get everything done that they had committed to in the sprint because the sprint end was coming up. But there was no consequence. If they didn't finish 100% of that work by Friday, nothing bad was going to happen. They weren't missing any true deadline, but the amount of stress that caused people, the amount of rushing and all sorts of bad side effects were just so unnecessary.

So I feel like I've learned that you only add process when you really need to. When you can't just depend on people to know to do the right thing at the right time, then very lightweight start adding processes. I think following anything too strictly, especially before you've identified, like, "Okay, we have a problem here, how do we want to solve it? Is it a people thing? Is it a process thing?" If you just start with process, you never know what you're capable of doing and you kind of hold yourselves back.

So that's my super opinionated take on Agile, but we're seeing now that AI can fit in to every step of the SDLC. If your team is up for that, I'm going to say over and over and over again, so much of this is team alignment. It's deciding, where are we collectively comfortable? What do we want to do as a team? What's the right fit for this team, these people right here right now at this company? And as soon as you have a member leave or you have a new member join, I feel like you kind of need to realign because that dynamic can shift.

Looking at the SDLC and looking at, "What are all the ways we can use AI to enable us to do things faster, maybe raise quality?" There's a lot of benefits. So looking at each of those opportunities and deciding, "Okay, we are cool with generating documentation". Or, "We're not cool with that. We want humans to own that". Whatever your comfort level is, I think having set expectations ahead of time.

So at Quotient, we use Quotient. We dog food our own product and it allows us to go ahead and measure our team's productivity and where we're at. And so one of the things we look at is, what are you using AI for across the SDLC? What are the different areas that you're currently utilizing it for? And then you can kind of compare your team against the rest of the company. And what's cool about that and what really helped us when we were first getting into it is you might see like, "Oh, wow. Some people are using this for debugging and I had never thought of that use case". And so it can start a conversation and be like, "Hey, I see some of us are using it for this and I'm not. What are y'all using it for? What are you doing? How are you doing it? What prompts? What skills?" You can start that conversation and now you've unlocked a new efficiency for a new area of the SDLC.

So I think having those conversations with your team. Like, "Who's using it for what? How are you using it?" And learning from each other can really boost your performance from start to finish.

The Divide Between AI Believers and Skeptics [15:43]

Michael Stiefel: It sounds, however, that the true believers in AI and the skeptics about AI need to have a certain amount of respect for each other in order to have those conversations.

Erin Doyle: Yes. And this is something I'm worried about. I feel like we are getting to a point where AI attitude is becoming polarizing. You've got the people who are on board and they may be skeptically on board. They may be on board because they feel like they have to, or they may be on board because they think it's great. Whatever your reason, you've got the people that are on board and they're leaning in and they're trying to really use AI to its fullest. And then you've got the people who are resistant. Same thing, maybe they are ethically opposed, they see the environmental impacts and they don't want to be part of that. Maybe they don't like how it's changing their jobs and they don't want to be part of that. Maybe they just don't know how to use it effectively. Maybe they've tried on their own, they're getting crappy results, they're still super skeptical. They don't trust the output and they feel like it's wasting their time.

There's a whole array of reasons that you can be on one side or the other. But regardless, once you're on a side, it gets really difficult to move forward if you don't have team alignment. If you've got one member of the team that's not leaning in, they're not using AI to its fullest and their work's taking longer, or maybe they had a bug that took them days to resolve. And the rest of the team is thinking, "Ah, you could have just used Claude for that".

Now we've got these different concepts of how work should be done or how long it should take, and that just gets really dangerous and judgmental. And I think once you get there, you can get stuck. Your skeptics no longer feel comfortable sharing their skepticism. They no longer feel safe to share like, "I don't know how to do that. How do you do that? How have you found success because I'm still struggling?" They may not have that safety to ask those questions, to ask for help, so they can get caught up. That feeling of being behind can get so big, then it becomes a barrier.

Psychological Safety [18:07]

Michael Stiefel: So it seems in this age of AI, for lack of a better term, it can be as vague or as meaningful as you want. It seems humans become in some sense even more important because on one hand ... The managers become even more important because it's the managers who have to create this space of ... I don't know if you want to call it psychological safety, where the humans do not feel threatened by the AI. I mean, if you feel that this is going to replace your job or replace you, you're going to have a very different attitude than if you think it's a lever. And this also gets to, how do you measure productivity and how you reward people in this age of AI? I think we're moving beyond token maxing.

Erin Doyle: Totally.

Michael Stiefel: This is like the old days when ... How many lines of code did you write? That was a measure of your productivity Of course, when it came to languages like C or C++. It was semicolon, semicolon, semicolon, semicolon. I could produce as many lines as you want.

Erin Doyle: Yes.

Michael Stiefel: How do you measure the worth of humans? Is it team productivity and everybody's on the team? Is it individual productivity? Because we still tend to reward individuals as opposed to teams. How do you cope with that?

Erin Doyle: No, I think you're right. It's becoming even more of a human problem and a human solution than I think a lot of people think it is right now. We hear a lot of conversations, or at least we were hearing a lot of conversations about, what tools are you using? What skills are you using? What rules are you using? It was very technically focused, and I don't think the solution is, what tools or how are you using them? I think it's 100% psychological safety and trust. Does your team have that? And without that, I don't think you're going to be able to enable them to really unlock the full potential of what AI has to offer.

This is something that we went through on our team, that I think was a really interesting experience. So I'd say when I joined the team, I was coming from a place ... this was a year and a half or so ago. I'd say AI was still getting kicked off and I saw AI as a cheat or a crutch. Or like, "I only need to use it if I don't know what I'm doing and I know what I'm doing, so I don't need it". And when I would try it, the results were often bad because I didn't know proper prompting techniques and what it was really ready for and what it wasn't ready for prime time yet. Which use cases were good, which weren't. I was definitely behind the curve.

So I had this apprehension, I had this imposter syndrome about it. Like, "If I use it, does that mean I can't do what I'm using it for?" I didn't want people to judge me. And then I started to have that fear of, "Well, everybody's expected to use it now. Companies are mandating it. Interview cycles are expecting to see AI fluency. I may not be marketable if I don't get on board". So I started to shift my mindset, but it was still from a place of negativity. It was the stick versus the carrot. I'm going to use AI because I have to or else I'm going to be left behind.

And so I was in that place of, "I don't want to ask for help because I don't want people to know I'm behind". Or, "I don't want to ask what other people are doing that's working well because I don't want them to know that I don't know". So it became really hard for me to get caught up in that environment. Anyway, fast-forward a little bit, I joined the team I'm on now and we from the get-go shared our feelings about where things are at, how we've been using AI so far, are we having success? Things like that. And we found we were all kind of in about the same area. We felt about the same about things. We were still really cautiously skeptical. We weren't seeing the results that the hype was claiming to see, but we also were like, "Maybe this could be really great for us. We're an early stage startup. We need to move quickly. If we can figure this out, this could be really good for us and we can also share this with our customers".

That's a lot of what we do is we want to share how engineers can be more productive. So we shifted our framing from a place of fear and negativity, of like, "I'm going to lose my job". To curiosity and like, "Let's lean into this as a team. We're all a little bit uncomfortable, but let's do it together". And so from then on, we had this psychological safety of being able to share what was working, what wasn't working. We were able to commiserate. Like, "Oh, I spent all this time working on this prompt and it ended up giving me garbage. I feel like I wasted time". And we could kind of laugh about it or someone would be like, "Well, what did you try? I've tried this. I got some success".

And we were able to share and learn a ton from each other because we had that trust and safety that no one was going to be judged, no one was going to be punished for being behind. And so we saw a huge spike in our usage. And like I shared earlier, we've spread out across the SDLC. We're now using AI to help us with all of our tasks, no matter what domain it falls into. And so I think that's critical. A team has to have psychological safety and trust to be able to share what's not working, to be able to ask questions and to be able to share what is working without being afraid of making other people feel behind. I think without that, teams are not going to be successful. We're going to have that increasing divide between those that are on board and those that aren't.

And I think that if managers or engineering leaders approach this from more of a push, more of a mandate, more of a, "This is technical. We're looking at your usage, we're looking at your tokens". Whatever it may be, they're just creating more fear and making the problem worse. That's why I think you've got to look at trust and psychological safety before you look at usage metrics.

The Future of Engineering Education [24:40]

Michael Stiefel: Do you think there's a generational divide here? I mean, I remember when the defense establishment, maybe a decade or two ago, let go a lot of engineers. And I remember when I was working with a startup at the time, I interviewed some of them and they just didn't get it. For example, very specification heavy, they didn't have a great understanding of engineering skills, they just did not fit in and the way they looked at the world did not enable them to see the problems that especially a startup was facing. And that's on one side.

And on the other side, you have the question of, how are you going to educate software engineers now? On one hand, we read about people who are hesitant to go into software engineering. Do we have to change software engineering education to emphasize things that it traditionally had not taught, like the SDLC or things that reflect the reality of how software engineering is practiced as opposed to the theoretical concepts?

Erin Doyle: Well, I think what we're dealing with right now is not quite like anything we've dealt with before, but like you pointed out earlier, assembly and the compiler. There are analogs to the evolution of software engineering or analogs across general technology. So we have been through these kinds of evolutions before. Your example and the move from lower level assembly level languages to higher level languages, like C++, Java, et cetera. I think that's maybe a good analogy, a little bit.

When I went to school, I still learned about organizational theory or computer organization. I learned assembly, I learned compiler theory, though I'd never need to use those. At that time, you did not need to use those anymore, but we learned the theory, the foundation of the things that we were building upon. And I found that that was still useful. We could definitely argue whether theoretical foundation is useful to the job or not, but that's what the curriculum was and I thought it made sense. But we still learned the higher level languages too.

And I see where we're going right now as another evolution of a higher level language. We're using English or whatever, our human spoken language is to now write code or to write a lot of things. I think we still want to understand the foundation and theory of computer science, but also, how do we generate results, whether it's code, whether it's documentation, whether it's images, video, whatever? How do we generate output using human language, spoken language?

So I think we need to do both now. I think you're right. We definitely need to start teaching more practical skills in college. And there was a survey I think run pretty recently that showed the outlook of people in our industry today. The percentage that would recommend a high school kid to go into our field. The numbers were just dismal, and I feel that way. I don't know what this job is going to look like in another even year or two. Definitely it's generational. I think it impacts demographics. I think the more susceptible you are to imposter syndrome, it can impact because again, my earlier feeling was that if I say I've used AI to do something, does that mean that people will now doubt my ability to do that thing?

It took a while for me to get used to using it on things that I knew how to do, but I had to accept. And again, team alignment comes in here. My team had already aligned on the expectation that we want to be fast. We want to use AI to improve our output. And so if I have this thing that's going to help me do my job faster and maybe a little bit better, I should use it even if I know how to do that work. So it's not saying I can't do it. I didn't know how to do it. I had Claude do it for me. It's not that, but I certainly felt that way at first, and I think a lot of people still do.

It definitely impacts the people who are a little less confident in their abilities or a little worried about people judging them. And so I think that really impacts people who are maybe a little older and it impacts people that are a lot younger and they're not sure, when do I use it? Where is it okay to use? Where should I be writing code? Now I don't even understand this output that I'm getting and I can't validate it and I don't have confidence in it and I can't explain it to my team. So that's a really hard place to be.

Ambiguity is The Engineering Issue for AI [29:48]

Michael Stiefel: Well, one of the things that struck me as I was thinking about what you were saying, and especially when it comes to education. I like to tell people ... When people ask me what my software career was like. I tell them my entire software career, 35 or 40 years I've spent in this field. I've only done three things, trade off space and time, insert levels of indirection, and trying to get my clients to tell me what they really want.

Erin Doyle: Yes.

Michael Stiefel: And in the age of AI, maybe the AI takes care of the first two, but this idea of getting clients to tell you what they want and dealing with the ambiguity of that seems to be a skill that becomes even more important and should be taught how to deal with this. I'll give you an example. When we used to write, whether it was Java, C++, assembly language, you knew the computer was stupid. So you had to tell the computer exactly what you wanted. And if you were ambiguous, you got a bug report and it got fed back and you fixed it. So that raises two questions right away.

One, if we're dealing with the English language or any language for that matter. English, French, Spanish, Russian, Chinese, and an LLM, you're dealing with ambiguity all over the place. And then on the other side, when you get reports from the field, how do you factor that back in? How do you tell the AI, there's this bug or this feature that we want? Do you have to rebuild the whole code base from scratch? Can you do incremental change? As I think about this, the issue of ambiguity gets raised to a first class problem when you're dealing with AI.

Erin Doyle: I would agree with that. I feel very much lately like a big chunk of my job now is how to provide the right context and instructions to get good results. Even though that's gotten a lot easier, we've got really good skills that kind of allow us to repeat those instructions without having to type them out every time.

Michael Stiefel: For the people who may not know what a skill is, could you just give them an example of a skill?

Erin Doyle: Sure. And the evolution is, first we just had prompts and we had to say over and over again, "Make sure you write the tests and make sure they pass". We'd have to give those instructions every time or else it might not happen. And so then we had rules, then you could dump a bunch of those instructions in one place. And now we have skills, which allow us to tailor very specific behaviors or sets of knowledge. It's really just simple human language written in a Markdown file that maybe has some tips for ... If the human says these words, use this skill. Or if they give you this explicit command, use this skill.

I compare it to in The Matrix, when Neo learns kung fu. Suddenly he's like, "Whoa, I know kung fu". A skill is like teaching the LMM kung fu, when it needs it. It doesn't have to always know it, but when it needs it can learn it really fast. And so suddenly they've got way more knowledge and capability than they had before without us having to give all that instruction every single time.

Michael Stiefel: And presumably there were also scripts that provide sometimes the explicit way to do those skills.

Erin Doyle: Yes. If we look back a year ago or so, the agents were maybe junior engineers. We had to be super prescriptive. This is exactly what I want you to do. And now we're getting to the point where we can be a little more lazy and vague and give it, "Hey, go implement this ticket". And it can go out and get all the details. It can look at all these different sources and it kind of knows how to do good software engineering if you use the right skills and whatnot.

Now I feel like we're working with maybe senior engineers that you can just talk to as a peer. But zooming back on your original question, talking to humans has always been hard. That has always been a hard part of our job. And it's the people that are really good at communicating complex concepts to other humans that tend to find themselves being successful and moving up the ladder. People who can't communicate well struggle. So even though we deal with computers, we've always had to be good at communicating with humans. But I agree with you. I think that's so much of what we're doing now is making sure that what's going in is really clear and what's coming out, we can still communicate to our team members about these concepts.

We Must Invest in Developing Junior Engineers [34:59]

Michael Stiefel: You said something very interesting. First, it was like you're talking to a junior engineer and now it's like you're talking to a senior engineer. But where are all the junior engineers going to come from? It's clear from what you're saying that if the AI is like one senior engineer talking to another senior engineer, if you're already a senior engineer, fine, if you're in the business today. But how a business is going to train your junior engineers if all the tasks that the junior engineers used to do are now done by the AI? I've asked this question to so many people. In fact, I have a podcast coming up, which is devoted to this issue and no one has been able to give me a very good answer.

Erin Doyle: I think we're in a really tough time, where no one has that answer or maybe there's a few people and we need to hear it from them. I'm putting a plug out. If you think you have an answer, please share. We need to start talking about this. We need to start thinking about it. But I feel like right now we're in this slippery slope. Again, those of us that have gotten on the bandwagon, we can see what AI can enable us to do and it can be pretty amazing. The thought of taking that away, now at this point. I think scares a lot of people.

If the internet goes down or Claude's having an outage, people are like, "Oh, my God. I can't work anymore". So we've become dependent on it. We've already shifted how we do work and so we've become dependent. It's almost like a drug. It is enabling us to do so much more than we could before that we are dependent on it and therefore our organizations are expecting that from us. They want that speed unlock. They want to get more out of us for less money.

And so I'm not sure how we incentivize organizations to take the time to invest in junior engineers, to bring them in, to continue to give them the handholding and care and mentoring that we used to give them. That used to be a worthwhile investment. But now it's really tricky to make that argument.

Michael Stiefel: It's like we're eating our seed corn.

Erin Doyle: Yes. What the future looks like right now is kind of scary, not just the environmental impacts, but what does our industry look like if we run out of people to do these jobs?

Michael Stiefel: Especially if you're telling people don't go into the industry.

Erin Doyle: Right.

Michael Stiefel: It's a doom loop.

Erin Doyle: It really is. I hate to say I don't have an answer. I wish I did, but I think it's going to take a little bit of forward-thinking and kind of stepping away from the shiny object of like, "Oh, we could do more with less". Making some bandwidth to continue the pipeline of engineers and continue to make space for training and mentoring and doing human work.

We Are Unprepared for the Software Development Future [38:00]

Michael Stiefel: I have found this a fascinating conversation and I have so many more questions. I'll try to just have a couple more before I want to get to the architect's questionnaire because I don't want to burden you, but I find this discussion fascinating. It's a discussion that does not get emphasized enough because it seems we have two sorts of poles that this conversation has gone between.

It's one about the AI journey for teams and they get on the teams, you create an environment, psychological safety, and you can do so much. You can use AI in all parts of the SDLC. You don't have to rely on ritual to do what you actually can do now. On the other hand, you're also somehow reinforcing the AI skeptics because we look and say, "Where are the engineers going to come from? How are we going to deal with ambiguity?" Because traditionally, software engineers went into software engineering because they couldn't deal with people, because they like dealing with inanimate machines, and this is not what it is today. So we seem to be oscillating between these two perspectives and it seems almost like we have to keep this sort of bifurcated mind going for at least a little while.

Erin Doyle: Yes, we absolutely do. I wish I had some contrary thing to point out. I feel like the future is still very uncertain. We don't know where it's going to go. It may be okay. It may not be okay. I guess where I'm at is I'm trying to kind of focus on the present. I told you my journey, I started as a skeptic. I did not want to buy in. I didn't want my job to change. I love coding. That is something I've always loved about this job. Love solving problems with code and that's going away and I don't write code much directly anymore. My job has absolutely shifted already.

And if you had told me two years ago this is what it was going to look like, I'd be pretty upset about it, but the industry is doing what it's doing. I don't have control over it. So I have to decide how I want to shift my thinking and my perspective. Like I told you, when I shifted from the fear and negativity to the, "Well, what good can we get out of this? Where is the positive?" That really got me excited about things. And now my day-to-day has been pretty fun. I feel like things are changing so fast that I'm constantly having to keep my eye on things and learn and I'm trying new tools every week. I'm sharing with my team. We're constantly having little victories of like, "Hey, I tried this new thing and it worked great".

So there's a lot of positive there. We definitely do need to be responsible to still step out and we as a community need to start talking about the hard problems. What we're going to do about the environmental impact, what we're going to do about the pipeline impact, how this is changing our jobs. I want to find the silver lining of like, "What are the good things that we're getting out of it? How's it changing our jobs in a good way?" And maybe it won't be as terrible and scary as we think it is. Maybe just like when we moved from assembly to C, to C++, it wasn't a bad thing. So I think it remains to be seen.

The Need For New Tools [41:34]

Michael Stiefel: One of the things that I think could possibly help us out if we develop certain tools that help us navigate. I'll give you one example. I remember when I encountered my first symbolic debugger. I mean, I'm old enough to remember when there were no symbolic debuggers and you looked at hex dumps of code to figure out what was going on. But what happened with symbolic debuggers, it helped me understand what the software was actually doing. I mean, again, with multithreaded programs, it was a little harder, but I could go through the program step by step and see what it was doing.

So perhaps we need some software that helps us understand. I know there's a certain amount of active research here because for certain regulatory reasons such as automatic vehicles or ... Regulators will just not let technology in the world, unless you can explain why it did what it did. So maybe there will be the equivalent of debuggers that will help us understand what the AI is doing, what the agent wants to do. There's been research done that human reviewers are not very good at finding out where AI sabotaged the code or where it was making mistakes.

So maybe somehow if there's some technology that allowed us to monitor what was going on in some way, advanced telemetry. I mean, I'm just throwing ideas out there based on sort of ... Continuing the analogy that what made the higher level languages more bearable is we figured out how to do telemetry. We figured out how to have symbolic debuggers. We figured out how to do postmortem analyses of things. If we can end on a hopeful note, this is perhaps where the industry needs to go because if it doesn't, the outside world will punish us.

One of the most amazing things to me is that our industry has escaped the type of financial liability that other industries go through. There has never been a serious suit against any software or hardware firm that really was catastrophic, unlike you might see in the airline industry or some other environmental impact industries, but this will happen if we do not figure out how to do these tools.

Erin Doyle: Yes, I think you are absolutely right. I do think that's where we need to go and I think that's where we're going. It's naturally evolving. When we look back just a year ago, we have way more verification and validation tooling. Right now we are having to kind of patch all these tools together as they come out. We're having to stay on top of what's out, what's available. Oh, wow. This new tool just came out. It can help me do this.

Michael Stiefel: Could you give some examples of the tools you're talking about?

Erin Doyle: Sure. We use CodeRabbit for our PRs. We've tried a number of different tools before landing on that one. A lot of the services we were already using are now rolling out chatbots, which are experts in that system. So instead of us having to be an expert in the domain, we can now ask that chatbot for help. So for instance, we use Sentry and Sentry has a chatbot named Seer. And so I can now ask Sentry, "I'm seeing these values here, but I'm not getting these alerts here, what am I missing?" Or maybe I set up the integration wrong or maybe I just need help analyzing my data. I can now go to that chatbot and get help with that. Linear, there's a lot of connections with Slack.

So a lot of these tools are making it way easier and quicker to connect things, to analyze things, to display things. So it's making it much easier for us to wrap our arms around all this nebulous AI output. The next step is how do we make it easier and more accurate for humans to review, use, understand the various things that are being AI generated? That was actually another thing that my team went through on their AI journey. We were starting to produce a lot more code faster. We were putting up more PRs faster, but they weren't getting through the review process faster. And so we had this new bottleneck.

We talked about it and we asked ... What did each of us think humans were especially good at reviewing for or what we should be looking for? What did we think AI tools, such as CodeRabbit are really good at spotting and what do we feel like our responsibilities should be as human reviewers and what should we leave up to the AI? And we found that we all were thinking different things. So each of us was approaching the PR as a reviewer with the different expectation of what their job was. And some of us were doing work that maybe at this point now, that the AI tools are better at. They're better at finding certain bugs, certain edge cases, certain quality issues, security issues. There's a lot of things now that they're really good at doing that we as humans will miss or it'll take us a lot longer to find.

So by just adjusting like, "Hey, we don't need to do that anymore, but we can focus more on the business domain. Do we understand the code? Is it going to be maintainable? Does it follow our standards and patterns?" We can look at different things now. So I think the more stuff that makes it easy to verify and create trust then frees us up to do more complicated work.

The Architect's Questionnaire [47:53]

Michael Stiefel: Well, as I said, this conversation can go on for a lot longer, but I'd like to get to the architect's questionnaire to sort of make it a little more human or get more perspective on what it means to be involved with the technology we all both love and hate at the same time. What is your favorite part of being an architect?

Erin Doyle: I love being presented with a big ambiguous problem that has to be broken down and analyzed for what's the best fit solution. And usually that requires digging in and learning more about what that thing is or that domain, and I always love learning. So in order to be an architect, I feel like you've got to be constantly deep diving and learning, and that's what makes it fun.

Michael Stiefel: What is your least favorite part of being an architect?

Erin Doyle: I think for me, the thing that I have the most trouble with is deciding between trade-offs. I mean, that's such a huge part of the role. My inclination is to always over optimize or over-engineer and finding the right fit, finding a solution or an option that's sufficient versus perfect. Knowing what's good enough for our needs today, being okay with the fact that that solution might not scale, that's hard for me. So that's something I'm working on.

Michael Stiefel: Is there anything creatively, spiritually, or emotionally compelling about architecture or being an architect?

Erin Doyle: I think so. When I'm designing solutions, I'm not just thinking about like, "Does it solve the technical problem?" I'm very much thinking about, "How does this solution look to developers? Is this going to be easy to use? Are we creating any kind of footguns for ourselves in the future? Are these abstractions going to cause more harm than good? Is this going to make people's lives better or is it going to be more painful, more hard to understand, more hard to maintain?" I'm always thinking of the human aspect of the architecture, so I feel like it's very emotionally compelling.

Michael Stiefel: What turns you off about architecture or being an architect?

Erin Doyle: I find the role can be kind of isolating and isn't often as collaborative as I think it should be. You're often asked to go off and come up with a design for something, and then you have to thoroughly document and get buy-in or consensus on what you're proposing. We talked earlier, it can be hard to clearly communicate complicated things to make sure all aspects are documented, the considerations, the trade-offs, everything needs to be covered. And to be able to adequately communicate that to other people is hard. But it can also be really hard to motivate people to take the time to review technical designs and proposals thoroughly. People are busy, they've got their own stuff to do.

So it's like you have to somehow motivate, incentivize people to collaborate with you, which can be really difficult. Or oftentimes you'll end up with a lot of critical feedback that's very one-sided. Like, "I have a problem with this". Or, "How are you going to handle that?" Instead of, "Let's work together on a solution for this thing". Or, "I think you could do this". It's very like, "You didn't get it right. You go back, figure it out, and come back to me". Versus, "Let's work on this together".

Michael Stiefel: Do you have any favorite technologies?

Erin Doyle: I've always been a huge fan of JavaScript. It's my favorite language. I know it's contentious. It's not super popular in some camps, but I've always loved it. I love TypeScript and what it gives us. I love that we can now do full stack JavaScript or TypeScript, and I love AWS as a platform. I try to do as much as I can there. Those are kind of my loves.

Michael Stiefel: What about architecture do you love?

Erin Doyle: I love how a good architecture should make people's lives easier, make their jobs easier, makes the product better. To me, it's not just about enabling a feature or functionality. Of course, that's the core of what you're building, but I really like how ... When done well, it can really make everybody's lives easier.

Michael Stiefel: What about architecture do you hate?

Erin Doyle: I hate that more likely than not, every decision that might have been the right fit at the time it was made will no longer be at some point in the future. It feels like a decision was bad if it doesn't scale or it runs into unforeseen problems at some point. I think that's a mistake. We think architecture is set and forget and it should be perfect for years, but there's just no way it can be. So I really wish we had a little more patience for architectures that rightfully should have to evolve over time.

Michael Stiefel: What profession other than being an architect would you like to attempt?

Erin Doyle: I mean, I definitely love what I do today, even though ... Again, I wouldn't call myself an architect, but someone who architects, but I've also been really interested by the SRE field. I think field CTO is a really cool role that's kind of getting to be more common. And if AI takes all of our jobs, I'm going to go be a barista.

Michael Stiefel: Do you ever see yourself not being an architect anymore?

Erin Doyle: I think as long as I am in software engineering, it's always going to be a hat I wear. That's just a responsibility and role that I see myself always having, and I'm okay with that.

Michael Stiefel: When a project is done, what do you like to hear from the clients or your team?

Erin Doyle: I love to hear, "Oh, this experience is so much easier or nicer or smoother or faster". That I've been able to make a thing better for people, not just that there's a new thing they can do, but that the experience has been improved.

Michael Stiefel: Well, I enjoyed this conversation very, very much. I hope we can do it again sometime. It was a pleasure talking to you.

Erin Doyle: Yes, thanks so much for having me. I really had fun.

Mentioned:

• The Wall Street Journal on AI liability and insurance policies
• CMMI (Capability Maturity Model Integration)
• Rational Rose (by IBM)
• Scrum
• Kanban
• Quotient
• Recent survey on the attitudes of software developers
• CodeRabbit
• Sentry and Seer
• Linear