Join us at our first in-person conference on June 25 all about AI Quality: https://www.aiqualityconference.com/(url)
Dave Nunez 00:00:00:I'm Dave Nunez. I'm co founder of Abstract Group and a partner at Stellar Docs, and I love Cold Brew, making my own cold brew.
Demetrios 00:00:13:Welcome back to another MLOps Community Podcast. I am your host, Demetrios, and today I have the pleasure of having a co-host. We're bringing it back to the old times. I brought on my man Kuba because he is someone that I very much respect when it comes to the developer space, and I tend to ask him a lot of questions around documentation. And our guest today, Dave, we geeked out for about an hour about documentation and really what makes good documentation, why you want to strive for good documentation, and how to continuously update that. So you had some fun takeaways. Kuba, what were the things that you thought Dave said that you particularly enjoyed?
Jakub Czakon 00:01:02:Yeah, so I love this concept of thinking about time to first value, right, and having that North Star and then optimizing it, but ultimately pushing people to get to that value, get to that first dopamine hit. This is a really good metric to optimize and to think about.
Demetrios 00:01:22:Yeah, it's funny because I know that in the community we had a thread that was happening in slack that said something along the lines of don't conflate users and clients. It's not always the same. And somebody wrote underneath that clients are the ones who pay for the product, users are the ones who suffer from the product. And so hopefully because of this conversation with Dave, that is not going to be commonplace. If you listen to this, if you listen to the next hour that we talked to him, I mean, he was given value bombs after value bombs. In my way of seeing it, I really enjoyed just what he understands. It's almost like a second nature to him. And it really feels like he's been steeped in docs for whatever, a decade, two decades.
Demetrios 00:02:16:I don't know exactly how long, but it is obvious that he has woken up every day and thought about how to make docs better. And so I love getting to speak with somebody at that caliber and just learning from him. He talked all about these design cues that the docs can have and make sure. One thing that I thought was particularly cool was this idea of an escape hatch in your docks, just to make sure that the person is in the right spot. So when you're listening, let your ears perk up during that part when he starts to talk about escape hatches, and I think that's it. Kuba the main thing I want to say now is if anybody is listening and they have not subscribed or liked or given some reviews on Spotify. It would mean the world to us if you did that. And as always, just tell one friend and we are in business, baby.
Demetrios 00:03:15:Let's keep it rocking. We got to start with this, man. Like, you spent so much time at stripe. What are good dogs, what are bad dogs and how do we intuitively feel that?
Dave Nunez 00:03:36:Yeah, yeah, it's good that you said intuitively feel because I spend a lot of time talking about the first impression of a doc and just that, like subconscious feeling a developer gets when you're like, all right, I'm already upset that I got to go look at the documentation because I'm blocked on something or I have 30 minutes to try and see if I can set up this thing. It's my first time using this product. I need to go use the docs. And you know that feeling when you land on the docs, you don't even know where to go. Like there's no, there are no guideposts. Like, you don't know what to search for because you don't know the terminology yet. Or you click on the quick start, like get started in five minutes and then it's just like ten paragraphs of text. That is an awful first impression and it's probably likely the developer is going to be like, yeah, let me check my slack messages or I need a cup of coffee.
Dave Nunez 00:04:23:And the opposite of that is something that's inviting, depending on what you're trying to do. Like, developers, uh, also subconsciously knows what a doc looks like, what, depending on what they're trying to do. If they're trying to understand a platform product and the architectural pieces, if they land on a nice looking diagram where they don't have to read all these things, they can just see how the data moves and how the pieces fit together. Like, you're already inviting them in. Building that trust. If it's a quick start and you see one line of code, like, or you see like a, one sentence, like, here's what you're going to accomplish in this document. And then you start seeing some, like several lines of code. Like, that's pattern matching to them, simplicity.
Dave Nunez 00:05:09:And like, I want to learn more. So the documentation shouldn't be about like solving all of their problems all at once and being perfect across the board. It should be like, how can you build that trust and keep getting them to go down that path? So ultimately good docs have, like, if you don't have a good first impression, you're already behind with your user and then once you invite them in, the writing needs to be clear, it needs to be written for, and it's sad that it's not obvious, but like, it needs to be written for a human. A lot of times, companies with a lot of resources, they're mostly writing for their translation tools and they're thinking about the cost of translation. And how can you write for basically a robot that can turn this string of text into a lot of different languages, instead of taking on that complexity, writing in a more localized language and then dealing with, even if it's manual? I'm sure now with LLM tools it's a lot easier to do this well, but that's a user first example, and then just all the little elements. Is the writing good? Are there callouts, are there the right balance of inline links? Putting yourself in the shoes of the user? And what are they, what are they actually trying to do, and how much time do they have? And then bad docs are the opposite. I'm writing for me, I think that, or here's everything that the tool can do and let me just put it onto the page. Not worry about readability, not worry about how much time the user has or like what are the actual, what's the actual format or presentation style like they are going to want at this point in time.
Jakub Czakon 00:06:50:So maybe let's double click on it. I'm wondering. So two things. One is what are those signals, good signals for the dev landing on the docks. You mentioned diagrams, you mentioned the code. What are some of the other things that you think are very inviting to the dev? And then the second question, how do you sort of like sanity check what you said that you didn't write it for yourself.
Dave Nunez 00:07:19:Yeah, the signals. I think whitespace is a good signal. Like hey, this is an approachable piece of documentation, no matter what it is. You don't want to see blocks of text, you don't want to see a video and a diagram and a ton of visuals in your face code. It should be very clear where you want the user to start and what you want them to see. And then after that it should also be scannable. So that's where that white space is, because nobody is like cool, I'm going to crack my knuckles, sit down and read this document. It's like I'm, and it's called, you know, information foraging.
Dave Nunez 00:07:49:I'm looking for the truffles of good information and then I'll even like backtrack. Like there are a lot of eye tracking studies that have shown when users are looking at documentation. It's an f pattern. Like, I'm just going boom, boom, down, boom, like, across. I'm not reading it left to right all the way down. So, so the white space is really important. And then every sentence should be, should mostly stand on its own. Like, you see, a lot of good developers write three line sentences and you have to reread it several times in order to understand, like, what you're, what they're trying to tell you.
Dave Nunez 00:08:24:Um, but if you're very clear in, like, here's what this thing is, and you're clear about the sequencing, like, here's how this feeds into the next thing, I think those are, those are clear signals. A lot of it is subconscious, too, because, again, like, developers, especially 510 years ago, admittedly hated documentation. Um, and, you know, justifiably so they hated documentation or reading documentation they didn't want to be marketed to. But those things are still important. So how can you deliver information in a way that they're not frustrated, you don't need to make them happy. They're not going to be like, this is the best documentation I've ever read. Like, let me print this out. You just don't want to frustrate them.
Dave Nunez 00:09:03:Um, and then the sanity check. Uh, I mean, the, there, there's a lot of qualitative things, things like, one of the best things you can do. It's really hard to edit yourself. Like, when you write a piece of documentation, and then you go right back to the top and try and edit, you can clean up some stuff, but to get it really pared down and much cleaner, you need to give it a day, sleep on it, come back to it with fresh eyes. You'll forget a lot of what you wrote and you'll see, oh, this is bad. This doesn't make sense. This can be removed. And then another good one, too, is you give it to whoever the audience type is going to be a coworker and say, here, read this document.
Dave Nunez 00:09:48:Now, what were your takeaways? And think in your head, okay, I want them to learn this concept, or I want them to accomplish this task, or I want them to know what steps are required for this particular user journey. And so ask them, like, okay, what are your takeaways? And if they're not clear and what those things are, what those goals are, then keep iterating until you nail that. So that is better than nothing. We did a lot of user research where you're asking 510 users. What do you think about this guide? But a lot of times just asking a colleague, hey, what were your takeaways from this? And then just optimizing for what you want them to take away.
Demetrios 00:10:36:So I know stripe has a very strong writing culture, right. And it's almost like part of being indoctrinated into stripe as an employee. You have to write a lot of things down. Do you feel like that plays into how stripe manifested its docs and your ability to be able to go to just anyone that you're working right next to and say, what do you think of this? Where can it be more clear what could be better on that?
Dave Nunez 00:11:08:Yeah, absolutely. The writing culture played a huge part, and I'm trying to think if people, like, if it was self selecting. I think a lot of it was self selecting. People knew there was a writing culture. You were going to have some form of writing during your interview loop. I mean, people are analyzing, the hiring managers are looking at your emails, like when you're just corresponding and gauging your kind of, you know, intelligence and ability to convey thoughts, like, in all of your written correspondence, in your resume. And so there's some filtering up front, and then there's some reinforcement on the job, the career ladders themselves. Like, documentation was included in the engineering job ladder.
Dave Nunez 00:11:57:And the higher you go up, the more. The higher the expectation of your, of your writing. Like, you needed to be able to write a really good design proposal, architecture proposal when you're a staff engineer or above. But also, I met a lot of people that did not like writing. Like, they knew it was important and they played along, but that wasn't their thing. They were the people that liked the multiple choice tests in school. Like, hated the essay portion. So it was important to, like, when you're, when you're working with all those people, not assume everybody likes to write, but make it so there's a high floor for those people who are like, I just want to get the job done.
Dave Nunez 00:12:37:I just want to review or contribute my thing and then go on with my life.
Jakub Czakon 00:12:42:That is the reality, you know? Like, the writing culture is not very prevalent on the engineering teams all across the board. And I'm wondering, like, how do you manage those, those staff relationships? How you get more input and collaboration with, with engineers and, yeah, how do you kindle that culture?
Dave Nunez 00:13:04:I think it has to be, it has to start with the leadership, because if everybody, if all the developers care about docs, but the leaders are writing really terrible emails, like, that just seeps into the culture immediately, or this would happen a lot, too. This is a big signal that docs don't matter. You have a release and you picked an arbitrary date because most software deadlines are and the docs aren't ready and you ship it anyway. But if you say, hey, the docs aren't ready, we're not. We're holding back this release until they're ready, that's a huge signal that it matters. And we, we had to gate or take into account docs for every single release. At stripe, there's you usually a fire drill at the end to get them ready. But the product would like our CTO, our CEO, our CPO would be reviewing docs like before, before launch.
Dave Nunez 00:14:00:So those were, those are great signals. So the leaders themselves like showing that they care about their, right, their own writing and then also making it an equal factor in your launches.
Jakub Czakon 00:14:15:I'm wondering sort of what changes can you do bottom up when maybe the leadership isn't bad about writing, but they are not necessarily great, or they don't necessarily care about it as much as folks at stripe did. How do you make a change coming from the bottom up as a technical writer?
Dave Nunez 00:14:37:Yeah. With the bottom up, the company should have that component in the hiring process where everybody's doing some. I mean, I can think of every single role in stripe that I know of that had some type of project or take home component to the interview process. So you're bringing in people where you're gauging their writing. They don't need to be excellent writers. They need to be passable, and they need to know that writing is important. And then once, once they're actually in the door, make sure that come performance review time, like that stuff actually matters. Like, hey, you're being praised for the work that you've done sharing knowledge, or you're being penalized because you have not shared any knowledge or you're, you're not taking your writing seriously.
Dave Nunez 00:15:19:Um, and then what a technical, like the professional writer. So the, the first thing that I did when I joined Stripe and I did this late at Uber, but it was like, seriously, the, the first week at at stripe, I threw up the writing bat signal and like figured who all the professional writers were across. Um, the, and just, we got together and met bi weekly to share our resources, our relationships, our knowledge. And it's just really helpful to kind of create that co op because the marketing writers, the brand writers, need the same things that you do. Relationships to the technical side, the product managers, style guides. You don't want to have the documentation to have a certain style, and then the marketing site to have a certain style and support site to have a certain style. So coming up with those core artifacts and then sharing those out with the company, and that makes it easier for the engineers or the non writers to know too. Okay.
Dave Nunez 00:16:13:There's a unified set of guidelines I need to follow as I'm writing. So I think that's the base that you can do is like get your writers to agree upon, like the style and the, the writing expectations. And then after that, just keep giving as many resources as you can. Like, we ended up creating like an internal, basically like a self guided book for engineers to, because we couldn't serve everybody on creating their docs, and they were, it was basically like how to create docs, like a striped technical writer. We added to it over time, but it was eventually like an end to end guide with templates and annotations and, you know, checklists, and they could follow that process for their own, like self serve. That took years to accumulate. But it's something that if I were to join a company, I would, that's something I would start from the beginning, not like three years in.
Demetrios 00:17:06:Oh, fascinating. That's super high leverage. It's the docs on how to create docs and meta. But it is very, it makes complete sense on why that could be so useful. And you're empowering, you're decentralizing the ability to create docs.
Dave Nunez 00:17:24:So one more thought on that too, because this is really useful is everybody says they want templates, but templates alone kind of suck when you're talking about documentation.
Demetrios 00:17:32:Cool.
Dave Nunez 00:17:33:I have like four headers. I'm still, I still don't know what to do. So the next iteration of that is you annotate the templates of. Okay, in this section, here's generally what you want to put. Maybe you link to an example, but the, the best thing that we found is you create an example of every content type, let's say like three content types, an overview guide, a quick start and a how to guide. And instead of doing like templates on all these things that are fairly blank, you say the writing team is going to give you an example of a very good overview guide, a very good how to guide, and a very good quick start. And maybe you add some context of why we made these decisions, what goes in here. But that's, I think, the most useful for engineers to pattern match because they can say, all right, I'm doing this thing.
Dave Nunez 00:18:23:It's different, but I'm going to follow the same structure, I'm going to fall the same length and, you know, make sure I'm going into the same level.
Demetrios 00:18:30:Of detail when it comes to almost like, creating a movement around docs and having what I mentioned before we hit record, like a cult like following with stripe and the docs. I haven't heard people say that they like reading docs that much. That's not, as you mentioned, it's not really a pastime that engineers enjoy. However, whenever I ask someone, show me some examples of good docs, nine times out of ten, stripes docs get bundled into that. And I like how you were talking about, like, what not to do. It almost reminds me of the Charlie Munger thing on, like, I don't know what I want, but I know what I don't want. So let's start there and then let's go. And by knowing what I don't want, it'll probably get me closer to what I do want.
Demetrios 00:19:21:And so there's all kinds of ways that you can make sure you don't just do the obvious faux pas and the obvious things that are going to mess you up and make people even more painfully or more irritated when it comes to using your product. But are there things that you feel like you haven't mentioned so far that helped the user experience and the developer experience at Stripe really thrive?
Dave Nunez 00:19:51:Yeah, I think the mindset, because you're saying like a cult following, this started for me back at Uber, where you could write an RFC and get it approved if you, you got a couple yeses from some important people and nobody like, threw you under the bus, and then you got like unlimited headcount to do what you wanted to do. But I had an RFC out for creating a unified platform. We were using confluence at the time. I wanted to kill confluence, create our own thing from the ground up. It was like a really ambitious, big, risky project. And I had all of my data, like, here's how it would be cheaper, here's how it would be a better user experience. Here's how we've de risked it. And I was still getting some pushback.
Dave Nunez 00:20:40:And my mentor, who is an engineering director, was like, you're not going to win this argument with logic because it's so against what the norm is for everybody, and it's against what the norm is for leadership to think like, wait, why do we need an internal content platform for engineers? Why do we need to build that ourselves? And he told me, you're basically running a campaign. Everybody's wondering when you're trying to convince them to do something that they don't want to do what's in it for me? And so I was asking, like, what's a leadership book? Like, I read all the books on Apple. Like, what can I do to be more persuasive? And we had Barack Obama's old or former campaign manager working for Uber at the time named David Plouffe. And he's like, you should read his book and see how to win a campaign against all odds. And so it was, like, very grandiose to think of internal docs as, like, a historical president, president being elected, but same. Really? Yeah, I think. I think we. Yeah, it's pretty much the same in history in terms of impact.
Demetrios 00:21:45:Now, if you ever meet Barack, you're like, I stand in solidarity with you. I know what it's like.
Dave Nunez 00:21:50:He's probably reading my book and telling his team, like, you got to think about things.
Jakub Czakon 00:21:54:Yeah. Working on a death will start up right now for what I know.
Dave Nunez 00:21:59:Not crypto, that's good. But he. So that was a huge turning point for me. I mean, I was still. I was pretty far into my career already, but thinking about what's in it for me and not thinking about it in terms of jamming logic down humans throats. And so at stripe, a lot of that was the same thing. We'd get mean tweets all the time. Like, this thing is really confusing.
Dave Nunez 00:22:21:And instead of being offended or thinking, like, you're using it wrong, like, we would think, okay, what is different about this person? And so we had a particular user we were struggling with, but was really important, which was like the non technical co founder. And we obsessed over startups. We had obsessed over technical founders, but the non technical co founder was also really important, because then when they're installing a billing system, it's beyond just integrating the code. You have to think about what you want out of this billing system. What are the payment options, the subscriptions you're offering your users? Who is your back of house finance team, and, like, what are their needs? Like, there's a lot of non technical stuff that you need to serve in order for them to use the product. So putting yourself in the shoes of, like, what's in it for? Like, what's in it for me from a user's perspective, like, you just uncover all these little things that add up over time. And there weren't a lot of, like, big bang ships, I can think of it. Stripe with docs of, like, boom, we unveiled this thing, and it just changed your life.
Dave Nunez 00:23:29:It was like, little things over time. Uh, like removing one of the biggest, uh, things was removing inline links. Like, like everybody who cared about SEO, we'd stuff every sentence with an inline link. And then we did user research and saw that the developer would have like 30 tabs open. So we, we, we minimize that by like ten x. And we immediately saw like a better user experience, or we redid how we did diagrams, or we improved search, or we improved the information architecture. So once you start thinking about it from the user's perspective in terms of like, all right, what are the small things they're trying to accomplish and what are the little paper cuts and points of friction you can just smooth over? Because I know for me, when I'm using a product, if somebody adds that extra context that I needed to like get over a hump, I'm super. It's a great feeling.
Dave Nunez 00:24:24:I feel smart, I feel unblocked. And I think those are the feelings that collectively somebody later on is like, oh, I had a great experience and now I feel much different using this other tool where they're not thinking about all of the ergonomics of what I'm trying to accomplish.
Demetrios 00:24:39:Yes, exactly.
Jakub Czakon 00:24:40:I think to that point, to those things you mentioned. Funny that you didn't mention the one that I, I hear a lot about stride docs actually speaking to our engineers as well. They said that one of the things they love the most is that you do the smart injection of those API tokens somehow, or you can test it so you don't have to copy paste it and then everything gets messed up or something like that. It's injected, just takes that one boring, messy step that is like, yeah, you should do it, but, but it does it for you. So it, you know, you immediately feel taken care of.
Dave Nunez 00:25:18:That one we got so much praise for, that was actually done before I joined. But like that, I was so happy that mindset was there and there was so much feedback from users that just reinforced like, well, this small thing that you all did, we're so thrilled about. So how can we recreate that experience? And I remember one time at an all hands, Patrick Colson was like, I want to talk about this really exciting ship in the industry. And he held up his phone and we're like, oh, the iPhone. And he was so excited. Do you remember when Apple, I don't know if you all iPhones, I'm sure Android does this now too, but when you get a two factor auth code and you'd have to go to your text messages, copy and paste it, then go back, and then sometimes the browser would close or whatever. And then it started just showing it in the keyboard, like right above your keyboard and you could just copy and paste it or it would autofill it. He was so psyched about that.
Dave Nunez 00:26:13:And it's something that everybody was just conditioned to deal with and you, everybody hated it, but nobody was thinking like, oh, they should think of a feature. But I'm guessing at some point they did user research and saw like, oh, I got to do three clicks just to go back and do this thing multiple times a week. Like, how can we make that better? So that was the mindset we had, is just like, what are these hidden, unsexy things that will just make it a tiny bit easier for the user?
Jakub Czakon 00:26:41:Speaking of this part of user research, and you mentioned it before, if you have a chance, speak to your devs, ask them questions or do proper user research. I wonder what are your favorite questions or scenarios that you like to run to uncover those things, to find those frictions and find those problems with your docs? How do you like to run this kind of stuff?
Dave Nunez 00:27:04:Yeah, that's a great question. I think the best thing is to get on a call with a user, have them share their screen in the environment they're comfortable with. So if we're trying to, for example, was helping a client recently with a big information architecture overhaul, and they had the challenge, which a lot of companies do, is they had an open source product and a paid product, a hosted product, and they wanted to, like, the hosted product was much better, obviously, and they only made money if people use that one. So they wanted to present the information without neglecting the open source community. It's a really hard information presentation problem. And at the time they were going the opposite way, which is like open source first, and then, oh, by the way, we have this paid product and unsurprisingly, they weren't getting as much traction as they wanted to. The paid product.
Demetrios 00:27:58:It's totally typical.
Dave Nunez 00:27:59:Yeah, it's a hard problem to solve. So when we did user research, we had the user fire up a fresh browser and now go to the docs and we told them what docs to look for, but we didn't say start on this page. It was like, we want to see everything. Are you starting from Google Search? Are you starting from a different browser? Are you going to, or actually we'd even say look for information on this. Maybe they're starting with, with like appending Reddit to their search or going to YouTube. And so once we did eventually bring them to the docs, if they didn't start there. It was the same thing. We'd be like, think out loud.
Dave Nunez 00:28:42:We want you to accomplish this task. And just what would you normally do? Like, don't try and change your behavior. For us, we're just going to observe. And the thinking, telling them to think out loud is, is really key. Key. Like you're, you're not going to get all the answers. You're going to find really obvious stuff after one call, but you're not going to get all the answers unless you do a few of those because people have different learning styles. Like, I never trust search, so I'm always browsing and then I get really mad because the browsing experience usually sucks.
Dave Nunez 00:29:14:But a lot of people search first and so they're not browsing. So you get this like, experience and this feeling of how your users navigate your docs and the different design cues you're giving them intentionally or unintentionally. And you'll uncover a lot of things there because you may have an excellent document, but it's buried or it has a poor title. And then, so we don't ask a lot of specific questions. It's just think out loud. And then you may say, okay, why did you do that? Tell me more. And then a lot of really great themes start to arise after you do even just like two or three of those.
Demetrios 00:29:51:What are some design cues?
Dave Nunez 00:29:53:Like where you put the text box is an important one. The call outs of information are surprisingly important. You probably all read docs or both red docs where it's like, here's how you do the thing. But keep in mind you need this installed. And if you're thinking about doing this, think about this. And next thing you know, you're like three paragraphs in and you've only read caveats and you can't remember what you're trying to do. So to break up those cues of like, prerequisites, you may already have this stuff, but like, here's the checklist of things you need before you even start reading. Cause there's no worse feeling where you get halfway down the page and you didn't realize you needed to like install something that you didn't.
Dave Nunez 00:30:35:Um. Or like, here's a tip. And it's a, you know, it's a box that's elevated out from the docs. It's not buried in a paragraph or a warning or something like that. And then another thing that we did at Stripe were these things called escape hatches where it's basically a call out, but you're never going to nail the audience 100%. For example, we had hosted stripe checkout form where you can just very simple, no code, set up a form, but it's going to be a separate window with stripe in the URL. Or you can create your own, like write your own or customize your own. And it's code based and it's two very different paths.
Dave Nunez 00:31:21:Like a developer who wants very custom white labeled experience and somebody who just wants to get up and running in an hour, but they had the same name, they're both stripe checkout. So we were aware of the scenario where somebody's googling and they land on the wrong place and they don't know the other thing even exists. So we made sure that there was always an escape patch somewhere. Like oh, you're looking at the checkout, the hosted checkout. Do you want to go see stripe elements? Which is you can create your own and pretty much every product, especially when you're offering choice, there's some use case that you're not serving that particular user. So if you give them these escape hatches to feel, not feel like, oh, they're not serving me or they don't support what I'm looking for, then you have a really good experience to guide them down the right path.
Demetrios 00:32:15:It feels to me like a lot of the magic that people have felt when they are using docs that just work is that somebody has had that same pain or gone through that user journey and then thought about ways to make that as easy and as useful as possible. Right? Like you're saying how, okay, I want the no code solution. I google it, I see that, I click on this, it says checkout, but it's actually the code solution. I get through a little bit of it and then I go, oh wait, no, here's what I really want. And making it the best that they could and not leaving any stone unturned.
Dave Nunez 00:33:04:Yeah. And the feeling of being okay being behind the scenes when you're creating developer experience or content. Like I think the best feeling we could give to users is making them feel smart, like they're the ones with the accomplishment. Because a lot of times that's what it is. Like damn, I crushed that. I just built this thing. In 30 minutes they're not going to go say, hey colleague, the stripe docs are really great. They're like, hey look, I got this thing up and running in 30 minutes, maybe later they'll reflect back on like I want to go back to stripe because that was a good experience.
Dave Nunez 00:33:35:But at that moment, if you can make them feel smart and make them feel like they're the accomplished one, then you earn that trust, like probably subconsciously to start, but they're going to want to go back versus feeling like that was like pulling teeth. I don't want to go back to using that platform or those docs unless I absolutely have to. And one thing I see still, 90% of the time is documentation. Read like readmes. They don't describe what the thing is, and obviously it's a norm. And it's fine to name your products or your service something memorable, you know, platypus js or something that might actually be a real, real thing. I don't remember Mermaid JS. Like you can't intuit what these things are.
Dave Nunez 00:34:17:And if you go to the readme and it's just like, like we had this cert, we had 3000 microservices at Uber when I left, and they were all named, just random names, which is again, totally fine.
Demetrios 00:34:28:All right, real quick, some words from our sponsor Zliz, and then we'll be right back into the program. Are you struggling with building your retrieval augmented generation, aka Rag applications? You are not alone. We all know tracking unstructured data conversion and retrieval is no small feat. But there's a solution available now introducing Zliz cloud pipelines from the team behind the open source vector database Milvis. This robust tool transforms unstructured data into searchable vectors in an easy manner. It's designed for performance. Zliz Cloud platforms pipelines Zliz Cloud pipelines also offer scalability and reliability. They simplify your workflow, eliminating the need for complex customizations or infrastructure changes.
Demetrios 00:35:23:Elevate your rag with high quality vector pipelines. Try Zilliz cloud pipelines for [emailprotected]. you can find the link in the description. Let's get back into this podcast. So, well, on that note, why don't we talk a little bit about foundational elements that you need in your documentation, especially if you're starting out. What are some things that are must haves?
Dave Nunez 00:35:49:The onboarding experience needs to be really good, the getting started experience. Like, even if you have nothing else, don't even, don't even think about end to end tutorials or a lot of reference architectures until you've shown, like describe, like I mentioned, very simply and clearly and succinctly, like what the thing is, what the product or what your company is and why you should care. And then the the quick start should be a hello world experience where somebody is able to accomplish something in, you know, a sitting 1520 minutes, maybe 30 minutes max. And if you can't time box it to that like figure something out because if developers can help it, they're not, they're going to go find something else like get their attention for 1520 minutes and they can follow these steps and accomplish a thing to now go on to the next thing, then that's really key. So overview of what your company does in a short manner and then a quick start. And then from there, let's say you have a handful of features, maybe two products, you're going to have probably 20 to 30 documents. The information architecture is really important because it's your chance to create the user journey for the user. A lot of times it's either all flat, like all the pages in whatever order they're created is just listed in the side.
Dave Nunez 00:37:15:Nav. Yes. Or it's like a lot of times there'll be generic buckets, which is good. Developers are trying to do different content types, but it's like overview section, conceptual section, tutorial section, troubleshooting section. And developers don't think like that. They want to think, I'm getting started with your thing and then I'm trying to design the thing, then I'm trying to build the thing, then I'm trying to deploy the thing, then I'm trying to maintain the thing. So if you can get that sequential ordering to mimic the user journey that you want your users to follow in your information architecture in your side nav, you're going to create a pretty good experience for them.
Demetrios 00:37:58:Wow, so does get started, design the thing. What was the. Because I want to write that down.
Dave Nunez 00:38:04:Yeah, design, build, deploy, manage are usually like the steps, but figure out what those phases are for your product and architect your documentation around that to mimic that user journey. You're just removing cognitive load.
Jakub Czakon 00:38:23:Yeah, just taking maybe a step back because you mentioned that sort of like the minimal infrastructure of information in a readme. And to me, I don't know how you'd look at it, but to me readme is sort of like this. I guess the first documentation, the first doc you write is probably the readme. And then step from readme to those five steps for potentially multiple products. That's a big step, obviously.
Dave Nunez 00:38:50:Right.
Jakub Czakon 00:38:50:So I'm wondering, how do you think about transition? When should you even transition from that, you know, read me to an actual, you know, doc, whether MK docs or some platform that you use or whatever it is, and then, you know, build those sections in there right. Because I think it's like, easy to, you know, take leaps when you maybe don't need to.
Dave Nunez 00:39:11:Yeah, that's a good question. I think you can probably tell when you're spending a lot of time manually helping your users, especially early on. It's, it's pretty common. You don't want to, you don't want to set up your docs platform before you even have traction or know what your users need. But let's say at first, the readme is users are like, cool, I got up and running, you see your usage spike, and now you're getting a lot of inbound questions. Okay, but what about this use case? Or what about this feature? And you realize, oh, we support that feature, but it's not documented. So I think the readme accomplishes the front page of the repo, what the thing is and how to install it. But you need to graduate once the users are now beyond that.
Dave Nunez 00:39:51:Like, you have some use cases you want them to try out, there's some maintenance, there's some upgrading, like, whatever those things are that should not be in the readme. The readme should be contained to like a short overview, quick installation steps, and maybe some troubleshooting. But, like, once you graduate beyond that and you're that, and you're manually supporting users to get those things done, put it into documentation.
Demetrios 00:40:18:One thing Kuba and I talk at lengths about is how you actually measure with OKR's documentation. How can you say, all right, you're someone whose full time job is to work on the docks. How do you know what you're doing is being effective?
Dave Nunez 00:40:38:Yeah, I would say that's the easiest way to, like, if you want to end a conversation with a technical writer, just ask them that question and then they'll leave you alone because that's a really hard question to answer. So I've done it. I've tried a lot of things over the years because I think there's two ways to measure. One is like, the efficacy of your docs, like, how, how can we quantitatively measure the usefulness for our users? And then also internally, like, what's the ROI of putting resources on this? Like, are we actually, um, are we actually justifying the cost of putting engineers on the docks or putting full time technical writers on the docks? So most of my career was, you know, measuring page views, bounce rate, search queries, like driving clicks to sign ups. Like, those are all important, but it's not, it's not showing you the whole picture. And then the support ticket deflection to justify our costs internally. Like how many tickets are being booked after they read the docs, like, or how many took, how many tickets in general are being booked and how can we block tickets from being created or make them faster to be resolved through better documentation? But I've also changed the conversation or the mindset a bit because if you just focus on those metrics, you're going to get really terrible developer docs. You're optimizing for the wrong things.
Dave Nunez 00:42:02:You're making it really transactional. So I've thought about it and, and frame it more like design. Like, it's really hard to come up with hard metrics for design. Like, yes, um, driving traffic and driving signups, but, you know, good design when you see it. And a lot of that is, is, uh, is qualitative. So you have to have some trust there. Um, and, uh, but then some other things that are a mix of quantitative and qualitative driving. Like customer or customer satisfaction is good.
Dave Nunez 00:42:33:Like pretty much every doc set has page ratings on the bottom and those are good to make sure. I really just think there's a threshold you should care about. Like 60% or higher means it's, it's a solid doc. You're, I've never seen anything be 100%, unless it was like how to reset my password and then the doc, you know, the doc actually worked, but 60% to 80%, you're probably in really good shape. Nothing should be below 60% or it's probably terrible. But most importantly, especially when you're getting decent volume, you can see how long does it take for a user to accomplish a particular task that's really important. And it could be like mimicking whatever's in your quick start. Is it just installing your thing or is it creating a test charge with stripe? We were lucky enough to have data scientists that could like, tell us exactly how long it took and, you know, have different cohorts based off of their, their user demographic.
Dave Nunez 00:43:32:But that was the thing we obsessed the most about, was that, that time to first charge, that time to live, and then we mimic that for every product, you know, like time to first creating a subscription, time to first creating an invoice in it in a dashboard. So I would say those things that are not hard Google Analytics type metrics, but things that your docs obviously heavily influence.
Jakub Czakon 00:43:58:But so on that front, because I think that's a really good one, is the one you mentioned is basically sort of like a North Star, right? Like you're thinking that dog should support if that page is basically, quick concert to an FAQ. You want a high bouncer, like, you want people to go there, solve, go. Right. But I'm sort of like thinking what out of those, let's use, I mentioned Google Analytics metrics. Let's say you actually found valuable for improving pages, for improving those local things while keeping the north Star in mind.
Dave Nunez 00:44:33:Yeah, I think 100%. What you said is they're signals. They're signals towards your north star and you should treat them as such because you're totally right. I use that example a lot as well. The bounce rate on its own doesn't tell you anything. Like, are you trying to have them reset their password and go away? Or you don't want to brag that time spent on page is ten minutes for resetting your password, but those are signals once you know your business. And I think CSAT is a great signal as well. It's just an example of, it's shining a light on the things you should care about.
Dave Nunez 00:45:09:It's not giving you all the answers, but okay, I have a thousand pages. These are the top ten worst rated pages. I should probably take a look on if I need to improve these. Another thing too, that's, I guess, measuring, but it's super crucial and it gives you a lot of good product insight. What are people searching for in your site? What are the top ten search terms that is so illuminating? It's never what you think it's going to be like. You think that they're looking for information on all your new whiz bang features? Probably not. We, um, we didn't have any, we weren't using any metrics when I, when I joined stripe, like on, uh, for the docs. And so that was one of the first things I did was set up all that instrumentation and we looked at the top ten search terms and we're thinking, oh, we just launched this, uh, we just launched, uh, or not launched, but we're investing in Kinect, our marketplace platform, and it's really complex and we just need to provide more guides and better docs.
Dave Nunez 00:46:06:And then we go look at the search terms and it's like, connect's not in the top ten. PayPal's number one. Like, okay, well, we're not supporting PayPal, but then interestingly enough, number two, I think was like.net examples. Like people were looking, we didn't have, we didn't support.net. and that was feedback we were able to give to our product team and we're like, oh, we would have done this. We just didn't know. And then we updated all the docs. We have the tabbed examples with the different languages and we're obviously showing the ones or whatever the hottest new languages are.
Dave Nunez 00:46:35:But really our users were, they really wanted.net. so those search terms are super crucial. And then also you're probably not using the terminology that your users are using. Like you might have fancy feature names for your stuff, which is totally fine, but you need to know how your users think about your product and what they're searching for. And make sure that you're, if not writing in that way in your docs, like on the back end, setting up your search weights and your synonyms so they're routed to the right place because again they're going to be like, okay, I want point of sale. And then I go to stripe point of sale and it's like terminal. I don't want terminal. That sounds like a developer product.
Dave Nunez 00:47:20:Like see you later. But stripe terminal was the name of our point of sale product. So there are a lot of examples, like you can just find gold in the search terms in the top search queries.
Jakub Czakon 00:47:33:I think on the search for all those, interesting to me was that I assumed that people searching the dogs, but most of the folks, they actually go to Google and do branded searches. So something striped dogs and that was super interesting to me. And then actually you can do the same thing. You just look, what do people do when it comes to branded searches or especially branded with the docs in there? It's the same story. I think that is interesting because you're landing on some page through Google and that's where people start in the docs, they don't start on docs home. So you got to assume every page is page one. Speaking in that information foraging language that you mentioned at the beginning of England.
Dave Nunez 00:48:21:I rarely used r stripe doc search. I just had the muscle memory of Google search. I would start 95% of my search queries with Google for striped docs. Like I didn't, I don't know, it was just embedded, I didn't trust the search.
Demetrios 00:48:35:One question I had on the basically time to first action and how were you measuring that? Because if somebody's on the docs, are you using tools to know who they are or is it just that you're seeing them spin it up? Like how can I, as someone who is less sophisticated understand that? And I guess another piece to it is I have assumptions about what's important or what the first action that I want somebody to take. But then maybe there's ways that I can see what better first actions are. And so I can, ideally, with help from the docs, encourage people to take that first action.
Dave Nunez 00:49:20:Yeah. One of the best things that stripe had that looking back was huge for us, is I don't remember seeing a logged in experience for documentation before stripe.
Demetrios 00:49:31:Oh, wow. Okay.
Dave Nunez 00:49:32:Having a logged in experience, you got some tremendous data from your users going back for back and forth between the dashboard and the docs. So that's one where you can see exactly who the users are and their entire user journey. And then we just had world class data scientists, too, where we would get good anonymous data. We see where their ip address was from and their user session, all that stuff where you could create a shadow profile of this anonymous user session. Then in aggregate, you can get information that way. But creating a logged in experience with the docs, which I see is a lot more common now. You get really, really good stuff there and then. Yeah.
Dave Nunez 00:50:15:Deciding what to choose, I like to simplify it of what is your company's strategy? What do you want to accomplish? What do you want your users to see? And then your user, what is your user research telling you? Like, are they, it's great if that aligns, but what are they actually using your product for? And you should figure out, like, between those two inputs, like, what are the signals you want to focus on and measure?
Jakub Czakon 00:50:40:And, you know, as a technical founder, you know, what was the first action that really matters? Obviously, later things get more complicated and to your point, you know, you need research, you need, you know, you need to look into those things. But I guess at the very beginning, it's sort of, I mean, there are not that many candidates right at the beginning, right? Like, it's sort of, sort of clear, you know?
Dave Nunez 00:51:01:Yeah, but developers have so much choice now. And a lot of times when they're playing around with something, it's because they saw it on hacker news or somebody tweeted about it or they're googling. Like, I am not an expert in this thing, but I need to use a tool for this. And let me just Google, or let me search on Reddit and let me get, I have five minutes to give it a try. Because rarely are they, like, today, are they forced to sit down and use this thing. Like, okay, my CTO purchased this product, now I got to use it. A lot of times it's bottoms up experimentation, and there's so much choice. So how can you, like, create that really quick dopamine hit to make them come back or not say negative things about you maybe even say positive things.
Dave Nunez 00:51:45:And that's really how this flywheel starts for building a great dead brand where you hear two, two developers talk positively about a thing like that's very rare. You're going to be interested, um, to go try it out yourself. And it's already going to have this halo where you're going to, you're going to give it more trust because it's been promoted by people you trust.
Demetrios 00:52:05:The dopamine hit idea is exactly what I was thinking is, can you get someone that dopamine hit as quickly as possible and then boom, you don't necessarily got them, but you're on your way.
Dave Nunez 00:52:18:Yeah, like the oracle or the Java docs, it's just the equivalent of brutalist architecture. It's just like, what are you going to do about it? Read these, you have to. And they're just awful. They make you miserable. The way they look, the way the font is, the way the kerning is, the way, like, everything is just miserable. But. So how can you try and achieve the opposite of that?
Demetrios 00:52:45:Exactly. Going back to what not to do, that is, that is very helpful. Go do the opposite of those java docs. Well, dude, Dave, I appreciate this so much. I want to know what you've been up to now. Like, what are, what are yours, your day in, day out look like these days? And what do you excited about?
Dave Nunez 00:53:03:Yeah, I, when I left stripe, it was July 2022. And a lot of like, I felt that I was getting repetitive a little bit. Like, we didn't talk. We could do an another episode on just internal docs, which I may even be more excited about than external docs because it's a new frontier. But a lot of it felt like, all right, there's just basically maintenance. Like, I don't know what I want to do next. I just did this twice in a row. Do I want to start over again? And then three months later, chat chibute dropped and I was like, oh, I'm going to be out of a job soon.
Dave Nunez 00:53:41:I remember my father in law sending me an article. He's like, look, what's number one on jobs to be replaced by AI first? And he thought it was hilarious that it was technical writer. Everybody in my field had that thought, what does this mean for us? But I was so I was doing angel investing and advising, investing, that whole, you know, uh, pseudo sabbatical. And I started getting a ton of inbound from AI focused companies that were like, we have some traction, we have a product. We have no idea how to educate our users. So going on a year and a half of that, like, I have actually two consultancies. Um, one with the first tech technical writer at Stripe, a guy named Larry Ullman was there nine years. Like, we do, we take warm referrals for companies that want to do, like, improve their developer experience and their documentation, obviously.
Dave Nunez 00:54:35:And that's just been so exciting to like, learn and try and create the new way to present information because LLM focused products, it's, it's got to be different. Like, you can't deal with a really real time multimodal interface. And now I'm going to go look at static docs. That just feels so wrong. So we're spending a lot of time rethinking how to present information for LLM AI focused products and then also doing consulting for, I founded a group called Abstract Group, and we're also doing just go to market overall. Like, hey, you're a researcher, you raised a lot of money. You've never like, worked at a real company before. Like, how do you actually get your product to market and how do you get traction? So I've just been all in, like, it's very, well, maybe if I didn't just have a kid, I would have done this with the crypto phase.
Dave Nunez 00:55:31:Like, all right, I'm all in on nfts. It's so exciting to be part of a technology wave, but I'm, I'm trying, I'm just, yeah, unashamedly so excited about AI and trying to figure out how to help startups, like, with all those messy problems that I think a lot of us kind of felt like, all right, we mostly had the answers for SaaS. Like, we may not do everything how we're supposed to, but we know what, like, we have good examples, we know how to get there, and that just does not exist for AI companies. And I think that's really exciting for me.
Demetrios 00:56:01:The idea of internal docs out of everything you just said is what got me so excited. I love the LLM stuff, but that, honestly is what we, like, play in every day. I've been reading two books that I can't get enough of, and it's by the same author. One is writing to learn, and the other is like, on writing well, and it just talks about the power, basically the superpower of writing. And that was one of the reasons that I was so excited to talk to you, too, because you can accomplish so much in the written word. And it's funny that you're mentioning how now, all right, you can't just have only the written word. You do need to accompany that with other ways to keep people engaged and keep things more interesting. But when you have very complex ideas that are, put simply on a page, it is so incredible.
Demetrios 00:57:06:And internal documentation is like the last thing to get that. And especially with the remote culture, I am so on board with that idea. So yeah, I would love to get you on here again and talk 100% about that.
Dave Nunez 00:57:22:It's great because you and hopefully stripe open source is what we built. But we took Markdock, which we created for external docs, and we took the learnings and the philosophy and applied it to internal docs like nothing crazy. The hard part was convincing people internally that this approach would work, and luckily it did. But there are also less rules. There are so many things you need to worry about justifiably. So with user information, you have all the access to your users internally because they're your coworkers and you don't have to have this level of polish that you need for external. You don't have to worry about mean tweets and you work side by side with your users. So you can say like, hey engineer, this is for you.
Dave Nunez 00:58:07:Like you need to help me or this team. I'm helping you out so you can do some phenomenal things. And like they're, I hope they, they do a blog post or something on how amazing the internal docs platform is. I think it's rivaling the external docs platform now. It's right.
Demetrios 00:58:23:Excellent, dude.
Dave Nunez 00:58:24:The other thing too, that I guess it's a little self promotion, but I am excited about it because I want it to exist. I'm working on a book about, I mean, it's not about stripe product philosophy. A lot of people we're talking to are from stripe, but it's more about how you build great products with a particular mindset, not a framework, not a process. It's more what are the war stories from accomplished startup people on building things that developers and users love? So that should be coming out next year, and I'm working on it with Sid Orlando, who is the editor in chief of Stripe Press for a while. Phenomenal writer. And we've been interviewing a bunch of people from across the industry because that's how I've learned the best is reading these books from even companies that failed know or projects that failed, like Xerox parc or silicon graphics. Like what did they do? And just tell me the stories. Don't try and distill it into a philosophy or a framework.
Dave Nunez 00:59:25:Just tell me the stories and I'll derive my own findings. So I'm really, really excited about that because I don't think, and going back to the beginning of the conversation, Kubo, when we were talking about like, it's all the, all the functions, it's marketing, it's, it's sales, it's, it's developer experience, it's engineering, it's technical writing. And that's the core thesis of the book is like, we're going to try and tell the stories from all these perspectives because you need all of these functions in order to build great developer products.
Demetrios 00:59:54:Awesome. Dude, I really appreciate you coming on here and chatting with us about this. It was very self serving for my part because I got both of you, who I very much respect on the doc side of things, to get in a virtual room and chat all about it and really geek out. So I learned a ton. I mean, I have notes and notes. So just going from that idea of how you structure your docs to not be just like throw it up there, take, take a little bit of pride and thoughtfulness in what every bit and piece of the doc looks like. So going from what you were saying, how it's getting started, design, build, deploy, manage that I had never heard before. I thought that was really cool.
Demetrios 01:00:43:The dopamine hit, just try and get that quick dopamine hit as fast as possible and make sure that you can get that. And then what not to do. Like don't write complex sentences, don't put a bunch of, don't make it like web 1.0 where you're getting banner ads all over the place and everything is trying to vie for your attention. Make sure you are thoughtful in how you are doing whitespace because people are doing that f shape scanning pattern, which I fully agree. Whenever I will read stuff I'm not, I'm mainly looking at the headers to see if it's relevant and then I'll dive in.
Dave Nunez 01:01:19:So big same.
Demetrios 01:01:20:Yeah, both y'all. Thank you.
Dave Nunez 01:01:22:Yeah, good bud. Mitrios, thank you for having me.
Jakub Czakon 01:01:25:Thank you so much.
