Category: Uncategorized

AI in the Wild: Observations of how “Normal” People are Adopting AI

Posted on Posted in UncategorizedTagged , , , ,

The other day my book club (the book was Why Fish Don’t Exist by Lulu Miller, highly recommend) descended into a discussion of AI.  It was an interesting group.  I was the only person there who works in tech.  Most of the rest of the group was teachers, white collar professionals, and consultants.  All medium user-level tech-savvy with a clear understanding of navigating the internet but no one was going to explain how a database or algorithm worked.  So, basically, a pretty representative group of the average user for most of the applications I have worked on.

So what were some of the use cases that this sample was relying on AI for?

Search Replacement

The most common use case, by far, was using AI to search for information.  From “what should I make for dinner” to “historical facts about ARPANET” it has become a combination of Google and Wikipedia rabbit holes.  Many people in the group described getting lost in a conversation with AI and going to a lot of wild routes to unusual connections.  This feels like the internet I grew up on, on steroids.  And it’s impossible anymore to search without AI coming back with suggestions.  Google, Microsoft, Amazon, most large companies have replaced their natural search patterns with something powered by an LLM.  Because summarizing information is something AI is really good at.

Life and Social Coach

One member of the group mentioned using AI to come up with comebacks to a bad interaction with a neighbor.  Until her daughter criticized her for the energy use.  I let her know that according to Science Vs. that kind of usage has low environmental impact while videos have a significant cost.  She didn’t realize you can make videos with AI.  This resulted in a lot of discussion of using AI as a Life or Social coach.  Honestly, that is how I am most using AI right now: job coach.  I like having something I can talk through specific situations with where I don’t feel like I am burning out a friend.  It’s not about output, it’s about the things I would have previously just not worked through.  Like the  right way to approach a CEO of a company I want to work at.  And how to think about rejections within 24 hours of an application.  When is the right time to pivot?  It’s like having someone to ask for advice that has a literal encyclopedic knowledge and there are no social consequences of not taking the advice.  I think that’s a pretty cool addition especially for those of use who are neurodiverse.  That mother who asked for comebacks to her neighbor?  She didn’t use any of them.  It let her vent off steam without any social cost.  That seems like a good outcome.  Is that how we should be using this tech?  It doesn’t fundamentally change anything but, honestly, maybe it would make us a better world if we thought a bit more before we reacting to other people.  AI as a meditation supplement?  Introducing a pause before reaction just to see what the LLM says?

I also heard examples of using AI to give directions on the job or help write emails to coworkers.  The latter was not effective because your coworkers can tell it’s not you.  The former is straight up dangerous.  I think it’s really important to understand that AI can’t make decisions.  It can be very convincing but at the end of the day, it’s going to tell you exactly what the most common answer is, not what the right answer is.  One person in the group noted that when it’s a topic, she is knowledgeable on she has observed AI to only be right about 40% of the time.  This is one of the reason I will use the grammar checker in Word to correct my structural problems but I don’t let AI write my blog posts.

Research Assistant

The teachers in the group complained about students feeding homework assignments, especially essays, into AI and then having to use AI detectors to determine which kids were doing this.  Meanwhile, the parents described how their kids would use AI to start a project and then laboriously rewrite what the AI wrote to avoid the detectors.  This whole pattern seems to miss the point of what the AI is good at and what is clearly cheating.  Generating your own ideas and then using AI to help structure those ideas coherently, e.g. having it help create an outline or review and offer revisions, seems like a good use.  Plugging in the prompt and then turning in the paper just seems like a missed opportunity to actually learn the content of the class.  No real difference from just googling the answer and scaping an essay from the internet.  Only it’s way easier now to do that.

What was really interesting in the debate about AI in schools was that, at least with this group, it hasn’t yet matured to the point of teaching children how to make smart use of technology.  There is a lot of reactivity to the tools kids have access to now.  Both of my kids are in Girls Who Code and have played with Scratch.  We have used physical games to understand the logic patterns of coding.  We have a family rule that they have time limits on using device time consuming other people’s content (ug, the Minecraft watching videos) but they can use their devices for creating.  Both have played with Suno to create their own music (one plays guitar, the other sings in the choir).  This is a mature reaction to emergent technologies (if I do so pat myself on the back).  We aren’t there yet in having clear guardrails in our schools or in society.

Another member of the group described her personal use of AI as a research assistant.  She dumps her historical writing and her research into the AI.  She then prompts it so compile what she is looking for in that particular project.  She is training it on her writing style.  She is also checking sources again before publishing anything.  She is having the AI generate a first draft she can react to and adjust.  This feels like the cleanest, real, example of using AI as a research assistant.  I imagine in the past someone would have used a college grad student for exactly this kind of work (pin that destruction of the employment pipeline thought for later).  It’s a fascinating use case.  This is a legit professional using AI to make herself more efficient but not to replace her own thinking or expertise.

Intern

Related to the research assistant, the most common usage I have seen inside companies, and engineering teams, is treating AI as a really capable intern with no business expertise.  It has no context to the environment and it can’t make decisions, but it sure can generate a lot of code.  I have talked to engineers who see a real value from AI in throw away scripts and, in certain very greenfield situations, writing production code with guardrails.  But you can’t unleash it at a legacy system without serious consequences.  So this is back to that pin I put in the research assistant discussion.  I fear that AI is destroying the talent pipeline.  There is a lot you learn in your early years and if we replace 0 years of experience roles, both inside and outside of tech, we miss out on those opportunities to learn and grow our talent naturally.  We will see how this works out in the next few years.

Jennie’s brief diversion into a history lesson

 At  one point in the conversation I finally outted myself as working in tech and noted how strange watching AI use in the “civilian world” has been from where I sit.  AI is one of the first technical tools I have seen in my career that has been broadly launched to the public before it has the natural pressure testing and guardrails introduced by a more technical audience.  It was almost a decade between the birth of the internet (as ARPANET) to when AOL became common place (in the mid to late 90s).  APIs revolutionized the way computer systems could communicate with each other in the early to mid-aughts but they still aren’t something a civilian really can make use of.  Blockchain was born in 2009, but it’s really become the last few years when bitcoin started to become mainstream with platforms like Coinbase in 2012 and the ability to buy them on traditional brokerages, most notably Fidelity or Schwab) not until 2024.

AI has conceptually existed since the 1950s and Deep Blue won that chess competition against Kasparov in 1997.  Machine Learning algorithms have been used by tech teams aggressively for the last two decades.  But generative Ai has exploded in the last year, both inside and outside the software industry.

We are in a fascinating time with this emerging technology and I look forward to more opportunities to observe users and inventive uses.  I look forward to looking back on this blog post with head shaking amusement.

Note: I don’t use AI to help write my posts or create example pictures. I do use AI to create the header image and as part of searching for historical information (because it’s unavoidable). In this case I prompted Claude, Gemini, and ChatGPT by giving eachmy blog post.  In this case ChatGPT won.  I also asked them all to give me a title and LinkedIn summary.  ChatGPT and Gemini both came up with similar ideas which I iterated from.

Questions Before Answers

Posted on Posted in UncategorizedTagged , , , ,

Questions Before Answers

Usually on my desk, prominently displayed beneath my computer monitors, there is a bright purple index card with the words “Questions before Answers” written in my best fancy script.  I have currently misplaced it and need to make a new one.  I feel it’s absence.

After one particularly intense meeting with a stakeholder I walked away from my computer for a bit, breathed deeply, and created this card.  Anyone who has worked with me is probably now thinking “I know exactly who she is talking about” but that’s the funny thing: you don’t.  In every team I have worked with it’s a different difficult stakeholder, even varying day to day, often more than one.  At this company it’s account managers, at that one it was operations, at this one here sales, over there engineering.  I made the card because it’s a reminder of human nature more than a particular stakeholder.  And in that conversation that day, it was me that needed the reminder.

Often in business we are trying to move fast.  We come into a conversation, and someone is presenting something hurting them.  Often, it’s something in the software that is not helping them.  A missing filter, an interaction that isn’t quite right, a missing field, a process that is incomplete.  Behind that is a whole set of business processes that have been built up by people doing their best with incomplete information or operations.  People dealing with reality that we are trying to fit into software.  Behind that is money, customers, demands.  The real pain present, and all the things those people have done to solve their problem before talking to you, short circuits the conversation.  We immediately get into solutions.  “Just make this change”.  Urgency drives and understanding falls by the wayside.

There are three things my kids know before anything else.  The first is that I love them and they are safe.  The second is that anything they want to do requires practice and there are no short cuts.  The third is that we can solve any problem if we all agree first on the problem statement.  Weirdly, all three of those also make it into my workplace.  I’m at least very consistent.  I am very diligent on how to bring those conversations back from “implement this change” to “how can we solve this problem together?”.  How do I do that?  The same way I do with my children.

The first thing I do is work to get down to the root problem.  This includes questions like

  • Can you walk me through that in the application?
  • Please show me what you are doing when that happens.
  • Help me understand what you are doing, what are you trying to do here?
  • Explain it to me like I’m a German Shepherd.

That last one came from one of my favorite engineering teams, it’s great for breaking the tension.

Then I stop and I listen.  I listen for a long time.  I ask follow-up questions.  I don’t try to solve their problem; I just keep coming back to make sure we on the same problem statement.  The most powerful statements to help with this are:

  • What I hear you saying is:
  • I am seeing you do X, am I getting that right?
  • It seems like the current parameters are X, is that right?  Where within that are you hitting a problem?

It is so easy to slip into “if we just change this one thing, it will solve their problem”.  That is quick wins thinking.  And when you can find a quick win like that, celebrate.  Write about it.  Post to LinkedIn.  Take a pause and tell all your friends.  Because those so rarely happen.  If the problems we faced had easy solutions where would the fun in that be?

Only once I can clearly articulate their problem, by asking a lot of questions, do I start to offer any solutions.  And this is the beauty of that bright purple index card.  It’s a reminder to me to stay humble and curious.  Fundamentally, that is what product is.  It’s solving business problems with technology, but we can only do that if we truly understand those problems.

Note: I don’t use AI to help write my posts or create example pictures. I do use AI to create the header image. In this case I prompted both Gemini and ChatGPT by giving it my blog post.  In this case ChatGPT won.

CodeMash Conference 2023 – Consuming Endangered Pachyderms

Posted on Posted in Uncategorized

Slides

consuming-endangered-pachydermsDownload

Blogs on related topics

Mental Model Minute: Seven Levels of Authority

We are all growing and learning as professionals in a collaborative workspace. It can be helpful to take a step back and look at different models of individual and team behavior. A “mental model minute” takes just a moment to describe a different view on how to perform or interact better that we can react…

by jenoak

The Next Right Thing

You’ve seen Frozen 2, right?  I mean, if your household is anything like mine, it’s playing on repeat in the background as children scramble, color, and fight.  Admittedly, I have only sat and watched it end to end once.  Still, there is one part of the movie that stick out to me and, I think,…

by jenoak

Build Things People Want

When I started at my current company, the mandate I was given by the developers was to help them build things people want.  It is something I have heard over and over again from developers throughout my career.  And frankly, it’s a goal that the organization and the developers should be aligned on.  The business…

by jenoak

Also check out other presentations.

CodeMash Conference 2023 – The Interview Lab

Posted on Posted in Uncategorized

Slides

the-interview-labDownload

Blogs on related topics

Processes for Establishing Teams

Often when I am coming into a new team much of the work I am doing, while I get up to speed on the product and market, is helping the team focus and work together.  Over the years I have come across, modified, and developed several tools to help facilitate establishing teams, norms, and roles. As…

by jenoak

Mental Model Minute: Intent Based Leadership

I have so many things to share with you but so many discussions I have been having keep leading me back to this Mental Model Moment I thought I would share. And it’s easy, because it’s the author and this great illustrator do so much of the work for me. On the other side I…

by jenoak

Remote Work is Here to Stay

As operations leaders across the world stumble and struggle to figure out their “return to work” strategies, I have been reflecting on the nature of the fundamentally collaborative work we do in software development and what it will look like in this new world.  I don’t know what the post-Covid world of work will look…

by jenoak

Also check out other presentations.

Processes for Establishing Teams

Posted on Posted in Team Building, Uncategorized

Often when I am coming into a new team much of the work I am doing, while I get up to speed on the product and market, is helping the team focus and work together.  Over the years I have come across, modified, and developed several tools to help facilitate establishing teams, norms, and roles. As always, no tool is exactly right for all teams. Adapt and change for the nature of the team you are dealing with.

Norms

Goal

Establish a common set of team level rules for how we collaborate and interact.  This can be everything from SLAs around PRs to the number of meetings we have, from how we handle conflict to support rotations.

This process should be team led.  The team leader should be a participant in the discussion.  As such it can be very difficult for the team leader to also facilitate the discussion.  A team norms discussion also becomes a way the team actively works through their communication and conflict management style.  If possible, it is better to do this synchronously.  The conversation is part of the point.  Approximately an hour should suffice for this discussion.  Some teams can do multiple exercises (norms + mandate or norms + decision making) in one session.  Once norms are established, they should be updated over time.

Process

Let the team know you will be doing an exercise so they can think about it in advance.  The prompt is “Think about a team you worked on that was great.  What made it great?  Think about a team you worked on that was terrible.  What made it terrible?”  Capture one thought per sticky.

Then group stickies together into themes.  Talk through the stickies in each theme together as a group.  As a facilitator make sure that people are engaging in debate on whether that idea is important to them as a team or not.  This will allow you to come to a set of statements that basically boil down to “We believe…”

Examples

From a very mature team with a lot of baggage, this set of norms evolved over the 2 years the team existed. This example is only a subset of the total norms the team had. They did a remarkable job of holding each other accountable by simply saying “thank you for following our team norms”.

At a very different company, a very different result. This was a team coming out of a lot of process whiplash.  Also, they were globally distributed (designer in India, team lead in South Africa) so there was a lot of concern for async communication and clear hand offs.

Roles and Responsibilities

Goal

Clearly define who owns what part of a process or has decision making power is essential to clean hand offs and action.  While many activities fall clearly into one job title (HR is a great example of a job with clear boundaries) in the space between Product and Engineer, or between levels in an organization, things can be a lot murkier.

This process can either be leader with team or leaders among themselves depending on where you are seeing grey areas.  It is best that anyone who’s position is being defined is part of the process.  This process requires a willingness to engage in some difficult discussions about both skills and responsibilities and can become heated so it’s best to have someone with position power that can break an impasse.

This process should be done in conjunction with establishing a DACI process or establishing a 7 levels of authority so you can clearly talk about both the “owner” of a process and who should be involved in the decision and how involved.

Process

This is a classic card sort exercise focused on job responsibilities.  This should happen in two phases.

Phase 1: Collect responsibilities

Bring the affected group together to collect all the responsibilities that need to be defined and owned.  Working alone capture one responsibility per sticky.  This can be as simple as “review a PR”, as mundane as “schedule meetings”, or as complex as “architectural direction”.  Set a timer for the independent brainstorming.  At the end of that time, go through all the items and group them and chose a “winning” card to represent each group of ideas.  Take these cards to a new clean area.

Phase 2: Card sort

Have boxes for each of the roles you are trying to define.  Together as a team determine which box each sticky belongs in.  If the sticky belongs in multiple boxes, duplicate it.  If a sticky represents a “decision owner”

Example

Most of this kind of discussion I have done has been in person, so I don’t have a captured shot.  This is a heavily cleaned up example from our process of hiring a Head of Eng and establishing teams with a “pod captain” for each.

Decision making structure

Goal

Sometimes it is helpful to separate the Roles and Responsibilities discussion from the decision-making structure.  This can especially be true when there are a lot of people involved in decision making and it can be separate from the actual job responsibilities.  This has been the case for my current role as we have moved from a single Engineer team of 25 to 5 pods with pod captains.

Process

Same as ‘roles and responsibilities’ card sort with the initial part focused on decision making stages.  In our process we also added in the variable of where and how the decision should be made and SLA because of how async we are.

Example

I am a Unicorn (and other insights from community building)

Posted on Posted in Uncategorized

This is not (necessarily) work related.

This is my daughter, Finley. Yesterday at school she learned how to sign “I am a unicorn” (gif link). I consider this an important life skill.

She regularly comes home and tells me about the various new signs she has learned like “exuberant” (one of her favorites). Finley does not have any hearing or speech impairments. Her Kindergarten teacher is using American Sign Language (ASL) and Spanish as tools for teaching reading and writing. She has a friend Felicity (the daughter a dear friend) who lost her hearing and recently got a cochlear implant. While the other kids teased Felicity, Finley showed curiosity and compassion. She has told us that when she grows up, she wants to be “a voter” and wants to live with “deaf people because they can teach me so much”. I am indescribably proud of her. But this doesn’t happen by accident.

My uncle Ron taught me ASL for the alphabet, numbers to 10, relationships (mother, father, etc), basic courtesy, and some animals that would interest a 9-year-old. It was survival level ASL and a gift that he gave to as many of his nephews and nieces. It was also a glimpse into a world that most people don’t know exists. Ron was born blind and deaf because of a congenital condition that also impacted my sister to an extreme degree. He would teach ASL by moving your hands into the correct shapes. I learned the nuanced differences between F and 9 that a sighted practitioner of ASL wouldn’t care about (even within the deaf community, there are subcommunities). He also taught braille at the Iowa School for the Blind and was a professional computer programmer. I also learned recently that he was a well-respected advocate for disability rights (Ron referred to himself as disabled. He hated the recent trend towards “differently abled” because, in his words, “being blind isn’t an ability”). He built communities everywhere he went in his life. Through him I learned about nuanced parts of the deaf community that a hearing person would never interact with.

I have always talked to my children about my uncle and my sister and the things they experienced. I can see in the three generations how much the world has changed. Ron struggled and fought for understanding and support for his whole life. I was ambidextrous leaning towards left-handed as a kid, the teachers wouldn’t let me write with my left hand. Finely is learning ASL in Kindergarten and her teacher protect and encourage her left-handedness. The world has evolved. Slowly and in ways sometimes too small to see. But at least now my child knows how to say “I am a unicorn” in multiple languages.

This part is work related.

I think my new hero is Frances Haugen (who also is an Iowan, btw). What she has done, the bravery it displays, is breathtaking. She talks about the possibility for social media that brings out the best in humanity. Recently, every time I have written a document I have thought “how would I feel if someone like Frances was looking at this?” This has been especially true as I have been trying to get clarity around our data capture and tracking policy. Because I want what we do here to be for the general good. Because I believe that there is a need for people to build communities. And that should happen in a way that isn’t creepy and isn’t just trying to manipulate people. I am equally motivated in my work by a consciousness of how understanding different cultures and life experiences makes us less cruel and more compassionate towards each other. And by my 6-year-old who is being exposed to those differences in a way that makes her curious instead of afraid. How different her world is, already, from the one my uncle was born into.

Mental Model Minute: A Decision Making Model

Posted on Posted in Uncategorized

Collaboration is good. Making decisions is good. These two things don’t have to be in conflict. I have seen a lot of small but growing companies where this feels like an either/or choice but it doesn’t have to be. The key is knowing who makes the decision and how much input is required. The more explicit we can be about who makes a decision, the better we are going to be at actually making the decisions.

To that end, I have started using a really lightweight framework for this and wanted to share that here. I am going to get more diligent about this personally. In meetings, documents, and other decision making spaces I am going to put out this little block. Example from a recent meeting:

EPIC: Better Sidebar/Links Organization
RFC(s): None currently
Priority: Very High (1c)
Meeting type: Discovery

Description: Allow Creator to have better control over the left bar with some options including:

  • More granular organizational structure / more sections than just “other”
  • Specify the open on site vs open in new tab options
  • Dominance of iconography / style

Stakeholder Roles:
Decision Owner: Jennie for this meeting, PM moving forward
Consult/Advise: CEO, Designer, Engineering team
Inform: Client

During the meeting, I will verbally note the things we are trying to decide and, if appropriate, what the decision was. After the meeting, I will document the decisions made at the meeting and any logic for that decision. This is especially important in our async environment. The idea here, though, is to make decisions quickly without a lot of process and clearly. I am sure we will iterate on this over time.