Wednesday, July 29, 2015

Have a Theory

There is a phrase I find myself employing pretty frequently at work, when discussing new features or products. While I am not a product manager, I am responsible for making sure that we implement features well, and thinking strategically about what we are spending our precious time implementing. So, when I am asked about my thoughts on a new product or feature, I usually have one and only one question:

"What is your theory?"

In this day and age we sometimes get lazy about thoughtfulness, and rely on data and experimentation to hill-climb our way through the world around us. Or at least we say that we rely on data and experimentation to drive our features. But the reality is that we're working in such complex multivariate environments that we cannot possibly test all permutations of even the simplest change. We do make choices about what features we build, and these choices are not entirely data-driven.

So, given that our choices cannot be entirely perfectly data-driven, how then do we decide what to build? The only way that we can make sane choices in a complex world is by actually being thoughtful about the choices we are making, creating a theory, and creating experiments that actually test that theory.

For example, in my current world of e-commerce, we often are faced with the mandate to implement a new feature that will make the customer feel better about the product in some nebulous way (it's cooler! it's more high fashion! millennials will love it! whatever). This feature, while it might not cause customers to immediately buy more up front, should cause them to be more loyal over time. Sometimes, this is the right instinct. But beware: if you're going to try to get second or third order effects from a feature, you'd better have a really solid theory of the chain of events that leads to those second or third order impacts. And you need to figure out what you can measure to validate the chain of events. Don't just look at the number of people buying the product and hope it goes up. What does making the look and feel "cooler" DO for your customer? Do they visit more often? Spend more time? Tell more friends? Have a theory!

Failing to have a theory, and a solid experimentation plan for proving that theory, leaves you open to all kinds of irrational outcomes. The worst of these is the "you just didn't implement it well enough" outcome. The original idea was good, but you implemented it poorly, and that's why it failed. And that could very well be true! But it's impossible to prove or disprove without anticipating the question ahead of time, thinking through the logical conclusions of the theory, and setting up a good test to understand its outcome.

So the next time you are building a feature, ask yourself: Do we have a theory? What is it? Are we measuring the immediate expected effects of the theory, or are we just measuring the same stuff we always measure and hoping that it changes?

Tuesday, July 21, 2015

Ask the CTO: Going Rogue

I often get asked one-off questions about engineering leadership and management, and thought it would be fun to share my answers here. Asker's question has been anonymized and generalized.

The challenge:
I have an employee that was supposed to be adding a needed feature to one of our core systems. A few days ago I notice in GitHub that he has created a new repo and been working solely in that repo for the past two weeks. Instead of adding a feature to the new system, he is completely rewriting it! Furthermore, the repo is in a new language that no one in the team uses and that we have never put into production. I feel bad, I should have noticed this before it got this far, but I never expected someone we consider to be a senior engineer to go off and do this without at least checking in. How should I address this?

My thoughts:

Oof. This has happened to most of us at least once, in some fashion. Engineers want to be able to use what they think is the right tool for the job, even when the right tool is brand-new to the company. And generally speaking, this should be OK! The last thing you want to do is stifle people's initiative to create solid solutions and learn new things.

That being said, there is a high overhead to adding new things to your stack, and at some point, it usually makes sense to have some policy around how to add new things. I whipped together such a policy for my team about a year ago, when we had reached around 40 on the team and there were some folks discussing creating a new service in a language that we had very limited experience with. Our policy looks something like this:

Before any new language/framework is chosen for a production system, the following needs to be in place and approved by Camille and an architecture review board (group of senior engineers who are knowledgable in the area of change and would be impacted by it)
  1. the engineers advocating for the language/framework will present a case as to the benefits it will provide over existing choices
  2. there is a plan for what sorts of systems this language/framework should be used to implement, and what existing systems could be rewritten in it
  3. a style guide and templates for readability, testing, continuous integration, monitoring, logging, deployment and production standards will have been created
  4. at least four engineers on the team must sign up on learning how to write readable, production quality code and support the new systems in production
This must be done before the start of any project for engineering to commit to supporting the resultant code.
 This is a bit of a heavyweight list, but it articulates some of the challenges with bringing new languages and frameworks into teams. If you are in a team that wants to be conservative with new languages, frameworks and tools, clearly articulating the process for adding new things is an important element to avoid unexpected surprises on both sides of the equation.

So, you can put such a policy in place and point to it in the future to try and prevent such things from happening, but it has happened now, and there is an argument to be made that part of being a senior engineer is knowing when to communicate scope changes such as the need to move to new languages or frameworks. Even if you don't agree with my conservative approach to adding new things, you probably appreciate it when you get a heads-up on important changes early in the process.

The conversation you have now should involve first understanding their perspective: Why the new language? Why didn't they grab you earlier to tell you about it? What you learn from their perspective might surprise you. Perhaps you skip all of their 1-1s and they think you are unapproachable. Perhaps they are frustrated with the way you make decisions. Perhaps they knew you would be mad and were simply afraid to show you new work early when you would shoot it down.

Once you have gotten their perspective (and perhaps some takeaways for you), now it is time to clarify your perspective and expectations of them. As a senior engineer, you need them to push information to you. You expect them to communicate the scope of changes and approximate timelines, and let you know when these things change. They need to think about their peers, and have empathy for the needs of the team as well as their own interests; all too often teams will reject projects they weren't aware of and didn't have any say in. Socializing change not only to your manager but to your team is part of the role of the senior engineer.

So, to sum it up:

If you care about having a somewhat conservative process for new languages/frameworks, clarify what that process is and share it with your team

When someone ignores the process or otherwise surprises you, first ask why and try to understand their perspective

Finally, clarify your expectations to them, helping them understand the impact that their actions have on others and the importance of communcation

Sunday, June 28, 2015

Vision and Trust: External and Internal Leadership

Fred Wilson recently wrote a post where he defined leadership as this:
It is charisma, it is strength, it is communication, it is vision, it is listening, it is being there, it is calm, it is connecting, it is trust, faith, and belief
Trust, faith, and belief. These are all words for the same thing, right? Well, not exactly.

I have observed that many leaders, especially the ones called visionary, are often evaluated in the court of public opinion on the following subset of those qualities:

Charisma, strength, communication, vision, connecting, faith, and belief

Listen to them talk to a crowd and they will blow you away with the clarity and strength of their vision, with their ability to connect with their customer. This in turn translates to a level of faith in that vision and belief in the overall direction that they are guiding their company towards. Awesome. Literally, awe-inspiring to witness. These are the public qualities of leadership that show up in the media, that the whole company can see in an all-hands, that you can see firsthand when they speak at a conference. These are the qualities that the board members see, that the venture capitalists invest in, and it is pretty hard to get into the position of successful startup founder without them.

So where do the rest come in? Listening, being there, calm, trust. These qualities are more difficult to evaluate based on an interview or a presentation, these are the “internal” signs of leadership.

Trust is a contract between two people. You are constantly creating and building trust in a long-term relationship with everyone around you. When you listen, and are there for people, and are calm when interacting with them, you build trust. But without that listening, without “being there” in a way that lets people feel the ground beneath their feet, without calm, trust is fragile or non-existent. Trust is regularly tested and negotiated based on our ability to show up. I would say that these more private qualities, these relationship qualities, are the qualities of management that every great leader must possess. Managers know the value of steadiness, of showing up for that 1-1 every week, of reacting slowly and listening to the people around them.

So even great external-facing leaders need some management skills. What about the managers?

Managers fail when they lack communication, connecting, and strength. A manager who can’t communicate with their team cannot execute effectively. A great manager connects with their employees as human beings (without turning into full-time therapist), and has the strength to shoulder the challenges of making hard calls in the trenches with their teams, the firings, the resignations, the projects being cut and the delivery or missing of deadlines. The day-to-day pains are very real for the manager and without strength it is almost impossible to do the job well.

But what about vision? We think that vision is the realm of the strategist, but vision also has a place in the manager's skill set. Instead of business strategy or architectural future, the manager’s vision sees what their organization looks like in 2 years, what their team can grow to be capable of accomplishing, what the successful day to day looks and feels like for the employees on their team.

As a final data point, my former CTO coach and one of his partners wrote about the Management/Leadership split in their new version of The Art Of Scalability, excerpted here. Between these three sentiments, perhaps you can triangulate your own path to being a great leader who manages enough, a great manager who leads enough, or whatever the situation calls for.

Tuesday, June 2, 2015

The Trials and Temptations of the New Leader: "Cool Factor"

Being a new leader at a startup is hard for many reasons. You think you're good at a lot of things, only to discover that you're not. You were fooled by success gained inside of a company with hidden structures that helped you succeed, the invisible backpack of big(ger) company privilege.

One very common area of weakness is recruiting. When I started to lead engineering at Rent the Runway I fancied myself pretty good at recruiting. After all I had done it well in prior gigs, I was friendly and engaging in interviews, so I'd be fine! Of course I quickly realized that startup recruiting is enormously different than at a company with a whole recruiting department sourcing candidates, making sure the process goes relatively smoothly, and of course, paying them fat industry++ salaries. Outside of this structure you experience a world where you reach out to so many people and get nothing but silence, blank stares, or polite dismissals. Your CEO tells you that you've gotta sell, and asks for your sales pitch. And often, faced with a string of failures and pressure to grow, you land on the need to do something "cool" with technology to up your "cool factor."

You know what I mean. Chase the buzzwords: microservices, Go, big data, event-driven, reactive, functional, etc etc etc. The only way engineers will want to come work for me on my relatively straight-forward application development is if I give them the carrot of Cool New Technology!

I have learned a few things over the years, and one of those things is that usually engineers that are only interested in Cool New Technology are not going to stick with your Boring Business Problem for long enough to be worthwhile, or worse, they'll stick with it long enough to leave a trail of one-off solutions that no one else on the team understands before they walk away and leave you holding the bag.

If you're tempted to reach for Cool Tech, then I'm going to guess that you're not at a company where the primary challenge is purely technical or scaling. Instead, the interesting problem that your company is solving is almost certainly a combination of a) figuring out how your business, possibly the first of its kind, is going to survive, and b) growing, changing, and evolving to create a functioning organization. Once your company is successful, many of the problems that seemed trivial become surprisingly challenging to solve at scale, but in the early days oversolving with cool tech only leads to distraction from tackling the real challenges.

So resist the urge to adopt any technology for the cool factor of recruiting. Instead, look for people who want to own big parts of the system, who are interested in the business, who are really passionate about the customers you're serving, who are looking for leadership opportunities. Don't undersell the opportunity you have just because it isn't Cool New Technology. Be honest with yourself about the real problems that make you excited about the business that you're in, and that's where you'll find your best sales pitch.

Thursday, May 21, 2015

Entrepreneurial Gap

I recently came across a blog post that mentioned an intriguing concept, the "entrepreneurial gap." The idea is straightforward: give people more accountability than they have the direct resources to accomplish.

In many organizations employees are generally held accountable only for things fully within their control. So while the CEO has both all the resources of the company and the ability to make decisions that cause major tradeoffs (sacrificing profit for growth, for example), a manager of a team is only given projects big enough for that team to accomplish and goals big enough for that team to meet.

The entrepreneurial gap comes in when you give people bigger accountability than they have the direct ability to execute against. This is in many ways the classic startup move. You hire people and give them enough freedom to accomplish whatever they can manage to accomplish. Because early startup employees tend to be very entrepreneurial they often find a way to do just that, to gather support from other people in the organization and align efforts to produce outsized results. I think this is an important element of a growth business, something that inspires great people to work for a company, and produces a really fun culture to work in.

But, there's a trap.

The entrepreneurial gap works incredibly well when you give teams aligned goals. If you read the first case studies in the paper, you will see that in all of the examples the CEOs focused the company on a few goals, and in fact in all three examples there was a very strong emphasis on the customer. In companies where everyone is working ultimately towards the same goals, the entrepreneurial gap is awesome because it encourages people to work together and identify opportunities and efficiencies, especially cross-functionally, that may otherwise be lost.

Unfortunately, it is just as common to see companies put in place both an entrepreneurial gap and too many or misaligned goals. When you are in direct competition with your peers for scarce resources and you are not going to be graded on the same outcomes, the entrepreneurial gap produces a toxic environment of politics and power plays. Perhaps the best idea sometimes wins in these situations, but more often the best political players rule the day.

So what do we take away from this?

Organizational alignment is important because it lets you successfully ask more from people than the resources they have at hand. Without organizational alignment, you get political maneuvering. Without stretching people beyond their direct control, you get a lack of collaboration and creative cross-functional engagement. Set the right priorities and give people the freedom to stretch to them, and you will see the full potential of your organization come to life.

(The paper again: The Entrepreneurial Gap: How Managers Adjust Span of Accountability and Span of Control to Implement Business Strategy)

Saturday, March 21, 2015

Get Curious

The best advice I’ve gotten in recent years is simple. Get Curious. For those of you who don’t know me personally, I can be rather aggressive, especially with things I don’t trust. Whether by virtue of personality or experience, my instincts always lead me to attack and take apart the unknown. This is in some ways a good trait in an engineer, after all, the unknown is what causes you to get paged at 2am. Unfortunately, when it comes to interpersonal interactions, aggressively looking for flaws in things you don’t understand looks less like smart defensive behavior and more like obnoxious trolling.


So how can one honor the need for clarity without attacking and picking apart things in a way that causes others to get defensive? The tactic I’m using is “Get curious.” Instead of assuming that the other party hasn’t thought of your objections, try to understand deeply the context in which they are making their statements. Why do they seem to want to do something that may sound illogical to me? What is their perspective on the circumstances? What have they tried to do already?


A few years ago I wrote about “Yes, and...” I won’t lie; I still struggle to this day with “yes, and”. For me, “get curious” is a bigger picture version of "yes, and," one that I find easier to keep in mind. If you’re always curious and looking for the context, you’re open to the possibilities that are out there. The possibility that you’re missing something important. And that openness makes people want to work with you.


“Get curious” could be the mantra for the learning organization. Curiosity in the face of failure, instead of blame, leads to a safe environment for people to fail, and lets us discuss our failures. Curiosity leaves us open to new ideas. Curiosity enables us to recognize our differences and embrace them to create a stronger whole.


So for those of you who are quick to blame and judge and want to get out of this habit, get curious. For those of you who feel like your team is afraid to take chances and fail, get curious. Even for those of you who are afraid that you’re being too nice and are not challenging your team or colleagues enough, instead of getting mean or aggressive, get more curious. You’ll be surprised at how much more effectively you can create change when you approach situations with open curiosity.

Sunday, February 8, 2015

On the role of CTO

I get asked frequently to define the role of CTO, and I thought I would share some of my thoughts.

First, let's start by talking about what a CTO is not.


1) CTO is not an engineering role. CTO is not the top of the technical ladder, it is not the natural progression for engineers over the course of their careers to strive to achieve. It's not a role most people who love coding, architecture, and deep technical design would enjoy doing.

2) From 1, it follows that the CTO is not necessarily the best engineer in the company.

Now, with that out of the way, what is a CTO, if not the best writer of code and natural conclusion of the engineering ladder?

The challenge with defining CTO is that if you look across the folks who hold that title you will see many different manifestations. Some are the technical cofounders. Others were the best of the early engineers. Some started at the company with the title, while others such as myself were promoted into it over time. Some became CTO after holding the title of VP of Engineering. Some focus on the people and processes of engineering, hiring/recruiting. Others focus on the technical architecture, or the product roadmap. Some are the face of the company to the outside technical world. Some CTOs have no direct reports, others manage the entire engineering organization.

In looking at these examples the best definition that we can make is that the CTO is the technical leader the company has in its current stage of evolution. To me, that is rather unsatisfying, and missing the hardest part of the job. I believe instead that the CTO should be the strategic technical executive the company needs in its current stage of evolution.

Strategic: Thinking about the longer-term, and helping to plan the future of the business and the elements that make that possible

Executive: Taking that strategic thinking and helping to make it real and operational by breaking down the problem and directing people to execute against it


So, what does a CTO actually DO?


First and foremost, a CTO must care about and understand the business, and have the ability to shape business strategy through the lens of technology. The CTO is an executive first, a technologist second. If the CTO does not have a seat at the executive table and does not understand the business challenges the company is facing, there is no way the CTO can guide the technology to solve those problems. The CTO may identify areas where technology can be used to create new or bigger lines of business for the company that align with the overall company strategies. Or the CTO may simply ensure that the technology is always evolving to anticipate and enable the potential futures of the business and product roadmap.

No matter what, the CTO must understand where the biggest technical opportunities and risks for the business are and focus on capitalizing on them. When you see a CTO focused on recruiting, retention, process and people management, that is because it is the most important thing for the technology team to focus on at the time.

Strong CTOs also have significant management responsibility and influence. This does not always mean that they are deeply involved in the day to day management, but part of maintaining your ability to shape the direction of the business and the business strategy is the ability to put people behind solving the problems you believe will impact the business. Other executives will have ideas and needs for technology. The CTO must protect the technology team from becoming a pure execution arm for ideas without tending to its own needs and its own ideas.

Things get tricky when the team grows to be very large, and the CTO starts to hire VPs to manage all the people. Many CTOs give up all of their management responsibilities to their VPs, sometimes going so far as to not even have the VPs reporting to them. It is incredibly difficult to maintain influence and effectiveness as an executive with no reporting power.

I saw this clearly at my previous employer, where the most senior members of the technical staff in large business areas would often hold the title of CTO for that area. These people were always highly respected and technically capable. They understood the business and the technical challenges for the business, and were often called upon to help inspire the engineering team and help with recruiting.

Their downfall was that they often lacked the direct management oversight of any teams, and because technology was frequently viewed as an execution arm, they did not have much strategic influence.

If you are a leader with no power over business strategy and no ability to allocate people to important tasks, you are at best at the mercy of your influence with other executives and managers, and at worst a figurehead. You cannot give up the responsibility of management without giving up the power that comes with it.

The CTO who does not also have the authority of management must be able to get things done purely by influencing the organization. If the managers won't actually give people and time to work on the areas that the CTO believes are important, the CTO is rendered effectively powerless.  If you give up management, you're giving up the most important power you ever had over the business strategy, and you effectively have nothing but your organizational goodwill and your own two hands. Hope you're really a 10X engineer!

My advice for aspiring CTOs is to remember that it's a business strategy job, first and foremost. It's also a management job. If you don't care about the business your company is running, if you're not willing to take ultimate responsibility for having a large team of people effectively attacking that business, then CTO is not the job for you.

Special thanks to my editors Amanda Peyton, Cate Huston and EIC ChrisK.