Four things that effective managers of large teams do well

If you are reading this, you are either at the stage where your company is experiencing significant growth, or you have just found yourself responsible for more people than you can comfortably manage. Perhaps with multiple managers below you.

I was in a similar situation a few years ago. Fortunately, I had experienced managers to learn from and a very patient team. Over time, I realised that managers who were effective at managing at scale did the following things,

They built meaningful relationships

Effective managers were really great at building relationships, not just with their direct reports but also others in their organisational unit. More importantly, they cultivated relationships with stakeholders from other departments and business units.

Deep meaningful relationships and trust take time to build, and they happen outside formal meetings in the many micro-interactions we have with others. Effective managers invested time in having these micro-interactions in between meetings, not just to get to know people but also to get a pulse of their organisation — what worked for people, what didn’t, what they looked forward to, what they didn’t; and what they thought about their company and its culture.

They Identified key metrics

A large organisation has multiple teams, each team working on a different problem or area within the business. It can get hard to keep track of what each team is doing and more importantly whether everyone is aligned and working towards the same goals.

Effective managers helped distil the expected output into a couple of key metrics that the team can work towards. They created a culture of transparency where team metrics were visible within the organisation. This not helped foster a sense of oneness but importantly helped members of different teams understand each other’s goals and pressures and created an organisation that was fair and reduced company politics.

They constantly communicated

Effective managers know that the #1 problem facing organisations is that of alignment — people working towards the same shared goal. It isn’t enough to state the company goal once during an annual all hands and assume each department, team and individuals understanding will align with it.

They not only communicated goals and changes to prioritisation but also their fears and the risks they perceived. They communicated transparently and allowed everybody to voice their concerns or opinions. They used multiple channels of communication and communicated the same message over and over because they knew that sometimes it takes a few tries for the message to stick. They brought people along on their decision-making journey. They built trust through communication.

They empowered others

No matter how efficient you are, you only have 24 hours in a day. To truly operate at scale, you must create teams that are autonomous, understand their goals and have the freedom to execute on the “how”.

Effective managers know this and constantly work to eliminate themselves as the bottleneck. They invest time in defining boundaries on what needs to be escalated and what doesn’t and then empower managers and teams to make decisions within their teams.

While effective managers delegate and empower others, they hold them accountable as well. They create channels of communication, so that they are still in the know of what’s happening and can step in when required.

In closing

I learned a lot from the people I worked with, learning to avoid mistakes they made, while making a bunch of new ones. I use these four key areas today, and I hope I can share some of what I learned with you.

The only way to structure your engineering teams

Four years ago, I wrote an article on the different ways one could structure their software engineering teams. The article turned out to be rather popular and it’s one of my most viewed articles.

Since then, I have had to rapidly scale teams from zero to 50+ software engineers four times across organisations like Facebook and the OVO Group. I have learnt a lot and I now firmly believe that there’s only ONE way to structure an engineering team.

Build teams around products

Gone are the days when products were built using Waterfall and new versions came out every few years. Today, many companies do continuous deployment where software is updated multiple times a day.

Speed to market has become the #1 differentiator of a successful company. How then, can you create that speed?

By intentionally building teams around products (or micro-products) with all the functions needed for the product to succeed within the team. For instance, if one was building a simple app, the ideal team could contain the following functions,

1. Software Engineering — to build the product.
2. Product Management — to help prioritise what gets built next.
3. UX — to shape the user journey within the app.
4. Design — to make the app look good.
5. Data Science — to provide feedback on what’s working, what isn’t and inform the product direction.

Why build teams around discrete products

When you build a team around a product, a number of wonderful things happen,

1. Team gains autonomy

Once the goals and metrics are set, The team is empowered to make decisions as long as the decision moves the metric in the right direction. Almost all decisions can be made within the team without going to an external stakeholder, reducing turnaround times. The team also makes better decisions as context isn’t lost between the team and an outside decision maker.

2. You build better teams

When people are working together for a while, they learn to work more effectively as a team. They develop trust and learn each others strengths and weaknesses. They have each others backs and stick with each other through times good and bad. This team not only is better able to predict their delivery but remains together for longer with lower attrition. (As a manager, you are responsible for creating a balanced team with people who complement each other)

3. Team has ownership and feels accountable

With this structure, the team feels complete ownership of the products they have built. They aren’t just building a component and passing it on to another team, like an assembly line, but are building the whole product. The team starts to really care about their product — it becomes their baby. This change in mindset not only improves quality and the speed at which things are delivered

Where do you begin?

If you are a large company and your teams aren’t organised in some kind of a pod structure, this is going to be a hard transition. It may be easier to seed this structure on a greenfield product.

There are three things you need when building a team around products,

1. Set goals and metrics

Be clear on the outcome you want the team to achieve. Then create metrics that help track against that outcome. Set intermediate goals or milestones. Be sure to take your team on the journey so they don’t feel that the goals have been top-down. You want a team that feels in control of its destiny. Let them be a part of setting it.

2. Hire the right people

It’s ultimately people who will drive this and make it successful. You need to ensure that the team is balanced with the right archetypes and skills needed. These factors change with the stage of growth you are in and the nature of product you are building.

Importantly, keep your teams small. I’d say the ideal size is 6 people with about 3–4 developers. Any more and decision making suffers.

3. Get out of the way

Most importantly, let the team run with the goals. Let them choose the what and the how of what they want to build. Tempting as it may be to step in, you hire smart people so that they can tell you what to do and not the other way around.

Your role as their manager changes and you will spend a lot more time on the people side, ensuring that the team has all the resources it needs to be successful and that they have autonomy in decision making. You will also be focused on ensuring that the outcomes and goals remain consistent with the rest of the business over time and that the metrics still help you track against those outcomes.

In closing

If you are running software engineering teams, and don’t currently run a pod structure based around products, do give it a go. It creates happier motivated teams which deliver better software with better predictability, reduces collaboration and management overhead ultimately leading to better business outcomes.

Code does not get rusty

“We need to rewrite this system from scratch”

I heard that statement a few years ago at a startup I was working with. I’m sure you’ve heard that before. Perhaps you have said it yourself. This statement always has excellent reasons behind it.

Rewriting something that is live and works is a scary prospect. It’s a lot like rebuilding an airplane while it’s in flight. Software engineers invariably introduce tech debt as changes and additions are made to the system.

At some point the tech debt seems unsurmountable. The team feels like it’s spending more time battling the system architecture rather than releasing value. The symptom is true. The reason, however, always varies and it’s important to get to the root cause.

I wanted to understand the issue and its severity and got them together.

The first session

The first session was to really understand the WHY behind it. I am a huge fan of the 5 Whys system. It tries to find the root cause of a problem by repeatedly asking the question “Why?”

During the first session, I listened a lot and tried to understand their concerns. The session itself was done in a way one would do end of sprint retrospectives, with everybody writing stuff down on post-its and later explaining their post-it notes.

I asked a bunch of qualifying questions when the reasons seemed vague and towards the end I was able to validate the following,

There were areas of the codebase that the team was uncomfortable with. Changes to those areas had historically caused issues elsewhere — from a broken build to severe data corruption downstream.

This was due to a few factors,

1. Knowledge attrition — The business had been restructured a few months ago and with that, a lot of the knowledge had left the business.
2. Poor ownership — Some parts of the code had no current owner. Teams owned part of the codebase and there were gaps of no ownership
3. No testing — My biggest bugbear, the code lacked testing of any kind.

The good news is that all of this was fixable.

The second session

I set up the second session the following week. Before the session, I worked with a few team members on how we could best structure the teams so that there was complete ownership of code.

During the second session, I acknowledged that most people were new to the codebase and that there were areas that were poorly understood. I introduced the new team structure which created owners for all parts of the code.

On a parallel track, service design workshops were set up to understand what the current system did and what was the expected outcome for the business. This would then feed back into the development teams to help them fill in that knowledge gap.

The most important thing however was to improve test coverage. This would do two things,

1. Bring confidence in our development and release process and
2. Most importantly, improve knowledge within the teams through writing tests.

Until we achieved a test coverage we were happy with, every sprint would have 20% time set aside for each developer to add test coverage in their areas. I also set up a series of sessions to introduce TDD and Testing for people who weren’t familiar with it.

The third session

In this session, I wanted to set the stage for the team to work on improving the existing platform. Rather than build a parallel platform which would mean having two platforms to support and the feature disparity between the two, I believe the right way is to chip away at the current platform and replace existing bits with newer ones.

The service design workshops identified a bunch of areas that needed improvement or were bottlenecks. This was both from a business and a tech perspective. We created a prioritised list based on impact v/s difficulty and this would feed our backlog to release the most business value in the coming weeks.

We also identified areas which were working well and wouldn’t need touching, at least not for now. It turned out to be a large percentage, something that did surprise us.

Code does not get rusty.

Unlike physical things, code does not undergo wear and tear. It does not need maintenance to keep it running smoothly. It executes in exactly the same way every time it is run — irrespective of whether it’s the first run or the billionth one. In fact, with improvements in microprocessors and compilers, that code executes quicker today than the day it was written.

So dear reader, if you are thinking of rewriting an existing system from scratch, do have a good think. Focus on areas that are broken, and for the rest, “if it ain’t broke, don’t fix it!”

Building your Communication Strategy

Today, I want to discuss communication, which I believe is the most important aspect of your role as a people manager.

As the number of people in your organisation grow, a number of things about your day to day change,

1. Less face time

You invariably have less face time with your individual reports. This means you may not always be know as much about what is happening in their work/personal lives as you once did. Only highlights and lowlights are shared during 1:1 sessions.

New joiners may start seeing you as a “boss” and someone whom they can’t necessarily approach.

2. Missed opportunities to provide context

Less face-time means the limited communication that you do have with people can have one of two effects,

1. They carry more weight than they normally would. Anything you say will be easily misconstrued. People try to find the hidden meaning behind statements that you make. Not because of bad intentions, but due to a combination of the recipient’s emotional state and the brevity of the face-time.
2. Or they will be forgotten. This time, because the recipient is uncertain whether you really meant it or was it you merely trying to be helpful. Or because “if it was important, I would have heard it repeatedly or from multiple sources”.

3. More moving pieces

You will find yourself responsible for more projects and processes, requiring you to context switch more frequently. Unless you are very clear on what you want from a project or process and make that explicit, things will be lost in translation. Spending time thinking about your message and writing it down helps you organise your thoughts and gives you clarity.

The objective of communication

Communication helps achieve,

• Alignment. It’s important that people always know what you are most passionate about and what you care about the most. This helps them make local decisions on what direction to take when they have a bunch of possible routes. I do want to stress that this isn’t a top down thing, but setting vision is something a leader has to do and get others behind the vision. (Your vision may well be wrong and then the smarter people you have hired will be able to point it out and you have that aha! moment.)
• Consistency. When people get the same message from multiple sources — peers, managers, others in the org — they begin to trust the message. They also begin to trust the sender or senders of the messages. It reinforces the message and makes it important.
• Reduced ambiguity. Engineers (usually senior) have enough technical ambiguity to deal with on a daily basis. They need to make architectural, tooling, process decisions. You can help reduce any ambiguity around vision, strategy or people and help your engineers focus on their jobs. A side effect of this is that you are able to reduce churn/attrition rates.
• Building consensus. In larger organisations, teams often work with other teams, and there is a significant amount of collaboration required. Collaboration across teams is always challenging. Effective communication helps in influencing other teams, giving them a heads up on potential collaboration opportunities/conflicts, increase internal adoption of your product or just letting the rest of the organisation know what a great job your team has done.
• Manage Change. Change is an inevitable part of the modern workplace. People get reassigned, projects get cancelled or pivot, and then, there are always external events. Communication helps set expectations, reduce rumours and help build confidence in the future.

Creating your communication strategy

So how do you go about building your communication strategy? One way to begin is to answer the following questions,

1. Start with Why

Why am I communicating? What outcome do I hope to achieve?

Establishing the why is the first step towards building your communication strategy. What are you really trying to achieve? If you manage engineers, some of the why’s could be to,

• Establish Priorities
• Communicate Progress
• Highlight dependencies and blockers
• Gain alignment
• Set expectations
• Get input on processes and ideas
• Build strategy
• Re-iterate vision or purpose

Some why’s will be tactical and require a small amount of communication. Others will be strategic, need repetition and will be formed from many individual pieces of communication over a length of time.

2. Move on to the Who

Which groups of people do I want to communicate with? And with what frequency?

Next, you need to establish who you want to communicate with? Some examples of these are,

• Your team(s)
• Your peers
• Management
• Stakeholders
• The whole org

3. And then the How

What channels (face-to-face, email, slack, yammer feed) should I use for this communication? Are they effective? Do I need to change something?

After establishing the who, you need to think about the channels of communication. It should almost never be a single channel. Repetition through multiple channels help cement the message.

For instance, if you are introducing a new way of doing things, you could speak to key stakeholders and managers first, then announce it in your newsgroup, followed by a presentation/QA session for everybody affected. This would need to be repeated after a period of time, say a month, to (1) get feedback after the new process has been used in practise and (2) to repeat the (possibly amended) message and make it stick.

4. Finally, the What

This is usually the easiest bit of the process. It involves generating the actual content of the message you want to deliver.

5. Evaluate your results

How do I know people recieve my message?

Next, it’s important that you have set up means to check if the people on your teams are (a) all receiving your message and (b) are interpreting the message as intended.

Face to face is perhaps the best way of really understanding if both (a) and (b) happened. If you team is small enough, you can do a face-to-face with every one on your team. If not, use a random subset, making sure to include people you don’t interact with often.

6. Iterate

Am I doing an effective job at communication? In which groups is it easier to land my message? In which groups is it the hardest? Why?

As with everything, you need to analyse the outcome. It’s a constant process of self-improvement, learning from mistakes and trying to get better every day. It’s incredibly hard too, and tremendously satisfying when you get it right.

Tips

1. This process will seem like a time sink initially, do keep at it.
2. Block time in your calendar for this task. Twice as much as you think you will need.
3. Have “defined” channels for certain kinds of communication like “general news”, “important decisions”, “strategy”. Set expectations for each channel.
4. Repeat your message. In multiple ways. Through multiple channels. No, people will not get annoyed by it. If people do, then you are doing a good job.
5. Establish cadence of communication. So people begin expecting updates with that frequency.
6. Download the communication cheat-sheet.

Feedback

Finally, dear Reader, what are your thoughts? What do you do to land your message?

Engineering Archetypes

Today, I’d like to talk about engineering archetypes. Engineering archetypes or engineering personas are a skill-agnostic way of classifying engineers. I find it’s a handy tool for managers to answer these questions,

1. How do I think of different engineers on my team? What career paths within software engineering make sense for them at their current stage of development?
2. How do I best structure my team? What personalities are missing on my team? What have I overindexed on? Do I have a healthy tension on my team?
3. How do I compare engineers working on different tech stacks (ex. front-end, back-end) and normalise their impact to the business?

For software engineers, in large organisations, it’s a handy tool to answer questions like,

1. What kind of work do I prefer to do? Do I want to grow in that area?
2. What kinds of projects or teams best suit my archetype?

Before I begin, I’d like to say that every individual engineer is unique and this isn’t presented to take away that individuality, but to act as an aid to help build a diverse and complete team. Also, an individual isn’t stuck with an archetype - it can change as their preferences change over time or it can change situationally, based on their team’s needs.

The archetypes

Code machines

As the name suggests, code machines are really good at writing good code and well versed with their choice of programming language and technologies. They are able to produce massive amounts of code v/s their peers. And are usually experts in their area.

However, they tend to dislike what they perceive to be time wasters – like meetings and interacting with people.

Fixers / Optimisers

They love diving deep into the problem and get tremendous satisfaction from making the least amount of change while maximising impact. You will find them disappearing for an amount of time and coming back with a (usually) successful outcome.

However, they dislike day to day iterative development and tend to get bored if they find themselves doing repetitive tasks.

Product engineers

They are constantly chasing business value, and love delivering products. They aren’t afraid taking shortcuts or making short-term engineering decisions if it leads to product in the market. They are usually very comfortable presenting ideas to non-technical stakeholders.

However, they dislike focusing on platforms or over-engineering.

Architects

They love architecting systems and drawing block diagrams. They enjoy working on complex products with many systems and moving parts.

However, all this may come at the cost of actually writing code.

Theoretical engineers

As the name suggests, they love working on abstract concepts. They may spend a lot of time reading research papers and using a pen and paper. They may find algorithms and mathematical concepts more appealing.

However, they aren’t the best at writing production quality code.

Systems/Infra engineers

They really know their physical machines. They have an innate understanding of how the kernel works, how memory allocation happens, the pros and cons of different file systems and storage media, cpu architectures. They prefer to work on low level system code or build services that other developers would use like databases. Some of them may enjoy devops.

However, they don’t enjoy building user facing products, and dislike the frequent (and in their opinion irrelevant changes) to UI and user workflows.

Mentor engineers

Potential future managers, mentor engineers love working with people on their and other teams with an aim to improve them. They are usually seen doing pair programming or mentoring junior engineers. They’d be the first to give feedback on issues that affect the team and come up with improvements to their teams ways of working.

However, they dislike being on projects where they have limited interaction with people or where they are the only developer.

Conclusion

There are no right or wrong archetypes. People are on a spectrum across these archetypes. Nobody is a 100% one or the other.

There is no right composition that makes up a team. Different organisations will need a different numbers of each archetype and this will change over time.

What do you think about these archetypes? How would you use this concept? Are there any archetypes that I have missed? Do let me know in the comments!

Structuring your Engineering team

At Affectv, we grew extremely rapidly over the course of 3 years. Our headcount exploded from 2 to 70 and the Engineering & Product teams more than tripled each year. This is a compilation of what I learned optimising product delivery and reduce bottlenecks through trying different team structures.

In the beginning — No model

When we added our first few engineers, we didn’t have a structure in place. All engineers reported directly to me. I was their manager, tech lead and the systems architect.

As our business grew and we started adding more engineers, this model just couldn’t scale. It was time to try something else. Since all the companies I had worked in before had used the time tested functional model, that is what I decided to adopt.

The Functional model

This is the the oldest model and is heavily used not just in software development, but in pretty much all areas ranging from the military to manufacturing and pharmaceuticals.

In this model, you divide the team by function. For instance, you may have the front-end team responsible for web development, the back-end team writing server side code and the data-science team responsible for analysing the data.

There are a number of advantages to this model,

1. It’s simple to understand. People know exactly what’s expected of them and their area of ownership.
2. Fosters experts. Over time, people gain expertise in their area and experts from outside the company can be hired to do specific functions.
3. Common tools and processes. It’s easier to standardise on platforms, coding standards and processes, thus improving productivity.
4. Management. Reporting lines are simple and performance management and KPIs become easy within each function.

However, there are disadvantages too,

1. Silos. An unintentional consequence of this structure is that your teams become siloed. Instead of focusing on product, sprint planning discussions tend to focus more on the APIs/interfaces needed between the various teams. At best, your processes change from agile to waterfall. At worst, it becomes a blame game. Then again, who is responsible for the areas that don’t clearly fall with their defined “functions”?
2. Lack of ownership. Another consequence is that engineers do not feel ownership of the product anymore, and ownership moves to their code or function. This problem exacerbates for functional teams building multiple products.
3. Slow Development. With growth in product features and teams, development slows down as communication between teams is harder to manage. Beyond a certain team size, this is perhaps the hardest structure for agile development.
4. Measuring team impact. How does one measure the performance between the Backend, Data and Frontend teams in a fair way? It becomes a non-trivial to set performance expectations with the team that align with the business outcomes.

When I first created the Web, Server and Data teams, it was really great experience for the company. Having some structure, where previously there had been none, was really great! Engineers now knew what they were responsible for and were excited at becoming domain experts. Our sprint planning processes changed.

As the team sizes increased and we added a bunch of products, it became harder to get things done due to the disadvantages mentioned above. The time-tested functional model wasn’t suited to a startup trying to iterate quickly and try new products.

The Pod model

It was time to try the Pod model — where one built a team around a specific product. An example of this, could be a product team with a DevOps engineer, PHP developer, HTML developer and a Javascript developer responsible for building a client’s website.

Companies like Facebook use this model quite extensively and I was keen to understand if it would help with our situation. Before jumping to it, I wanted to learn about it’s drawbacks.

1. Line Management. Line management is a challenge. If line managers are responsible for a pod, they may not be able to fairly evaluate all the different functions. If line management remains functional, the line manager will not be part of every pod and thus lack context.
2. Built-in Redundancy. Since functions are replicated in each pod, the wheel may be re-invented in each pod. Show & Tell sessions are useful to increase flow of information and reduce work replication. Also, headcount may have to be replicated across different pods, compared to the Functional model where one person would’ve been able to support both products.
3. Standardisation. Unless different pods talk often, use of frameworks will diverge. Also, pods might choose to use esoteric technologies. Care must be taken to ensure that technology fragmentation is managed.

The benefits as I saw it were,

1. Autonomy. Pods can be made to operate as autonomous, self-directed units with minimal red-tape.
2. Self-sufficiency. Each pod was self-sufficient as they had exactly the people and skillsets needed on the pod.
3. Flexibility. Pods are immensely flexible. They can be formed and dismantled as per business needs. They are technology agnostic. And immensely replicable.
4. Replication. As your business grows, and there are new product ideas, you can simply add more pods.
5. Measuring team impact. Team impact equalled product performance in the market, can be tied to revenue and is so much easier to measure. Each pod can operate it’s own P&L.

We divided our teams into various pods, each responsible for a single product. We ensured each pod had a Product Manager and a Tech lead/Manager who would be jointly responsible for its output.

The results were pleasantly positive. Morale improved significantly, more ideas were generated and the pace of production increased. Product Managers were happy as they knew exactly what resources they had and didn’t have to negotiate for resources. Meetings became more efficient and less frequent for most people. The slight downside was that the team leads/managers had the additional responsibility to ensure knowledge flowed between pods and common standards were maintained.

Why is structure important?

As your business grows and you add more engineers, planning your growth becomes essential. Growth, if not managed carefully, can quickly get out of hand resulting in loss of productivity and morale. Things that were crystal clear become fuzzy all of a sudden. For example, take ownership — when a team is small, ownership is apparent. Decision making quick. And adding new features easy as the the system is simpler and most people can keep it in their head.

So if you feel like your current team structure isn’t working effectively, and your team is demotivated and productivity has plummeted, perhaps its time to try something different?

There is no perfect team structure. Companies, as they evolve must adapt and change their structure and processes to maximise their teams productivity and happiness. This is the lesson I learned.

What I look for in my hires

Building a high-throughput engineering team is hard. You need to structure the team with the right engineering archetypes and ensure that each engineer has certain essential qualities.

Early on in my career, I had over-indexed on how “smart” someone was. I used to be in awe of people who could solve the toughest computer science and math problems – problems which took me days to get my head around. During my working years, I realised that the following virtues were more valuable,

Predictability

The growth of any business relies on its predictability. For sales-driven businesses, its predictability of revenue. For engineering-driven businesses, its predictability on product delivery. All functions depend on it – from marketing collateral and customer outreach to internal and external training.

I learnt over time that the best engineers weren’t the ones who were the smartest, but the ones who were most predictable. If I give a person a task, will that person be able to complete it correctly and on time?

Empathy

Humans aren’t well oiled machines. There are always times when we need to compensate for others and have others compensate for us. Having people with empathy for their peers and for the business creates momentum on the team and helps with its morale.

Team morale is a fickle thing. It can quickly snowball in either direction. Having a pulse of the team; especially when it comes directly from empathetic individuals can alert you to situations and help mitigate them. As their manager, no matter how grounded you think you are, you will quite often fail to notice certain things.

Vocal, empathetic people who bring things to your attention, who have the courage to stand their ground can make a humongous difference to the team. These are the people who always volunteer to help. Who reflect on themselves, the needs of the team and the needs of the business.

Autonomy

The third quality is Autonomy. Can I give the person a job to do and then forget about it, knowing that the person will either,

1. Get it done and report back to me OR
2. Raise it early with the issues they are facing.

Or do I have to keep following up with them? How much hand-holding do they need?

I love autonomous people. I must admit, I am really bad at micromanaging people. I hate it. Firstly, because I personally don’t enjoy being micromanaged. Secondly, micromanagement imposes an additional workload on me and a further drain on my already limited time.

Ask yourself - Is the best use of your time looking over someone else’s shoulder? Or thinking about the next most awesome thing?

So how do you identify autonomous people? I try to assess this in face-to-face interviews. I usually ask them to walk me through a few large problems/challenges that they faced at their previous job or university and ask pointed questions on the exact nature of the difficulties faced and how they went about resolving them.

Adaptability

Roles invariably change over time. Responsibilities change, tasks change and expectations change. In the fast paced world of startups, a job description becomes obsolete the moment the new hire steps through the door!

Under such circumstances, the people who are most adaptable to the changing needs of the business are the ones who have the biggest impact. It is really hard to predict the trajectory of a startup pre Series-B, and during this usually tricky time it’s your highly adaptable people that help pull the team through!

I look for adaptability with my hires. How open are they to different programming languages? How happy are they to roll their sleeves and dive into the problem? How eager are they to move between stacks? At the end of the day, it doesn’t really matter if you used a wrench to hammer the nail in – so long as the building is built!

Growth Potential

The aim of any good manager or CTO should be to make themselves obsolete through coaching and transfer of skills. In any team structure, there should be a clear path for progression for each individual.

When I hire, I look for candidates who can grow quickly, can challenge their managers and who can take on more and more duties over time.

1. It makes my job easier as I can delegate some my tasks. This frees up my time to focus on things that have the most leverage for the business.
2. It sends the right signal to the team, saying, if you are able to grow, you will get more responsibilities and promotions.
3. I am not a huge fan of hiring externally when there is talent available within.

Ownership

Ownership is about,

1. Feeling responsible for the product’s success or failure.
   1. Feeling especially responsible for the product’s failure.
2. Understanding the WHY behind the product and the business problem it solves.
3. Driving the future direction of the product.
4. Knowing who all the stakeholders are and effectively communicating with them.
5. Having the right level of technical understanding of the product.

Think back to your team. How many people truly own their respective areas?

Problem Solving

In the original series of Star Trek, Captain Kirk often quotes, “I do not believe in no-win scenarios”. This is a really powerful statement and something I wholly believe in. No matter what the situation, it is salvageable; and then it is winnable. All you need are people with unconventional problem solving ability.

During my 10+ years working at multiple startups, I have been amazed multiple times with how many dire no-win situations were turned completely around due to the resilience of problem solvers. And then in some cases where situations could have been turned around but weren’t because of the lack of such people in the organisation.

In closing

I have tried to build teams which people with a healthy dose of the above qualities. Looking back, the teams that failed to be high-performing were because the individuals didn’t exhibit these qualities.

I am really proud of my team here at Facebook. We are working on some really hard product problems and trying to transform the way advertising currently works. With the people on the team ticking all of the above, I am really confident about getting there :-).

2016 is going to be an amazing year!

What I learnt about hiring

I have been the CTO of Affectv for 3 years now. During this time, we have grown from 3 to 40 people, more than doubling our Engineering team in the last eight months. In the next six months, I would like to double this number again. I have made mistakes, learnt from them and avoided some. This is a compilation of some of the things I learnt.

Writing the job spec

This is by far the most important stage of the hiring process. When I wrote my first job spec, I didn’t do a very good job. It looked exactly like the other job specs out there, listing a bunch of skills that I was looking for.

I only considered what I wanted the candidate to do right now. I didn’t consider projects changing, what the candidate’s role would grow into and what exactly the candidate would be responsible for. Understanding what it is you need the candidate to do now, then in six months’ time and even in a year’s time goes a long way in defining the rest of the interview process.

Take the time to write that down. You will realise you have made assumptions on the skillsets you require, or you aren’t clear on what the role will become once the “project” is over. And once you have clarity on the role, you will be able to better assess a candidates’ CVs and also articulate it during the interview process.

Very often, the job spec is the first thing candidates see – whether they are actively looking or have been approached by a headhunter. This is the first glimpse of your company. This is where you set the stage. What kind of company are you? Are you a fun place to work at? Is there something that sets you apart from other companies? Are there any special benefits that you offer?

I use the job spec to sell my company. I use it to sell the role. And most importantly, use it to tell candidates exactly what I am looking for.

I use the following format on a Job Spec. I believe it is concise and gives a good idea of the company and the role.

• About the company - This is where I describe what we do, what makes us different and some of the challenging problems we are attempting to solve.
• About the role - Here, I describe the role. What do I expect this person to do? Who will they report to? Which teams will they interact with? Will they manage a team? If so, what kind of team will they manage (Front-end, Backend, Sales, …)
• Responsibilities - And lastly, I list the person’s responsibilities. Will they be responsible for a set of projects? A team? For creating reports? For PnL?

Recruiters

Most CTOs I have spoken to have a love-hate relationship with recruiters. When I began recruiting for the first time, working with recruiters was quite hard. They were necessary to cast a wide net, but the logistics of dealing with them drove me up the wall. I would be sent a hundred CVs a day (across 5 recruitment agencies) and would be called incessantly regarding those. From looking at the CVs coming in, it was obvious that the recruiters had a limited understanding of the job spec and the kind of candidates I was looking for.

Something was seriously wrong. Surely, filtering candidates was their job, wasn’t it? Why was I spending over half of my day going through CVs and rejecting 98% of them? It was time to change things. I got in touch with more recruiters, spoke at length with them on the phone or in meetings, explained what I was looking for and made it clear that if I started rejecting more than 50% of the CVs, I would not be working with them anymore. I asked them to come back with any questions that they had in a week’s time, after which I would not be clarifying the job requirements.

During these conversations, I had interesting reactions – some recruiters were quite offended. This wasn’t the way they worked with other companies! They had to send me lots of CVs because “you might miss some good candidates!” And then, other recruiters weren’t. They were patient, asked intelligent questions and understood that I couldn’t allocate so much of my time to recruiting, and that it was better to send a select few CVs. They were the ones I wanted to work with.

Over the course of a few weeks, I narrowed down the number of recruiters to two. I created a process – CVs would come in on Monday and Wednesday mornings and I would respond on Wednesday and Friday afternoons. Interviews would be set for Tuesdays and Thursdays the following week.

This system worked wonderfully for me. I wasn’t interrupted throughout the day and expectations were managed across recruiters and candidates. I have continued working with these two recruiters (even after they changed agencies) and will do so in the future.

When it came to technical candidates (developers and scientists), the process was a little different. I introduced a test as a filter and only considered the CVs of candidates who passed the test criteria. It made filtering candidates so much easier! I will describe this in the next section.

When you have a new job spec, take time to explain the requirements to your recruiter. When they come back with CVs, tell them why you liked some and why you discarded the ones you did. Good recruiters will understand what you are looking for and will start sending you more relevant CVs.

Another thing I strongly suggest you do is follow up on how the recruiter communicates with candidates. This means

1. Follow up on the job spec document that they send candidates. Recruiters create their own version of the job spec before sending it to candidates. This is done to hide the name of the company to make it hard for the candidate to apply directly without going via a recruiter. Please ensure this document portrays your company and the job spec in the way you intended it to!
2. Follow up with the recruiters on where they post on the internet. Again, the above point applies.

The recruitment process

Based on my previous experience, I had a fairly good handle on what the technical part of the recruitment process was going to be like. I was hiring technical candidates, and it is easy to test their skills through a structured interview. At Affectv, I created the following interview stages which we currently use,

The email test

The first stage in the interview process is a programming test. It is a homework assignment consisting of a few non-trivial problems. The problems are very close to what the candidate would be working on at Affectv. The test is not time bound, and the candidate is encouraged to use different approaches and try to optimise the time complexity.

The solutions are sent to an email alias which executes the code against a set of test inputs and returns a score as a response. If the score exceeds the top x percentile of candidates, then the person is called in for a face-to-face interview.

The email test is the first filter we apply. It helps us weed out the people who are not genuinely interested in the role. We don’t look at CVs. We only look at CVs after they have successfully completed the test. Before coming in for an interview, we ask the candidate to list their top 3 skills. Examples of skills for a data scientist could be PCAs or Logistic regressions. If you are hiring an engineer, examples of skills could be a specific programming language, a class of algorithms or knowledge of a library or architecture.

We ask for the top 3 skills as we want to test the candidate in the areas that they are the strongest in. We ask for 3 skills because we may not necessarily be familiar with the candidates’ #1 skill.

Interviews are stressful. They are even more so when the candidate does not know what to expect. I like companies that tell me what the interview structure will be like, who will be doing the interviewing, what their background is and how many rounds of interviews I will have to go through.

Before an interview at Affectv, we inform the candidates about,

1. Venue. Phone number to call on in case there is a problem.
2. Time and the duration of the interview.
3. Dress code.
4. A bulleted list of the interview stages.
5. Detailed explanation of the current stage:
   • What to expect in the interview?
   • What you will be tested on?
   • Who will be taking the interview?

Interview: Stage 1

Each stage of the interview is a filtering process. For engineers, in stage 1, we test the following areas

1. Coding ability - Can the candidate code in the required language? If the job requires it, do they have an in depth knowledge of the language or certain libraries?
2. Problem solving - Does the candidate,
   • Have the ability to correctly solve the right difficulty of problems?
   • Try different approaches to solving a problem?
   • Persist on a difficult problem, or does he just give up?
3. Previous experience - It’s only after we have vetted #1 and #2 do we get to the CV and ask the candidates questions about their previous experience. Here, we ask questions on what you did and not what we, as a group, did. Some people tend to use “we” more, but after one butts in with, “Sorry, do you mean I or we” a few times, they start using “I” more.
4. Top skills - This is where we test the candidate on their top 3 skills. We go in depth and gauge whether they really know what they claim to know.

At any step, if the candidate fails to impress, then the interview is politely concluded. This ensures that we aren’t wasting the candidate’s or our time.

Interview: Stage 2

Stage 2 of the interview process varies with the role. For junior roles, we skip this stage. For more senior roles, they are given a hard problem that we are facing and are given a week to think about it before they back with the solution. They will then present to a small group (usually 3 people) and there will be a QnA afterwards. Examples of the problems could be

• Architect role - How would you architect this system? What modules do you use and why? What technologies do you use and why?
• Product role - How would you manage the lifecycle of this product? SWOT analysis. Roadmap. Skills needed.

Interview: Competency based

The last stage of the interview process is the competency based interview. I get a non-technical person to do the interview as it gives me another perspective on the candidate.

We have broken down competencies into eight categories and have a scoring system with a pass mark against each category. This pass mark varies depending on the level (junior/mid/senior) of the candidate and helps create a common framework across different departments. Besides, the decisions we make aren’t based on emotional responses but are backed by evidence.

Success

It is only after the candidate is successful in all these stages that we make them an offer. It is a long and arduous process, but then, I wouldn’t want it any other way. After all, products don’t build themselves. People do. And hiring the right people is what makes or breaks a business.

I am proud of what we have built here at Affectv. We have an amazing product. One that may fundamentally change the nature of advertising on the Internet. This is all due to the people here. Smart, dedicated and hard working people who thrive on challenges. Who make difficult problems seem a tad easier. Who make me look forward to each day in anticipation.

And that, is truly amazing.

Final Notes

1. Revert back quickly after an interview. Inform the candidate or the recruiter of the next steps on the day of the interview. Both will really appreciate this.
2. Hire people who are smarter than you. I suppose you have heard of the saying, “A people hire A people, B people hire C people”. You want people who are better than you so that they can take things off your hands and solve problems that you may not be able to solve.
3. Do not go for smarts only. Also look at how well balanced they are. I know smart people are notoriously hard to find, but beware of individual idiosyncrasies. It will affect the rest of your team.
4. Hire for complementary skills. If you have a person who is amazing at something, it makes more sense to hire someone who fills in the shortfalls.
5. Your time is important. Come up with a process to quickly eliminate the wrong candidates.
6. Jacks over Kings. During the initial stages at a startup, generalists are more valuable than specialists.

Cookie Mapping in Advertising

Picture your daily activities on the internet. You search for things, you browse some news articles - perhaps commenting on a few; you watch a few videos - perhaps liking a couple and you post to your favourite social network.

Everything you do online leaves a trail on the websites you visit or on the tools you interact with. This data is invaluable to advertisers who build extremely complex models to understand you and to predict what products or services you would be interested in at any point in time.

However, this data is heavily partitioned into silos. In most cases, the only way of identifying you is by storing an identifier in the browser cookie. This identifier is typically a string of randomly generated characters which is unique to you. Every time you visit a page on that website, your identifier is read from the cookie and the website stores your visit to that url in their logs.

Advertising companies partner with websites to access their log level data. However, there is a caveat. There is no way of telling who user 123 on that website is on the advertisers system. Cookie mapping is a process with which this information can be shared.

If that didn’t make sense

Let me give you an example to illustrate what I just said. Let’s assume all you do in a day is visit your local store and the cinema.

The local store knows a fair bit about you. It has an itemised list of food you buy and how much you spend. Your cinema knows the kind of films you watch and what popcorn and soft drink you enjoy. For an advertiser, this is data that it can use to build a profile on you.

Remember the caveat I mentioned earlier? Neither the store, the cinema or the advertiser know your name. They can’t ask you for your real name because you will be uncomfortable revealing that information and may choose not to shop at their store! So they store stuff about you under a pseudonym. The owner of the store calls you “the cute redhead” and files their copy of the receipt in a folder. The ticketmaster at the movies notes the movies you watch under the name “red shoes” while the advertiser calls you “customer 29”.

Now, although each party has bits of information about you, the information is partitioned. The owner of the store only knows what you have done in their store. The ticketmaster at the movies only knows the kind of movies you watch and the beverages you consumed. While the advertiser has no way of knowing that “customer 29” is the same as “red shoes” and “the cute redhead”. Without this crucial piece of information, all the data from the store and the cinema is worthless to the advertiser!

To solve this, the advertiser comes up with an innovative way. He employs a bunch of errand boys who stand with a notepad at the store and the cinema. Now, every time a new customer is named with a pseudonym, the errand boys look up their notepad and write that pseudonym against their customer id. At the end of the day, when the advertiser picks up the folder from the store owner and the ticketmaster, he can match the pseudonyms against his own customer list using the errand boys’ notepad.

Back to where we were

So coming back, although an advertiser has access to log level data from all the websites he works with, the data is no good without a way of mapping their cookie ids to his own.

The job of the errand boy is done by a process called Cookie mapping or Cookie syncing. This enables the advertiser to track a user across several websites and internet tools.

How

So how does this process actually work? When you vist a website, there is a tiny pixel down on the page called a mapping pixel. For this example assume this is an image pixel of the form http://website.com/1x1.gif. Do note that at the backend, the 1x1.gif is NOT a static image but a script that looks like,

<?php
$user_id = $_COOKIE['user_id']; // Read the user cookie
// Set the user_id in the advertiser url
$advertiser_url = "http://advertiser.com/map?user_id=$user_id";
header("Location: $advertiser_url"); // Mapping call (302 redirect) to the advertiser 
?>

When the browser tries to get the image pixel resource, the php code redirects the browser to the advertisers’ server. A script on the advertisers’ server reads the user_id from the user_id parameter and stores the mapping in a database using a script like,

<?php
// Read the user cookie. This is the cookie id set by the advertiser
// It is different from the user_id above
$user_id = $_COOKIE['user_id']; 
$website_id = $_GET['user_id']; // Get the website's user id from the URL.

// Store the $user_id and $website_id in the database
store_mapping($user_id, $website_id);

header("Location: http://advertiser.com/1x1.gif"); // 302 redirect to a 1x1 image pixel
// Alternatively, just return the 1x1 pixel here.
?>

Now, at the end of the day when the advertiser receives log files from website.com, he can use the mapping table in the database to translate website user_ids into advertisers’ user_ids.

And that folks, is the how and the why of mapping calls.

Building a url shortener in 10 mins

This article is a 10-minute guide to building your own url shortener. A url shortener is a service that lets you shorten a long url ex. http://pravin.paratey.com/posts/building-url-shortener to http://goo.gl/q3jEH. Some url shorteners that you may be familiar with are

1. Goo.gl
2. Bit.ly
3. TinyUrl
4. TinyCC

A shortened url has two parts to it, the domain name (ex. goo.gl) and the url string (ex. q3jEH). In this article we will concentrate on building the second part of the url. Then, we will build a working url shortener in Node.js.

Some mathematics: Set theory

The task is to create an injective, non-surjective function from the domain of full urls to shortened urls. An additional constraint is to minimize the length of the shortened url. As shown in the figure, this class of functions has the properties,

1. An element in the domain (bubble on the left) maps to only one element in the co-domain (bubble on the right).
2. There may be elements in the co-domain that do not map to any elements in the domain.

Note: The domain is the list of urls that we have shortened till date. This is NOT the list of ALL the urls on the internet. The elements in this domain will grow and more and more urls are shortened using our service.

The challenge is coming up with a scheme to ensure that this property is maintained. There are a few hashing algorithms which ensure a one-to-one and onto property – among them are the MD5 and SHA series. Do note that MD5 and SHA-1 have a very tiny probability of collisions.

Take a look at the following python code,

>>> import hashlib
>>> url = 'http://paratey.com'
>>> hashlib.md5(url).hexdigest() # 32 chars
'2325ff901c7e76ddc0d78731c11e8152'
>>> hashlib.sha1(url).hexdigest() # 40 chars
'd01b36a809ab5b254377ed29635d3923026f21cf'
>>> hashlib.sha256(url).hexdigest() # 64 chars
'bd5c0e507f9c1f6477a5ccdb591e22ccfb7b610f746dd8a8a5f963bb5b72401a'

These hashing functions are meant to work on arbitrarily long strings (well, long enough for practical purposes) and therefore map to longer hashes to maintain uniqueness.

A look at the url string

A look at RFC 1738 tells you that the valid characters in a url can be,

lowalpha       = "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" |
                 "i" | "j" | "k" | "l" | "m" | "n" | "o" | "p" |
                 "q" | "r" | "s" | "t" | "u" | "v" | "w" | "x" |
                 "y" | "z"
hialpha        = "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" |
                 "I" | "J" | "K" | "L" | "M" | "N" | "O" | "P" |
                 "Q" | "R" | "S" | "T" | "U" | "V" | "W" | "X" |
                 "Y" | "Z"
digit          = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" |
                 "8" | "9"
safe           = "$" | "-" | "_" | "." | "+"
extra          = "!" | "*" | "'" | "(" | ")" | ","
national       = "{" | "}" | "|" | "\" | "^" | "~" | "[" | "]" | "`"
punctuation    = "<" | ">" | "#" | "%" | <">
reserved       = ";" | "/" | "?" | ":" | "@" | "&" | "="

Which means in addition to the alphanumeric characters, we have 33 characters (safe, extra, national, punctuation, reserved) to work with, making it a total of 95 characters. If we use all of these characters in a url, a 5 character url string will be able to represent 95**5 or over 7.7 billion unique urls. We will be using the alphanumeric characters only in this tutorial as it is easier for most people to work with these characters. Since there are 62 alphanumeric characters, we can shorten 916 million urls with a 5 character string (62**5).

We can spend time coming up with an awesome function. Or, we could cheat.

Cheating

We are going to use a global auto-increment number to reduce the length of the generated string. This means, the first url shortened will have the “short” url as 00001, the second, 00002 and so on.

Implementation

Take a look at the nodejs implementation below. A link to the source code is at the end of the article.

/* Node.js Application */
var http = require('http');
var parse = require('url').parse;

var SERVER = '127.0.0.1';
var PORT = 8080;

var id = 0; /* You can start from a non-zero seed */
var url_to_index = new Array();
var short_to_url = new Array();

/* Randomize CHARS if you dont want people to guess the next url generated */
var CHARS = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHUJKLMNOPQRSTUVWXYZ';

function num_to_base62(n) {
    if(n > 62) {
        return num_to_base62(Math.floor(n / 62)) + CHARS[n % 62];
    } else {
        return CHARS[n];
    }
}

var srv = http.createServer(function(req, res) {
    var param = parse(req.url, true);
    if (param.pathname == '/add') {
        if (param.query.url != undefined) { /* We have a url to add */
            /* Check whether the url has been added before */
            short_url = url_to_index[param.query.url];
            if (short_url == undefined) { /* Nope, not added */
                short_url = num_to_base62(id);
                while (short_url.length < 5) { /* Add padding */
                    short_url = CHARS[0] + short_url;
                }
                url_to_index[param.query.url] = short_url;
                short_to_url[short_url] = param.query.url;
                id++;
            }
            res.writeHead(200, {'Content-Type': 'text/html'});
            /* You may want to drop PORT in production environments */
            short_url_string = 'http://' + SERVER + ':' + PORT +
                               '/' + short_url;
            res.end('Your short url is: <a href="' + short_url_string +
                '">' + short_url_string + '</a>');
        }
    } else { /* Redirect user to the right place */
        long_url = short_to_url[param.pathname.substring(1)];
        if (long_url != undefined) {
            res.writeHead(302, {'Location': long_url});
            res.end();
        } else {
            res.writeHead(404, {'Content-Type': 'text/plain'});
            res.end('404 - Requested url not found');
        }
    }
}).listen(PORT, SERVER);

console.log('Server running at http://' + SERVER + ':' + PORT + '/');

Run this with node urlshortener.js. To add a url, use http://127.0.0.1:8080/add?url=TARGET_URL. For example,

• http://127.0.0.1:8080/add?url=http://www.google.co.uk
• http://127.0.0.1:8080/add?url=http://example.com
• http://127.0.0.1:8080/add?url=http://cto.me.uk

Assumptions

1. An in-memory javascript array is used here. In production, you will want to store the mappings in a database.

Resources

• Source Code