Taking the Pragmatic Engineer’s Developer Culture Test
What Defines Places Where Developers Thrive?
Here at FloSports, engineering leaders & managers have been deep diving into various topics around building a high performing engineering culture. No, not about weekly catered lunch and tacos, ping pong tables or bring your dog to work-every day (FloSports does provide all of that).
We’re talking about things that truly matter when it comes to engineering productivity and happiness at FloSports — alignment, trust & flexibility, building & shipping & production monitoring, learning and growing. Topics ranging from improving our engineering career laddering docs, engineering-wide standards & technical workflows, decision making process, consistency in quality and documentation, how tech debt is prioritized, are just few of many other aspects that helps us to become “one engineering” at FloSports.
Knowing where we’re at guides us where to go next.
We partner with our People team to send out eNPS surveys at least every other quarter as a way to measure employee satisfaction, retention and engagement. Won’t deep dive into what employee Net Promoter Score is here, but what’s a good score you might ask?
An eNPS can range from -100 to 100. Anything above zero is acceptable, though companies have varying standards. In general, a score of 10–30 is considered good, and a score of 50 is excellent.
Our engineering eNPS score last round was +45. Pretty solid by any standard. As engineering leaders at FloSports, we are proud of our team and have put in a ton of care and work to build a strong engineering culture. Know there is always something that can be improved (heck, even maintaining that level of employee satisfaction takes ongoing work). Nevertheless, we set a goal to achieve +50 next round.
Let’s take the test and do a bit of reflection!
The Developer Culture Test by Gergely Orosz was published on April 13, 2020 — super timely and relevant when the article showed up in my inbox a few days ago.
I’ve talked with dozens of software developers about what they like and dislike about their workplace — team, and company — professionally….I’m surprised to see not much done to quantify these observations in 2020. We have the Joel test that is still attached to the Stack Overflow job listings, …but the Joel Test, written in 2000, feels too outdated and doesn’t address things that are important for productive developers in the 2020s, like autonomy, code reviews, or continuous professional growth. So I took a step back and re-imagined what a test like a Joel-test would be in the 2020s: predictors of a workplace that has a great developer culture.
— Gergely Orosz, The Pragmatic Engineer blog
The test focuses on 3 areas with 5 questions each for a healthy organization, where developers thrive. Gergely states that from his experience, any decent tech company should have the 3 basic points nailed, and cover at least 4 out of the 5 points in each area.
Let’s see how we assess ourselves below! ⬇
Basics
“These are things that all decent companies have in place.”
1. Psychological safety and a blameless culture. ✅
Psychological safety. From 1–1s, code reviews, sprint retrospectives, guilds, architectural and RFCs discussions, to department-wide meetings, we want people to feel safe being themselves and voicing their opinions, even if they might disagree with each other. There’s no right or wrong when it comes to building software, just better or poorer solutions. Disagreements and conflicts will arise, but know that we encourage team members to confidently deliver feedback to each other and with their managers, while also graciously accepting feedback.
Blameless culture. When something goes wrong, we run a blameless postmortem process (aka learning reviews), which focuses on systemic root cause over who caused it. We desire to foster a culture that aims to improve our systems quickly through ownership, openness, and a willingness to share information about outages and technical issues without fear. We’ve done countless blameless learning reviews since October of 2017 and they have always been helpful and well adapted. Perhaps we can incorporate more “pre-mortems” to reduce “post-mortems?”
2. Fair compensation. ✅
We partner with our amazing People Team (thank you Paige, Ashlee and Ryan!) as well as our Senior Leadership Team on ensuring our engineers are paid a fair wage, that’s competitive with the local market. We also have a process to evaluate merit increases as well as market adjustments annually while referencing the salary band we have for each engineering level.
3. Common-sense flexibility on working hours. ✅
100% no butt in seats here. We have Engineers based out of our Austin HQ, Orlando, and various remote locations. Engineers can work in a flexible way that fits their team as well as for them. The focus is on work getting done over what time people start or end the workday.
We have reasonable WFH guidelines such as ideal “core hours” where collaboration can overlap synchronously. We are highly flexible on short-notice schedule changes and personal circumstances. Some engineers work 10am–6pm in the office, some few hours earlier, some will relocate mid-afternoon to beat traffic. On rare occasions, devs opt to stay up late at night to ship something or handle an incident off-hours; they have the flexibility to start the day a bit later or take a day off.
Heck, sometimes you just have to make an appointment to get a haircut, your car doesn’t start, your kids are sick — We understand, life happens!
What’s the common-sense behind this? Just communicate it via the right channels, be respectful of your own time, and the time of others on your team and those you integrate with.
Since Coronavirus (COVID-19), we’ve been able to instantly transition to 100% WFH during this time with little friction as the flexibility has already been a part of our DNA.
Clarity, Autonomy and Collaboration
“Companies that get this right will have innovative and autonomous people thrive. Those who don’t will see these people leave over time.”
1. Understanding the “why”. ✅
One of our core values is “Make Stuff Fans Love: Authenticity.” We encourage engineers to learn more about FloSports’ business while seeking an understanding of how users interact with our product/services. This helps engineers understand how they are contributing to their team, department, and organization-wide success.
Our process to share the business reasons for work to be done with the engineers is kicked off quarterly within the Product-Engineering department. We get company goals from the CEO & Senior Leadership Team, which then cascades to Product department goals and each Product-Engineering team will define their own goals/initiatives and prioritize items on their roadmaps.
Product leadership does Monday kickoffs where we alternate between product-engineering goal updates and company-wide goal updates throughout the quarter. We also encourage product managers to review goals within their team on at least bi-weekly cadence, share how the team’s work is helping the business, etc.
At FloSports we recently launched a book club as part of our “FloGrow” initiatives and kicked off with the book “Start with Why” by Simon Sinek (great Ted Talk if you missed it).
2. Backlog/roadmap and engineers contributing to it. ✅
All teams have bi-weekly or weekly refinement/planning meetings to discuss and prioritize the work to be done towards quarterly goals. We have visible roadmaps and sprint goals. While product managers own the vision for the product and maintain the roadmap, engineers are encouraged to bring suggestions to the table that end up on the team’s roadmap to support the “why”. Sometimes this means identifying and advocating for addressing tech debt (or building technical wealth) that needs to be prioritized alongside sprint work.
Overall, we believe the product-engineering department does a pretty good job here in making it easy to answer “what we will work on next, and why?”
We can improve on tech debt conversations as it is inconsistent between teams today in how it gets discussed and prioritized with product managers. We want to do better in coaching up engineers to bring up tech debt as part of the engineering work estimation.
3. Direct communication in solving problems. ✅
Engineers at FloSports are encouraged to go directly to engineers on other teams to solve problems rather than requiring devs to go through their manager, who talks with another manager.
As an example, one of our codebase has 9 developers across 3 teams (Revenue, Acquisition and Engagement/Retention) that contributes to it even though each team works towards their individual team goals. One way we support direct communication in solving problems is through code-base specific slack channels where devs can ask questions, surface up PRs for code reviews or help unblock each other. We also have hybrid “guilds/chapters” Spotify-like community where problems/ideas are discussed within and cross-teams.
4. Cross-functional collaboration. ✅
Our developers work directly with product managers, designers and other stakeholders without having to rely on a proxy such as their manager. In some cases, a new engineer may not have all the organization context or a developer wants to stay heads-down so that’s where their manager could come in to support & unblock, facilitate some of those conversations, and enhance the collaboration without being a bottleneck.
5. Celebrating people taking initiatives. 🌱
Another FloSports’ core value is “Raise the Bar: Innovation” — as engineers, we know that everything can be at least a little better, the best solutions are often the simple solutions as we gain more experience and that we want to build systems that are reliable and efficient while consuming the least resources to solve a business problem.
Where we can do better as a department is to celebrate and reward those who take initiatives behaviors more often (even the small wins). There are aspects where it could feel like a distractions, especially when it isn’t clear to a product manager or even another developer on the business value.
Sustainable Engineering Culture
“Get this right, and developers won’t burn out or get endlessly frustrated.”
1. Distinguishing between functionally complete vs. ready for production. ✅
We differentiate between prototypes/MVP/functionally complete and production readiness state of code by these definitions:
- Prototypes or POC’s. Often code that does just enough for us to evaluate a solution or to learn something towards making a decision.
- MVP. Typically version one of a feature. It may be functionally complete but only a sub-set of the fully desired set of features which may come in a later phase.
- Functionally complete: AKA “dev complete” is code that has all the planned end user features/requirements, but may not be ready to ship due to bugs or performance issues.
- Production-ready. Code meets functional requirements, performance requirements, etc.
We are working on a more formal checklist for what additional quality bar means for something to be production-ready that could be shared among teams consistently.
2. Code reviews and testing. ✅
Automated testing and code reviews is an essential process for software quality and is a part of our everyday software development.
We do reviews using Github. A common workflow is developers would put up a PR and request reviews in the appropriate slack channels.
For automated REST API testing and JS end-to-end, we’ve standardized across most teams by using our in-house built tool Flagpole. Some projects also use Cypress.io, Runscope, and Ghost Inspector. Unit testing tools vary across backend services, frontend web, and native iOS/Android projects.
On top of code reviews and unit/integration/UI tests, we are also building out a load/performance regression testing using open source k6.io.
3. CI and CD or developers deploying to production. ✅
At FloSports we’ve been doing CI many years back but began our CD (continuous delivery) journey in mid-2018 (Embrace the Continuous Delivery Culture by Doing Things that Don’t Scale). We were able to transition a few of our largest projects (with the highest # of developers) to a “developer-release” process where devs push code they own to production after CI is complete (currently use Jenkins, CircleCI, Google Cloud Build across different projects).
Some projects have “review apps” support where developers pushing up a PR will get an automatically generated ad-hoc environment, and a merge into master will auto-deploy to staging while leaving it up to a “business decision” for production deploy.
It’s really beautiful. That said, we do have a few more projects to transition the culture+process+tools towards supporting full CI/CD/Developer Releases.
4. Healthy oncall. ✅
We have a process & tools in place today to support both on-call and escalations when there is a critical issue that was mostly formalized in December 2019. Each team manages their own on-call schedule that works for their team. For each critical issue, we make our best attempt to fix the issue so that the same issues do not happen repeatedly causing on-call responder burnout.
We use PagerDuty to manage services, schedules, and escalation policies as well as integration into Slack for human triggers. DataDog for automated monitoring alerts.
5. Internal open source. 🌱
We haven’t fully adopted an internal open-source model where any developer can read and contribute to any other codebase. All devs do have access, it’s just that we have not been intentional nor consistent in supporting this model.
A recent example pointing us in this direction: Our infra team initially built a Kubernetes based job scheduler service. An engineer on another team (let’s just call him Harold) needed to enhance the service for a particular use case. Our infra team as current owners of the code could do the work, but the team’s plate was already really full, so Samir our Infra team lead decided to let Harold build it, discussed architecture together and pushed up a PR. Harold was able to unblock himself to deliver value, plus we now have additional knowledge shared in this service/codebase.
The question for engineering is do we want to encourage this? What is the business value in enabling this model? I believe we do.
But we’ll need to have appropriate code ownership in place, have more standards & consistency that matters between team/projects within reason.
Career Progression
“People who are driven want to keep growing throughout their career. Does your company have a track for senior developers growing outside of moving into management, and is professional growth supported in a variety of ways?”
1. Technical managers who build trust. ✅
Technical managers. All of our current development managers are technical, have done software development at some point in their career (or still does as “Player/Coach” managers) and have been at least Sr. Software Engineers themselves. While it is totally possible for non-technical managers to be great managers, having hands-on experience means they can better relate to the development struggles, be more effective advocates for sustainable engineering, and more capable of mentoring other software engineers.
Regular 1:1s. All engineering managers do regular 1:1s with their team members, either weekly or bi-weekly cadence. While every 1-on-1 is different and can have various “stages”, this is a set time for managers and their team members to build rapport & trust, give and receive feedback, discuss career development & goals, discuss ways to improve the team and company and general happiness check-ins.
2. Career ladder. ✅
We created our first FloSports engineering career ladder & growth framework back in August 2018 which included 5-levels (SE1, SE2, Senior, Staff, and Principal), scope of impact, and expected behaviors at each level. Since then, we’ve used it as a tool for engineers and their managers to discuss growth as well as evaluation for promotions. Engineering managers are now getting together as of this writing to improve on the framework with the goal of creating more clarity into the promotion criteria and process.
3. Parallel IC & manager tracks. ✅
While we have not yet formally published an engineering manager (people track) career ladder, we do have parallel IC & management career paths. Mainly, engineers do not have to become managers to grow professionally or to increase their scope of impact and compensation at FloSports. In other words, Senior/Staff/Principal IC Engineering levels have parallels to Manager/Sr. Manager/Director/VP level in terms of leadership, influence, and compensation.
4. A culture of feedback. ✅
We partner with our People team on performance reviews which includes 3 types of reviews:
- 360 reviews where peers give feedback to each other
- 360 where directs give feedback to managers
- Manager reviews of employees
From a company-wide perspective, our People team does employee-based NPS surveys at least twice a year, from which the feedback collected is shared with and acted on by the company and managers.
5. Investing in professional growth. ✅
Mentorship. In Q4 of 2019, we hired 7 additional software engineers across 6 teams (iOS, Revenue, Acquisition, Sports Data, QA, and Infrastructure). While we don’t have a formal mentorship program in place, we’ve supported both on-boarding specific type mentoring as well as informal mentorship that generally happens everywhere. When a new engineer joins the team, they typically pair up with the Dev Lead on the team to help them understand the new environment, assist with local development setup, ship their first line of code to production, goes over systems, designs, and process, etc.
The more natural mentorship happens when engineers work and collaborate together. This happens every day through code reviews, planning meetings, architecture and whiteboarding discussions, retros, etc. Learning happens naturally when devs are working on projects together.
Professional development stipend for books/trainings. Engineers can expense books or online training courses (such as Udemy) with a manager’s approval. We’ve also worked with our partners such as Amazon and Google to set up workshops.
Regular tech talks. We have various regular meetings where Engineers can learn from each other through sharing.
- Tech demos. Engineers take 3–7 minutes to show-off what they or their team have built/shipped. Opportunity for engineers to work on their presenting skills as well as learn what other teams are working on.
- Guild meetings. Engineers across the department can present and discuss all tech topics. Think of it as a community of members with shared interests sharing knowledge. On some occasions, we’ve blended these meetings with more “Chapter” like discussions that focus on a particular team/code base’s technical decisions and request for comments.
- 2–3 day long hackathons. Valuable for teams to learn something new either from customer/product perspective or technical perspective. Typically end with presentations by all teams as well as prizes for top winners.
Final thoughts and what’s next?
Based on my experience, this engineering culture test list is spot-on as predictors of a workplace that has a great developer culture for the 2020s. Each area is one that FloSports Engineering has intentionally worked on over the last few years and actively being improved.
In summary:
- Basics (safety, fair compensation & flexibility)
— 3/3 ✅ ✅ ✅ - Clarity, Autonomy, and Collaboration
— 4/5 ✅ ✅ ✅ ✅ 🌱 - Sustainable Engineering Culture
— 4/5 ✅ ✅ ✅ ✅ 🌱 - Career Progression
— 5/5 ✅ ✅ ✅ ✅ ✅
I’d say FloSports engineering is a pretty decent place to work. Yea, I know this is a self-assessment from an engineering leadership perspective, but our most recent eNPS score of +45 backs it up!
Where do we go from here? Continue building teams and building software of course! Let’s discuss, learn, iterate, and improve upon FloSports engineering culture.
What questions do you have for us? Do you think the test criteria is spot on?
Where does your organization stand? Take the test and share with us your results!
Big thanks to our engineering leaders & managers at FloSports: Jason, Kaleb, Charlie, Karl, Steve, Kodiak, and to all of our engineers for being an important part of making FloSports an amazing place to deliver value to our fans, ship code, learn and grow together.
Also thanks to Nick Schirmer, Jason Byrne and Kaleb Fulgham for fixing my spelling, grammar and passionate mistakes 😃
