How GitHub Uses GitHub to Be Productive and Secure
Before he came to GitHub, then DevOps consultant Martin Woodward worked to bring legacy apps to the cloud; and he admitted he envied cloud native companies.
“We were always jealous of these born in the cloud startups that had it easy — it’s just cloudy, cloudy, it’s all good,” Woodward, now GitHub’s vice president of developer relations, told audiences at last week’s Microsoft Build conference. “And then I came to work at GitHub.”
That’s when he learned that cloud native companies have their woes as well.
“What nobody tells you about these fast-moving, cloud startups is just dealing with that growth, making sure you don’t build up tech debt as you go — and if you don’t keep modernizing your engineering systems, modernizing your engineering practices, then you’re just going to get swallowed up in your own pool of [tech] debt,” he said.
Woodward was joined by Brian Randell, staff developer advocate in Developer Relations at GitHub, in the cleverly titled presentation, “How GitHub Builds GitHub with GitHub.” The presentation focused on demoing various GitHub tools used internally; but along the way, the audience learned a good deal about GitHub’s development practices and philosophies as well.
Happy Developers Mean Productive Developers
To give the audience a sense of GitHub’s scope, they offered a few data points about what they deal with daily:
- An engineering organization of approximately 1,500;
- More than 100 million accounts on GitHub;
- 80,000 deployments a year to the .com production website;
- 1.9 million commits for internal repositories;
- 1.5 million commits for the GitHub.com codebase, of about 16 gigabytes;
- 4 billion API requests served a day; and
- 66 services with one “massive” Ruby on Rails monolith in the middle.
It’s not surprising, then, that availability is the number one priority for GitHub, followed closely by performance and security, Woodward said.
They then joked about revealing GitHub’s ultimate master plan, which they said was to build happy developers inside and outside the company.
“If you want to unlock productivity amongst your engineering team, you build happy developers, you improve the developer experience, you make it easy to do the right thing,” Woodward said. “Because we are engineers, we route around failure; that’s literally our job. If the process isn’t working for us, then we’re going to just route around it to get our jobs done. So we want to make the process as easy as possible to do the right thing, and have the best developer experience.”
One way in which they strive for a better developer experience internally is to ensure that developers are up and coding by day one at GitHub, Woodward said.
“We set a really ambitious goal that we wanted to have a new engineer, when they arrived at GitHub, have code running in the hands of customers, on the first day in the job,” Woodward said. “And that’s crazy — I did a lot of consulting gigs and I could be there two weeks before [they’d] even give me a machine sometimes.”
“It’s amazing how much money I made sitting in conference rooms,” Randell added.
Improving Developer Productivity with Cloud, AI GitHub Issues
GitHub is a fully remote workforce, so that makes the goal tricky, Woodward added, particularly when they’re dealing with such a large codebase — 45 minutes to clone it over high-speed internet at home, Woodward said — and security concerns about it. To meet its one-day productivity goal, GitHub made Codespaces, a cloud-based development environment that became available to external developers in November 2022.
Also key to GitHub’s productivity: the AI programming assistant Copilot. Woodward demoed how a project leader — even a non-coder — might use Copilot, by leveraging the AI to create inline Ruby on Rails code (a language he does not know well) that adds emojis to the page in the controller. He could then use that code as a proof-of-concept to give to actual Ruby on Rails developers.
Another benefit of Copilot over searching for the answer online is that it’s less distracting than an online search for a coding question, which can lead to cat GIFs and social media, he joked.
“A fully remote workforce, having that AI assistant, having that AI pair programmer, something you can ask questions of — it really helps you work and stay in the flow,” he said. “It helps you be faster as you’re coding. And also within our engineering team, three-quarters of the team say that Copilot, in particular, helps them stay in the flow rather than getting distracted.”
GitHub also uses GitHub, of course, as well as GitHub Issues, which lets developers and others track the data they care about, Randell said.
“Across the company, everybody — this is from dev rel, marketing, and yes, engineering and legal — use Issues to track different types of work,” Randell said. ”That’s one of the key things that we want you to take away from GitHub Issues, is that it’s very flexible and can be used any way you want to manage your work.”
GitHub’s Howie
Image from Microsoft Build presentation, How GitHub Builds GitHub with GitHub
Randell also explained that internally GitHub uses a workflow process it calls Howie, which is short for “How We Work at GitHub.” Howie divides work into a hierarchy of four categories: Initiatives, which take a quarter or longer; Epic, which are tasks that take 3-6 weeks; Batch, which are tasks that take a few days to three days; and Tasks, which take a few hours to a day.
“GitHub wants you to be able to manage your work the way you see fit, without us being overly opinionated,” Randell said. This approach is designed to be flexible and support “Big A agile” to more relaxed practices, he said.
Additionally, GitHub uses GitHub Actions to drill down into the deployment of new code. GitHub actions is a continuous integration and continuous delivery (CI/CD) platform that allows developers to automate their build, test and deployment pipeline.
New GitHub Security Measures
Randell and Woodward also highlighted two new security features in GitHub, starting with push protection, which GitHub announced May 9 to ensure developers aren’t uploading security tokens or other sensitive data. Randell demoed attempting to upload a personal access token on his GitHub and showed how it would reject the code.
“This is something that we made available just earlier this month in May, that allows anyone who has a public repo to make sure that you don’t accidentally submit some kind of secret into your source code and get it pushed out to the world and then have to remediate that problem,” Randell said.
Randell also highlighted a new available tool called Code QL, which basically acts as a security force multiplier, he said. It takes insights found by security teams around the world, who can submit the type of vulnerability, and Github translates that into a special type of query to run against code as part of the pull request process, Randell said. During the pull request, it will detect bad anti-patterns that might be lurking in the code. He demoed how it would deal with a recently found vulnerability.
“When the vulnerability is found, it will fail the pull request, … and we provide you with information that tells you why you shouldn’t do this, and why it’s bad for your application,” Randell said. “This is available to you today, in the open source community, and you can even submit your own as you find vulnerabilities.”
